Challenges of Humor for Large Language Models Highlighted in Study about ChatGPT’s Jokes

Researchers from German Aerospace Center and Technical University Darmstadt have discovered that despite creating a few dad jokes, large language models, like ChatGPT, are not particularly original and usually rely on a limited number of frequently recycled quips. In a study of 1,008 trials, ChatGPT responded with one of 25 different jokes for over 90% of the time. The top four jokes were recycled in more than half of the responses. The ChatGPT-3 based model was found to have limited versatility in its response pattern and could not confidently create intentionally funny original content. Despite displaying an understanding of wordplay and double meanings, ChatGPT struggled to identify puns that did not fit into its learned pattern. The researchers can’t confirm whether jokes were hard-coded, leading them to conclude that computational humor is still a challenge for large language models. However, the authors said ChatGPT was a big leap toward funny machines.