ChatGPT found to prefer repeating 25 jokes repeatedly, according to researchers

German researchers have published a paper on OpenAI’s ChatGPT-3.5, a language model that generates humorous responses. The research revealed that ChatGPT’s ability to generate jokes is considerably limited. During testing, 90% of 1,008 responses were identical and contained one of 25 jokes, suggesting that ChatGPT memorized these jokes during its training, rather than generating them anew. Research was conducted without access to the model’s inner workings or dataset. When prompted to generate a joke, ChatGPT predominantly returned the same jokes, including the tomato joke. The researchers found that the humor detection and explanation were heavily influenced by surface characteristics like structure or wordplay, which show understanding of humor elements. While ChatGPT was not always capable of generating new jokes, the researchers concluded that their work illustrates progress toward a general understanding of humor in language models.