Move over comedians, there’s a new stand-up act in town. A recently released study from the University of Southern California found that AI-generated jokes outperformed those crafted by humans. Nearly 70% of the participants rated ChatGPT jokes as funnier than those written by regular people. By comparison, 25% favored the human jokes and 5% rated the jokes as equally funny.
While there’s evidence out there for how language models perform on analytical tasks, less is known about their creative side, said Drew Gorenz, a doctoral candidate in the psychology program at USC and one of the study’s researchers. As a comedy enthusiast himself, Gorenz was curious how ChatGPT would stack up to human comedians.
“They don’t know what it feels like to appreciate a good joke,” he said of language models. “They’re mostly just using pattern recognition.” The results, he added, “tell us a lot of cool things about humor production that perhaps we don’t need to feel emotions involved in a good joke to tell a good one.”
To conduct the study, both ChatGPT and humans were asked to write jokes based on a variety of prompts. One task involved coming up with funny acronyms for a string of letters. Another was a fill-in-the-blank type prompt based on the party game Quiplash, and the third involved writing a humorous way to describe an unpleasant situation. A separate group then rated the results. For example: When asked to complete the blank for “A lesser talked about room in the White House: ‘__________,'” humans came up with “The White Padded Room” and “The dog house,” while ChatGPT spun up “Lincoln Bedroom’s Alien Conspiracy Corner” and “The Situation Room’s Snack Closet.”
One important thing to note, Gorenz said, is that stand-up comedy jokes are a lot less funny when you see them in the text-only format. “Delivery is such a key part of humor production,” he said. In a second study, researchers measured how ChatGPT jokes fared compared to those crafted by professional comedy writers by asking the AI chatbot to rewrite headlines from the satirical site The Onion, “America’s Finest News Source.”
Here the human writers fared a bit better: the average humor rating was the same for the Onion headlines and those generated by ChatGPT, said Gorenz. ChatGPT came up with the top-rated headline “Local Man Discovers New Emotion, Still Can’t Describe It Properly.” In second place was one from The Onion: “Man Locks Down Marriage Proposal Just As Hair Loss Becomes Noticeable.” The USC study comes at a time when the entertainment professionals — comedians included — are fretting over how AI could reshape their jobs.
In January, the estate of George Carlin filed a lawsuit against a media company, alleging it used artificial intelligence to recreate the late standup comic’s style and material. As far as Gorenz is concerned, the results of the study indicate that ChatGPT could disproportionately disrupt comedy and entertainment, especially given that the bar for accuracy in those industries might be lower when compared to say science, education, and journalism.
Still, he doesn’t think America’s favorite stand-up comedians are going anywhere anytime soon. “I don’t think it’s able to create a John Mulaney level joke,” he said.
A study comparing jokes by people versus those told by ChatGPT shows that humans need to work on their material. The research team behind the study published July 3 in the journal PLOS ONE took on the serious task of comparing participants’ reactions to jokes written by ChatGPT 3.5 and others written by people. This was conducted in an effort to determine if artificial intelligence can outwit humans for a laugh, said Drew Gorenz, a doctoral candidate in social psychology at the USC Dornsife College of Letters, Arts and Sciences.
“Since ChatGPT can’t feel emotions itself but it tells novel jokes better than the average human, these studies provide evidence that you don’t need to feel the emotions of appreciating a good joke to tell a really good one yourself,” Gorenz said.
One study compared the funniness of novel jokes generated by regular people with the funniness of novel jokes generated by ChatGPT. Participants rated the funniness of all jokes without knowing their author. The researchers found that overall, nearly 70% of the participants rated the ChatGPT-generated jokes as funnier than those crafted by humans. A little over 25% of participants rated the human-crafted responses as funnier, and only about 5% rated jokes from both sources as equally funny. The results were consistent even when the researchers checked for differences across various demographics.
To see how ChatGPT might fare against professional humor writers, the researchers conducted a second study in which they challenged ChatGPT to develop new headlines in the satirical style of The Onion. They then asked another group of 200 participants to rate the funniness of original Onion headlines and of the new ChatGPT-generated headlines. Participants found ChatGPT’s headlines just as funny as the original Onion headlines.
AI jokes: First study
How does one get people and AI to generate original jokes? For the first study, both ChatGPT and 105 participants completed three tasks, each including three prompts.
In the first task, they crafted humorous new phrases for common acronyms. In a second task, both ChatGPT and the human writers generated funny answers to fill-in-the-blank phrases such as, “A lesser known room in the White House: _____.” They were also prompted to come up with a “roast joke,” a funny slam inspired by an awkward fictional scenario such as this:
“Imagine that one of your friends wants your opinion on how well she sings. She sings a minute or two to demonstrate her voice, and you cringe — she might be the worst singer you ever heard. When she asks, ‘So how was it?’ you decide to be honest so you say, ‘To be honest, listening to that was like: _____’”
These three tasks resulted in more than 945 jokes written by 105 writers. ChatGPT was asked to generate 20 humorous answers for each prompt for a total of 180 jokes. A new group of participants then rated the funniness of the jokes.
Investigating ChatGPT and humor: Second study
In the second study, ChatGPT was fed original headlines from The Onion and asked to generate new headlines in the same style. A group of 200 participants was asked to rate the funniness of the original Onion headlines and the ChatGPT-generated ones without knowing the authorship.
Gorenz said his idea for the study stemmed in part from the entertainment world’s debate over writers’ concerns about the use of large language models (LLMs) like ChatGPT in entertainment production. The issue has been magnified by recent Hollywood strikes by writers and actors who are fearful that the adoption of large language models poses an existential threat to their professions, to art and to human creativity.
Gorenz, himself an amateur stand-up comedian, acknowledged a lifelong love for comedy that inspired his research with co-author, Norbert Schwarz, who is a co-director of the USC Dornsife Mind and Society Center and an expert on consumer judgment with the USC Marshall School of Business.
Gorenz said their study raises concerns about the use of ChatGPT and other LLMs within the entertainment industry.
“The implications are more positive for people who merely want to reap the benefits of elevating their everyday communications with a dose of humor,” he and Schwarz, a Provost Professor of Psychology and Marketing, wrote.
“But for professional comedy writers, our results suggest that LLMs can pose a serious employment threat,” Schwarz noted.
Whether AI gets the last laugh remains to be seen.
Source: CBS News, USC News
