GPT-3 AI Matches College Students in Reasoning Tests, Raises Questions About Human-like Thinking
A recent study conducted by researchers from the University of California – Los Angeles (UCLA) has discovered that GPT-3, an advanced language model powered by artificial intelligence, exhibits reasoning capabilities on par with college undergraduate students. This finding raises intriguing questions about the extent to which AI can emulate human-like thinking.
The scientists tasked GPT-3 with solving complex reasoning problems that are commonly found in intelligence and standardized tests like the SAT. These tests are instrumental in determining college and university admissions decisions in the United States and other countries.
In one test, GPT-3 was presented with a series of shapes and was asked to predict the subsequent shape in the sequence. Astonishingly, the AI model correctly solved 80% of the problems. This score was comparable to the average performance of the 40 undergraduate students from UCLA who tackled the same challenges, just below 60%.
UCLA psychology professor Hongjing Lu, the senior author of the published study in the journal Nature Human Behaviour, noted that the AI not only performed at a similar level to humans but also made similar mistakes. This raises intriguing questions about the underlying mechanisms of AI’s reasoning abilities.
When it came to SAT analogy questions, GPT-3 outperformed the average score of the students. Analogical reasoning involves solving unfamiliar problems by drawing parallels to familiar ones. Test-takers were presented with pairs of words sharing the same relationship and had to select the analogous pair. For instance, when faced with the analogy ‘Love’ is to ‘hate’ as ‘rich’ is to which word?, the correct response would be poor.
However, GPT-3 struggled slightly with analogies based on short stories compared to the students. These problems tested the AI’s ability to read a passage and identify a different story conveying the same meaning.
The researchers involved in the study acknowledge that they are unsure about the inner workings of GPT-3’s reasoning abilities, as the model’s creator, OpenAI, has not publicly disclosed its details. The team hopes to conduct further investigations to gain a deeper understanding of whether AI models like GPT-3 truly think like humans or if they employ entirely different mechanisms that simulate human thought processes.
UCLA psychology professor Keith Holyoak, a co-author of the study, emphasized that the training methods for GPT-3 differ significantly from human learning, with the AI not having the same learning experiences as individuals. The researchers are eager to determine whether the model’s reasoning aligns with human thinking or if it represents a groundbreaking, independent form of artificial intelligence.
As AI models continue to advance rapidly, such studies contribute to our understanding of these cutting-edge technologies. Unveiling the level of reasoning AI can achieve not only sheds light on its capabilities but also opens the door to exploring new possibilities for human-like thought within machines.
