Bryan Caplan, an economics professor at George Mason University, recently revealed that he was stunned by the progress made by the chatbot ChatGPT in just three months. In January, Caplan required it to answer questions from his Fall mid-term tests, but it failed to earn anything above a D grade. In its current version, ChatGPT managed to outwit Caplan and other academics by improving its score to an A.
Caplan outlined in his blog that the questions used in his test are aimed at testing the student’s understanding of economics, not simply memorizing the answers. Unfortunately, the first iteration of ChatGPT failed to understand key concepts in economics, such as absolute and comparative advantages.
In addition, it was discovered that ChatGPT-4, the new version of the bot, was updated to be 40% more likely to return accurate responses. Caplan became so confident that he even made a bet with AI doomer Eliezer Yudkowsky that AI won’t lead to the end of the world before 2030.
However, ChatGPT’s improvement has caused a problem for some professors. ChatGPT’s responses can be plagiarised by students and it can be difficult to prove wrong-doing as there is no material evidence to back it up.
In an effort to curb potential issues, Caplan is considering no longer grading homework and has been regularly changing the questions to prevent students from learning the chatbot’s answers.
ChatGPT was created by Sam Altman, whose ultimate goal was to prove that AI can outperform humans in a variety of test scenarios. It appears that Altman has achieved his goal, as the chatbot is able to answer questions more accurately than the average student. It is also capable of analysing nuanced instructions and processing complex calculations.
ChatGPT has shown that AI can have a positive impact on education and is an exciting development for educators and students alike. It has certainly caused Professor Caplan to rethink his prejudices toward AI and its ability to outperform his students.
