An economics professor is astounded at the progress made by ChatGPT. After only three months, it improved its grade from a D to an A on a mid-term economics test set by Bryan Caplan of George Mason University.
In his Substack blog post, “Bet On It,” Caplan described his exam questions as an assessment of students’ understanding of economics rather than a memory exercise. Unfortunately, the previous version of ChatGPT failed to understand basic concepts such as comparative and absolute advantage and gave mainly political rather than economic answers, resulting in a score of 31/100 – equivalent to a D grade.
The new version of ChatGPT, GPT-4, achieved an A grade in Caplan’s exam by besting even his 50% median score, thus debunking a lot of the criticisms brought up against the previous version. It correctly answered questions regarding complex economic principles and fairly evaluated the works of economists like Paul Krugman.
Besides Caplan’s midterms, ChatGPT also passed a test from the Wharton Business School, although it made a couple of surprising mistakes on simple calculations – nonetheless, a lot of progress has already been made.
Familiar with making bets, Caplan bet an AI model wouldn’t score an A on 6 out of 7 of his exams before 2029, and he was not expecting to lose this bet anytime soon. Nonetheless, he was amazed by the progress made and is already feeding the bot tests it hasn’t yet seen before. He said that this time it’s doing even better than the 73% achieved on its maiden mid-term.
Yudkowsky, an AI doomer, then challenged him to a bet that AI would lead to the end of the world before January 1, 2030 – Caplan is now more confident he’ll win this bet.
However, AI bots are proving to be a headache for examiners. Exams may be plagiarized using material from ChatGPT, and it can be difficult to prove this as there is no physical evidence of wrongdoing. Caplan is considering getting rid of graded homework and hopes that his habit of consistently changing questions will be enough to deter students from learning and submitting the answers of ChatGPT.
ChatGPT is an AI-powered conversational assistant that unifies powerful Natural Language Processing (NLP) technologies with heuristics and large datasets to help both businesses and individuals automate mundane tasks and gain insights from their data. ChatGPT was created in 2020 by Sam Altman and strives to bring AI to the public by making everyday life easier and more productive.
