OpenAI recently launched its new AI chatbot product, GPT-4, which uses machine learning to generate natural language text. Due to its impressive capabilities, including passing the bar exam, 13 out of 15 AP exams, and even getting a near perfect score on the GRE Verbal test, researchers were interested to see how this tech would fare in accounting exams. To gain insight into this, Brigham Young University (BYU) and 186 other universities decided to put the original version, ChatGPT, to the test. The results of this research were published in Issues in Accounting Education.
The researchers found that first version of ChatGPT was still no match for humans when it came to accounting. Students scored an overall average of 76.7% on their exams, while ChatGPT scored 47.4%. Although the AI bot did slightly better on true/false and multiple choice questions, it had difficulty understanding higher-order questions, such as those posed in short-answer questions, and could often provide incorrect answers.
While it is clear that ChatGPT is still substandard when it comes to accounting tests, this should not discourage people from using it in education. In fact, the researchers acknowledge its potential to be a gaming changer and to improve teaching and learning. Apart from design and testing applications, it can also help with drafting portions of a project, raising questions about the value of existing curriculums.
OpenAI is an American artificial intelligence (AI) research laboratory, specializing in deep learning, founded in December 2015 by Elon Musk, Sam Altman, and others. The company is involved in the development of machine learning and artificial general intelligence, with a focus on making the technology available to everyone. OpenAI is backed by tech companies, including Microsoft and Qualcomm, and is largely funded by venture capital firms.
David Wood, lead study author of the research mentioned in this article, is a Brigham Young University professor of accounting. With 327 co-authors from 186 educational institutions in 14 countries, David managed to put together a team working on 25,181 classroom accounting exam questions. He also managed to recruit 2,268 BYU students, including his own daughter Jessica, to provide questions from the textbook test bank.
