Researchers from Brigham Young University (BYU) and 186 other universities have recently published an article in the journal Issues Accounting Education reporting the results of a study that compared the performance of OpenAI’s ChatGPT to that of actual accounting students. Their findings showed that university students were able to perform better on their accounting exam than the ChatGPT.
ChatGPT, which is OpenAI’s chatbot product, utilizes a machine learning system to generate natural language text. During the study, students were able to achieve an overall score of 76.7%, while ChatGPT scored 47.4%. The AI bot’s results were lower in tax, finance, and managerial assessments, while 11.3% of questions saw the bot’s scores outperforming the student averages, especially in accounting information systems (AIS) and auditing. Additionally, the bot was found to be more successful in true/false and multiple-choice questions, but less successful in short-answer questions.
Lead researcher David Wood, a BYU professor of accounting, explained that ChatGPT also struggled with higher-order questions, albeit in some cases providing highly knowledgeable written explanations for incorrect answers, or (sometimes) providing different answers to the same questions. Interestingly enough, the bot was found to invent its own references, and make various mathematical errors.
David Wood and his co-authors have gone on to discuss the implications of this study, expressing their belief that OpenAI’s technology is impressive and may be a game changer when it comes to teach-and-learning. Their research was supported by 327 co-authors from an international collective of 186 educational institutions, thus closely examining 25,181 classroom accounting exam questions and 2,268 textbook-based questions.
OpenAI is a research laboratory in San Francisco and is co-founded by world-renowned scientists and technologists such as Elon Musk, Sam Altman, Terah Lyons and Greg Brockmon. Located in the city of San Francisco, the company conducts research on artificial intelligence with the goal of guiding AI to be broadly accessible, and to benefit humanity. Founded in 2015, the company is recognised for its strong emphasis on ethical and safe research, something that their performance in the study further reinforces.
Lead researcher David Wood is a professor of accounting at Brigham Young University and an esteemed researcher in accounting technologies and systems. His research focuses on gamifying course materials to make learning more fun, interactive and effective. His belief that AI might have a positive effect on the teaching and learning process is evident in the study’s results and the public commentaries that follow it.
