A study conducted by hundreds of higher education institutions sought out to determine if an AI chatbot could fare better than university students in accounting exams. Unfortunately, the AI chatbot, ChatGPT failed to pass the exams, scoring an average of 47.4%, in comparison to the 76.7% average score scored by the students.
The research was led by Brigham Young University (BYU) professor David Wood and published in the journal Issues in Accounting Education, with 327 co-authors from over 180 institutions in 14 different countries collaborating in the study. Wood had recruited the large number of academics together through social media.
25 thousand exam questions were contained in the study, which tested the students on topics including accounting information systems, auditing, tax and financial accounting, in different formats such as true/false and multiple choice. It was found that the AI struggled to comprehend questions that required mathematical processes such as addition and subtraction and failed to answer questions correctly frequently, sometimes even providing authoritative but incorrect answers in order to defend its responses.
However, ChatGPT did manage to fare better in true/false and multiple choice questions, scoring 68.7% and 59.5% respectively. Wood expressed his fears that the technology increases the chances of students cheating in exams, and instead opts to focus on utilising the technology to improve the teaching process for faculty and the learning process for students.
ChatGPT, a creation of the research lab OpenAI, is a chatbot developed with AI technology, and is funded by Elon Musk, Microsoft and Amazon Web Services, among others. The chatbot is developed to have human-like conversation by using natural language technology.
David Wood, the lead author of the study, is an associate professor at Brigham Young University. Wood holds a Ph. D. in Accounting from the University of Waterloo and is an expert in AI for accounting and auditing applications. He has published numerous papers on these topics and is passionate about exploring the intersection between the two. Wood is a Co-President of BYU’s AI Club and is dedicated to helping students develop their AI knowledge, both professionally and academically.
