Researchers from Brigham Young University (BYU) and 186 other educational institutions conducted a study to test OpenAI’s ChatGPT on accounting exams. The results of the study showed that university accounting students performed better in the exams than chatbot product ChatGPT. Despite this, the researchers still called ChatGPT’s impressive performance a game changer that will transform the way teaching and learning is done more efficiently in the future.
In the accounting exam, students scored an average of 76.7 percent, while ChatGPT showed only 47.4 percent. While ChatGPT did well on accounting information systems and auditing, it had difficulty with tax, financial, and managerial topics. This could be due to the difficulty of mathematical processes in these areas. ChatGPT also had difficulty answering higher order questions.
Overall, ChatGPT did better on true and false questions and multiple choice questions, but struggled with short answer questions. Another issue the researchers highlighted was ChatGPT’s inclination for making up facts in references; it would generate real-looking references entirely from its own data, and the work and authors would not exist.
David Wood, a professor of accounting from BYU and lead study author, wanted to ensure the debate over this technology was discussed thoroughly, so he encouraged co-authors from 186 universities. 327 co-authors were eventually recruited and they provided 25,181 classroom accounting exam questions, along with 2,268 questions from test banks.
OpenAI is an artificial intelligence research laboratory, founded in December 2015 with the goal to advance digital intelligence in the way that is most likely to help humanity. It builds upon research from multiple disciplines, including deep learning, reinforcement learning, natural language processing and computer vision, with a focus on creating larger AI systems. Additionally, their focus is set to developing AI that has a positive impact on humanity.
David Wood is a professor of accounting at Brigham Young University. He is the lead study author of the research conducted to test OpenAI’s ChatGPT on accounting exams. He is known for his work in the social media industry, specializing in marketing and advertising. His research interests include accounting technology, accounting information systems, and teaching and learning technologies.
