A recent study published in the journal Issues in Accounting Education reveals that students are performing better than the AI chatbot product ChatGPT in accounting exams. BYU professor David Wood and his team of 327 co-authors from 186 universities in 14 countries ran 25,181 classroom accounting exam questions and 2,268 test bank questions from a textbook by the AI to analyze its performance.
The findings of the study showed that the students scored an average of 76.7%, compared to ChatGPT’s score of 47.4%. The AI proved to be competent in true/false questions, scoring 68.7%, and multiple choice questions with a score of 59.5%. However, it was found to have struggled in short-answer questions, with its accuracy between 28.7 and 39.1%. On the higher-order questions, the bot wasn’t as successful, with the researchers noting that ChatGPT often provided authoritative written descriptions with incorrect answers. On tax, financial and managerial assessments, ChatGPT was also found to perform worse.
OpenAI, the US-based organisation behind ChatGPT, is considered a leader in innovation and development of artificial intelligence, machine learning and natural language processing, and this current analysis of the chatbot’s performance was deemed “impressive” by Professor Wood. It is believed that OpenAI’s technology is set to change the way everyone learns and teaches.
However, the researchers also noted that ChatGPT often made mistakes, including falsely generating reference content and making mathematical errors. Additionally, there is an intense on-going debate about the implications of AI products such as ChatGPT and how they might be utilised in the education field.
BYU professor and lead study author David Wood is driven to find out how ChatGPT will fare in the examination system and have recruited many professors to join him in his research. The enormous turnout from 186 universities in 14 countries was a reflection of the public’s interest and the importance of this study.
OpenAI is a San Francisco-based company which focuses on research and development of artificial intelligence, as well as providing software to support this technology. Founded in 2015 by tech billionaires, Elon Musk and Sam Altman, OpenAI has raised over $1 billion in investments and has grown to become a global leader in developing AI-based programmes. The organisation is set on pushing the boundaries of natural language processing, reinforcement learning and robotics. With innovative projects like ChatGPT, OpenAI is paving the way for a future of machine learning-based technology which could potentially shape the way we access and share information.
