ChatGPT, the most advanced Artificial Intelligence (AI) language model to date, has been stirring up reservations about its potential to help students cheat on academic tests. From tackling the GRE to passing numerous AP exams, the AI platform had been gaining ground in the artificial intelligence learning space – until a recent study revealed its shortcomings. Led by Brigham Young University professor David Wood, this examination questioned ChatGPT’s capacity to solve mathematical tasks as it was compared to actual accounting students.
The study gave over 25,000 questions across accounting disciplines, including financial accounting, information systems, taxes, auditing, and managerial accounting, together with 2,268 test bank questions to accounting undergraduates at BYU. Questions were presented in a mixture of formats, such as multiple choice, true/false, and written response, that featured different levels of difficulty. The results portrayed a vast gap between ChatGPT’s abilities and the real-life accounting students, with the bot scoring an average of 47.4% in comparison to the students’ 76.7%.
On the 11.3% of questions where ChatGPT did outperform the students, the AI language model scored better on topics of auditing and accounting information systems as well as on multiple-choice and true/false questions. Nevertheless, the chatbot’s performance on the written response prompts was wretched – ranging from 28.7% to 39.1%. Professor Wood concluded that although AI has great potential in assisting students learn, it cannot replace their efforts in their accounting studies.
Upon closer examination, ChatGPT is a development by OpenAI, the artificial intelligence research corporation. Established with the main idea of creating Artificial General Intelligence (AGI) – a technology that would be on par with human intelligence – OpenAI started off as non-profit research organization, founded by tech leaders such as Elon Musk, although they eventually moved on to become a for-profit venture in February of last year.
David Wood is an accounting professor at Brigham Young University. He is an advocate of innovation in teaching, striving to use new technologies to aid students and instructors. His experiments with ChatGPT, albeit humbling to AI fans, bring to light an issue that has been previously overlooked – the importance of understanding mathematical operations in order for AI platforms to become truly capable of replacing human labor.
