Study Shows ChatGPT Performs Similarly or Better Than University Students in 9 Out of 32 Courses, Despite Plagiarism Concerns
A recently published study in Scientific Reports has revealed that ChatGPT, OpenAI’s chatbot powered by artificial intelligence (AI), has performed comparably to, or even surpassed, university students in answering examination questions across various disciplines. The findings come despite concerns among educators that using ChatGPT amounts to plagiarism. The study conducted by Talal Rahwan and Yasir Zaki involved faculty members from New York University Abu Dhabi (NYUAD) and surveyed students and educators from five countries.
In the study, faculty members from NYUAD were asked to provide three student submissions each for ten assessment questions across different courses. ChatGPT was then tasked with generating three sets of answers for the same questions. Assessments were conducted by three graders who were unaware of the source of the answers. Surprisingly, ChatGPT outperformed students in 9 out of 32 courses, including disciplines such as computer science, politics, engineering, and psychology. Only mathematics and economics consistently saw students outperforming ChatGPT. Notably, ChatGPT excelled in the ‘Introduction to Public Policy’ course, receiving an average grade of 9.56 compared to 4.39 for students.
The study also included a survey of 1,601 individuals from Brazil, India, Japan, the US, and the UK, with at least 200 students and 100 educators from each country. A significant finding was that 74 percent of students expressed their willingness to use ChatGPT for assistance with their assignments. However, educators consistently underestimated the proportion of students open to using AI tools, with 70 percent of educators considering it plagiarism.
Furthermore, the study explored the detectability of AI-generated text. Two tools, GPTZero and AI text classifier, were tested. Results showed that the ChatGPT-generated answers were misclassified as being written by a human 32 percent of the time in the case of GPTZero and 49 percent of the time for the AI text classifier.
The implications of these findings are considerable and could impact policies regarding the use of AI tools in educational settings. While some educators view the use of ChatGPT as plagiaristic, students are embracing its potential for assistance and learning. Striking a balance between harnessing the benefits of AI and addressing plagiarism concerns will be crucial in shaping future academic practices.
Overall, the study underscores the significance of ChatGPT’s capabilities in answering assessment questions, with its performance matching or surpassing university students in a range of courses. It highlights the need for educators and policymakers to adapt to the evolving role of AI in education and consider its potential benefits alongside concerns related to academic integrity.
