Scientists Achieve Over 99% ChatGPT Identification Accuracy

Scientists have claimed they can detect scientific papers generated by artificial intelligence models with over 99% accuracy. Machines have significantly improved at replicating human writing in recent years, making it difficult to distinguish between AI-generated text and human-authored documents. This poses a problem for educators concerned about plagiarism by students using such tools. Experts have warned against the use of unreliable software designed to detect AI-generated text to assess work. However, researchers at the University of Kansas have developed a method for detecting AI-generated science writing in the style of academic journals. The team trained and tested an algorithm with datasets of papers written by scientists and by a generative AI model called ChatGPT. The algorithm was able to discern the difference between human and AI-written texts 100% of the time, although accuracy dropped slightly to 92% at the individual paragraph level. The researchers believe the classifier works by spotting stylistic differences between the two types of writing, such as richer vocabulary and longer paragraphs among human writing. However, the extent to which the approach can be applied to other scientific journals remains to be determined.