OpenAI’s AI Classifier, a plagiarism detection software, has quietly been discontinued due to its low rate of accuracy. The announcement of the software’s discontinuation was buried deep in the company’s website, taking a week for anyone to notice its absence. AI Classifier’s landing page now displays an error notification, with two sentences at the top of the January blog post announcing the tool’s end. OpenAI cited the need for more effective provenance techniques as the reason behind its research efforts.
The inaccuracy of AI Classifier does not come as a surprise, as its initial announcement focused more on its limitations rather than its strengths. The tool was only able to correctly identify 26 percent of AI-written text and produced false positives 9 percent of the time, marking human-written text as plagiarized from AI sources.
The discontinuation of AI Classifier raises concerns in the education sector, where reliable AI plagiarism detectors are crucial. The prevalence of generative AI, such as OpenAI’s ChatGPT, has led academic institutions to rely on detection software to combat AI-generated plagiarism. However, with AI Classifier’s failure, these institutions are left without a reliable and effective solution.
Marc Watkins, a professor at the University of Mississippi specializing in AI in education, sees AI Classifier’s shutdown as indicative of a larger problem. He points to the University of Pittsburgh’s recent discontinuation of the A.I. detection tool Turnitin for similar accuracy reasons. Watkins believes that the short lifespan of AI Classifier, released in January and shut down by July, is a clear sign that the technology is insufficient.
Watkins is not alone in his skepticism toward AI detection technology. In a recent Twitter poll, only 15.3 percent of respondents out of 667 believed that it is possible to create a consistently accurate AI detector. The fact that OpenAI, the company behind ChatGPT, struggles to differentiate its own algorithm’s generated text raises doubts about the efficacy of other detection tools.
Inaccurate AI detection has serious consequences beyond missing a few AI-generated papers. Students can be wrongfully accused of plagiarism due to trigger-happy AI detection systems, affecting their academic reputation. Additionally, non-native English-speaking students are misflagged at higher rates as AI detectors use frameworks like perplexity, which scores non-native speakers lower.
Although the discontinuation of AI Classifier itself has minimal impact due to the availability of other AI detection tools, it underscores the overall ineffectiveness of current technology against AI-generated content. Watkins does not recommend any existing detection tools, and OpenAI’s retreat from classification software further emphasizes the need for a more reliable solution.
In conclusion, the discontinuation of OpenAI’s AI Classifier highlights the shortcomings of current AI detection technology. The education sector faces challenges in combating AI-generated plagiarism, as existing tools lack accuracy. The discontinuation serves as a wake-up call for the need to develop more effective solutions that can accurately detect AI-generated content and protect academic integrity.
