New AI Tool Can Detect Scientific Papers Produced by ChatGPT, Researchers Say

Researchers have announced the development of a highly accurate AI detection tool that can spot ChatGPT-generated text in scientific papers. The tool, developed by researchers at the University of Kansas, could be used to help journal editors manage the overwhelming volume of manuscripts they receive, by identifying and prioritizing articles for review. The researchers first analyzed sets of articles written by human scientists and by ChatGPT, identifying over 20 characteristics that could help identify which was the author. They then used this data to train XGBoost, an off-the-shelf machine learning algorithm, which was over 99% accurate in detecting ChatGPT-generated descriptions in a sample of test papers. While some experts have questioned the tool’s usefulness, others claim it could help streamline the peer-review process and detect plagiarism in student work, if appropriately trained. Critics also argue that the method of comparing 100% AI-generated text to 100% human-generated text is not realistic and could lead to false classifications. Regardless of these challenges, researchers say continued research into AI detection is important, and their tool could help improve scientific communication.