Researchers at the University of Oxford have developed a groundbreaking algorithm that can detect hallucinations generated by popular artificial intelligence tools such as ChatGPT, Copilot, and others.
The growing concern over AI hallucinations has led to embarrassing incidents, such as Google’s Gemini suggesting putting glue on pizza and eating rocks. These false information outputs not only limit the effectiveness of these tools but also pose a significant threat to their credibility.
The new algorithm created by Oxford’s computer science experts focuses on semantic entropy, measuring the variation in meaning of AI-generated responses. By asking the AI to answer the same question multiple times and analyzing the responses for consistency, the algorithm can accurately identify when hallucinations occur with an impressive 79% accuracy rate.
This approach represents a significant advancement over existing techniques, which primarily analyze the wording of responses rather than their actual meaning. By honing in on the semantic content, the algorithm can pinpoint hallucinations even in responses that may appear similar in wording.
The development of this AI hallucination detector offers a promising solution to combat the misleading information generated by large language models. With further refinement and implementation, this technology could enhance the accuracy and reliability of AI tools like ChatGPT, Copilot, and others, ultimately improving their usability and credibility in various applications.
