OpenAI, the creator of the ChatGPT language model, has revealed that it has trained a model to detect and mitigate AI hallucinations. AI hallucinations occur when AI models fabricate information and present them as facts. These are problematic for domains that require multi-step reasoning as a single error can derail larger solutions. OpenAI aims to build aligned artificial general intelligence by mitigating these hallucinations. To achieve this, they have trained an AI model to reward each individual correct step of reasoning and not just the final answer, which helps the model to produce a chain-of-thought endorsed by humans. OpenAI has released an accompanying dataset of 800,000 human labels that they use to train the model, and the team has noted that the process-supervised reward model performs better overall. Google and Microsoft are also working to improve their chatbots’ responses using LLM training models.
OpenAI is a research company focusing on artificial intelligence that was founded in December 2015 by renowned Tesla and SpaceX CEO, Elon Musk, and other prominent industry leaders.
Karl Cobbe is a MathGen Researcher at OpenAI and has co-authored the research paper discussed in this article.
