OpenAI, the research organization behind the language model, GPT-3, has proposed a new strategy called process supervision to combat the tendency of AI models to hallucinate or fabricate information. Sometimes AI models produce false claims when they lack enough information to arrive at a correct answer. The AI ‘hallucinations’ present a problem for AI models, particularly in domains that require multi-step reasoning, where even a small logical error can produce numerous false conclusions.
The researchers propose using process supervision instead of outcome supervision to train AI models to reward themselves for each correct reasoning step needed to arrive correctly at an answer—instead of just rewarding the final correct conclusion. In their report, OpenAI researchers wrote, These hallucinations are particularly problematic in domains that require multi-step reasoning since a single logical error is enough to derail a much larger solution.
Google’s Chatbot, Bard, produced an untrue statement in a promotional video in February 2021. Additionally, OpenAI’s ChatGPT referring to bogus cases in a recent New York federal court filing was found to produce false information. Attorneys involved in the case face possible legal sanctions for the faulty information. The researchers at OpenAI claim that their proposed strategy could lead to better explainable AI since AI models would follow more of a human-like chain of thought approach.
OpenAI’s proposed strategy rewards AI models for accurate reasoning rather than the final conclusion. This could improve the accuracy and reliability of AI in domains that require multi-step reasoning and prevent inaccurate conclusions about information.
