OpenAI Unveils Automated Approach to Supervising AI Models – Preventing Harmful Output with Neural Networks

OpenAI’s Superalignment team has published its first research paper, detailing an automated approach to supervising AI models. The team, led by OpenAI’s chief scientist Ilya Sutskever and head of alignment Jan Leike, aims to develop methods to prevent advanced AI models from generating harmful output. The concern is that human input may be insufficient to regulate future neural networks with superhuman reasoning capabilities. OpenAI’s proposed solution involves using a less advanced neural network to supervise and prevent harmful output generated by a more advanced neural network. This method, known as weak-to-strong generalization, was tested by using an AI model similar to GPT-2 to supervise the latest GPT-4 model. OpenAI’s researchers addressed implementation challenges and developed an algorithm to encourage the strong model to be more confident, even if it disagrees with the weak supervisor. While the quality of the output was reduced to some extent, the experiment showed promising results, with the advanced model’s capabilities being preserved. OpenAI has also released open-source code files to help developers refine the automated AI supervision method and has launched a $10 million grants program to support research in this area.