New OpenAI safety team will have power to block high-risk developments
OpenAI, a leading artificial intelligence (AI) company, has unveiled a new safety plan aimed at mitigating risks associated with the development of AI technologies. The plan grants the company’s board of directors the authority to veto high-risk projects, effectively giving them the power to overrule the Chief Executive, Sam Altman.
In a blog post, OpenAI acknowledged the existing shortcomings in studying and understanding the potential risks associated with frontier AI. To bridge this gap, the company has adopted the initial version of their Preparedness Framework. This framework outlines the processes through which OpenAI will track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful AI models.
The safety plan includes dedicated teams responsible for overseeing the potential abuses and risks associated with current AI models. For example, the safety systems teams will monitor AI models like ChatGPT. Additionally, OpenAI has established a Preparedness Team that will assess frontier models, while a superalignment team will closely monitor the development of superintelligent models. All these teams will report directly to the board of directors.
Although the development of AI surpassing human intelligence may still be a distant reality, OpenAI is committed to anticipating future advancements. The company plans to push new models to their limits, after which detailed scorecards will be provided for four risk categories: cybersecurity, persuasion (including lies and disinformation), model autonomy, and CBRN (chemical, biological, radiological, and nuclear threats). Each category will be assigned a risk score ranging from low to critical, with corresponding post-mitigation scores.
OpenAI’s safety plan also emphasizes accountability. If issues arise, the company is prepared to bring in independent third parties to audit the technology and offer feedback. OpenAI intends to collaborate with external parties and internal teams to track real-world misuse and emergent misalignment risks.
We are also pioneering new research in measuring how risks evolve as models scale, to help forecast risks in advance, the company stated, referring to their success in scaling laws. By anticipating potential risks, OpenAI aims to navigate the ever-evolving landscape of AI development responsibly.
The announcement of OpenAI’s new safety plan reflects the company’s commitment to ensuring the safe and beneficial advancement of AI technologies. Through their Preparedness Framework and involvement of the board of directors, OpenAI seeks to proactively address risks associated with the development of AI systems. With a focus on accountability and collaboration, the company aims to build a safer and more secure AI future for humanity.
As the field of AI progresses, it becomes increasingly important to monitor and mitigate the potential risks associated with these powerful technologies. OpenAI’s safety plan represents a proactive step towards responsible AI development, with the potential to shape the future trajectory of the industry.
