OpenAI, a technology and research start-up backed by Microsoft, has revealed the findings of a ‘red team’ of data scientists, who have been dedicating their efforts to testing the capabilities of their AI system GPT-4, known as ChatGPT. This testing has been in response to increased scrutiny from EU authorities who are concerned about potential societal impacts.
50 data scientists from the US and Europe were hired by OpenAI to “qualitatively probe [and] adversarially test” GPT-4. For the last six months, these ‘red team’ testers–people from varied backgrounds, including academics, lawyers, and risk analysts–have been attempting to break the system, in an effort to discover its limitations and determine how to most effectively use the technology without causing any harm.
The testers mostly posed potentially hazardous questions to the chatbot in order to assess its vulnerability to misinformation, verbal manipulation, and other issues like linguistic bias, prejudice, and plagiarism. This testing was paid at a rate of up to $100 an hour, and each tester dedicated up to 40 hours to it.
Chemical engineering professor at the University of Rochester, Andrew White, used ChatGPT to show how to create a new nerve agent and found that the tool could be used to make chemistry faster and more accurately, but that its potential for misuse cannot be overlooked.
José Hernández-Orallo, a professor from the Valencian Research Institute for Artificial Intelligence, also tested the system and found that even if GPT-4 was not able to learn or retain memory, it could become a powerful tool if combined with Internet plug-ins.
Linguistic discrimination was also detected by legal adviser and technology and human rights researcher Roya Pakzad. She tested the system for racial, religion and gender bias in two languages, Farsi and English, and discovered that hallucinations were more frequent in the Farsi responses.
For testing in an African context, Nairobi-based lawyer Boru Gollu joined the team and shared that when using the system, it acted like a white person talking to him. OpenAI has since brought these findings to their attention and they have been training the system in response to stop any further biases.
In the US and EU, increased regulation over the use of AI chatbots and systems like GPT-4 is being seriously considered, with a task force now set up by the European Data Protection Board (EDPB) to set and enforce AI privacy rules. OpenAI is now expected to comply with certain rules from Italy and Spain, including creating an information campaign to spread awareness and an age verification system.
Overall, OpenAI promises to continue to ensure safety for the use of their AI system ahead of its widespread use, but with the potential for misuse, data privacy must remain a priority for both the company and for the world at large.
OpenAI is a research laboratory based in San Francisco, co-founded by entrepreneur and inventor Elon Musk, Sam Altman, Greg Brockman, Reid Hoffman, and others in late 2015. OpenAI focuses on neurotechnology, artificial intelligence research and the development of artificial general intelligence and is backed by Microsoft, Sequoia Capital, Peter Thiel, and Amazon Web Services. Their primary tool, ChatGPT, is a language model system that can generate human-like and coherent text when fed a prompt, and has recently been subject to deep and rigorous testing by OpenAI’s ‘red team’ for any potential misuses or risks.
