This year at its hacker convention, Defcon 31, attendees will be putting a number of widely used artificial intelligence (AI) tools to the test. From Google BERT and Bing AI, to OpenAI’s ChatGPT, AI models are getting put through the paces in hopes of uncovering any known or unknown bugs. Defcon has united with multiple AI companies, such as Anthropic, Hugging Face, Google, Nvidia, OpenAI, and Stability, as well as partnering with government bodies—the White House Office of Science, Technology, and Policy and the National Science Foundation’s Computer and Information Science and Engineering Directorate—to make sure the testing takes place.
This is being led by AI Village, in partnership with Humane Intelligence, SeedAI, and the AI Vulnerability Database, with the intention of educating attendees about the capabilities, advantages, and limitations of AI models. They will be providing laptops, access to each model, and a prize for the participant who can find the most bugs in the models.
The White House has an AI bill of rights which protects people’s rights in the face of AI systems, a concept they intend to bring to fruition by having independent experts evaluate the AI models and determine if they meet the bill’s criteria. Vice President Kamala Harris announced additional funding for research on AI systems as well as discussed the government’s use of AI systems with executives at Alphabet, Anthropic, Microsoft, and OpenAI.
Google, OpenAI, and the other companies mentioned have made a number of AI tools that are publicly available. This gives developers a wide range of options to choose from, but they still need to understand the practical workings of each system in order to use AI responsibly and safely. This initiative looks to equip attendees with the skills necessary to assess AI models, understand their quirks and limitations, and use them to their fullest potential. Defcon will give participants the chance to prove their skills and knowledge so that companies, developers, and the public can more effectively use AI models.
