Blockchain security firm OpenZeppelin conducted a study on OpenAI’s GPT-4, the latest multimodal model, to test its effectiveness in identifying smart contract vulnerabilities. The study assessed the potential of GPT-4 to detect and propose fixes for vulnerabilities in smart contract code within 28 Ethernaut challenges. While the AI tool was able to solve most of the tasks, it generally lacks knowledge of events after September 2021 and cannot learn from its experiences. Additionally, the model made up vulnerabilities that don’t exist and relied on false information in its explanation. Thus, OpenZeppelin’s findings suggest that AI tools like GPT-4 may shift developer jobs and improve efficiency but will not replace human auditors soon.
OpenZeppelin is a blockchain security company that provides products, services, and education to help businesses improve their security posture in the decentralized ecosystem.
Mariko Wakabayashi is the machine learning lead at OpenZeppelin who led the study testing the potential of GPT-4 in identifying smart contract vulnerabilities.
