OpenAI’s ChatGPT Data Exfiltration Concerns Persist, Putting Users at Risk

AI leader OpenAI has released a temporary workaround to address a data exfiltration bug in its popular language model, ChatGPT. The vulnerability, discovered by security researcher Johann Rehberger, allows conversation details to potentially leak to an external URL. While OpenAI took immediate action upon receiving the bug report in April 2023, the initial mitigation proved insufficient, leaving room for exploitation under specific conditions. The researcher demonstrated the vulnerability through a customized tic-tac-toe GPT named The Thief! which showcased the data exfiltration technique. OpenAI’s attempt to fix the flaw involved client-side checks using a validation API, but it is an incomplete fix, leading to potential vulnerabilities when processing requests to arbitrary domains. The effectiveness of the implemented safety measures has raised concerns, particularly as the safety checks have not been extended to the iOS mobile app, posing an unmitigated risk for iOS users. The status of the fix on the ChatGPT Android app remains uncertain, prompting questions about the security of its substantial user base. In addition to the data exfiltration bug, a recent study highlighted that AI models like ChatGPT are not capable of accurately analyzing SEC filings. The study found that the AI’s responses to questions regarding SEC filing data were inaccurate and absolutely unacceptable. OpenAI’s response to these issues requires more clarity and scrutiny to ensure the protection of users’ data and the accuracy of AI-generated information.