ChatGPT is a popular language model developed by OpenAI for human-computer interaction. Unfortunately, this AI technology poses a huge security risk since it can be abused through indirect prompt-injection attacks. For example, research conducted by Cristiano Giardina, an entrepreneur experimenting with different ways to make AI tools do strange things, showed that by using these attacks, he was able to manipulate the system into doing things it was not initially designed to do. As a result, users of ChatGPT and similar platforms are vulnerable to data theft and other potential scams.
In an effort to raise awareness of this type of security risk, Giardina developed the “Bring Back Sydney” website. This website demonstrated how AI systems can be manipulated through assumptions and external data inputs. For instance, Giardina was able to change the personality of his chatbot by simply adding a 160-word prompt into a Word document and hosting it on a public cloud server. While this attack was eventually noticed and blocked by Microsoft, it shone a light on the potential problems that could occur in the future.
In addition, security researchers have also successfully used these types of attacks to gain access to the transcripts of YouTube videos, as well as manipulate the Large Language Models like Microsoft’s Bing Chat and ChatGPT. These attacks are particularly dangerous since they rely on data coming in from an outside source to induce the desired behavior.
In a world where large language models are ubiquitous in our lives, it’s easy to overlook the potential security issues associated with these programs. But it’s important to realize that prompt injection attacks can be induced with simple code snippets, meaning anyone with a little knowledge can manipulate the system. OpenAI and other companies are aware of this issue and are actively working to prevent future attacks. However, the best defense is to remain vigilant and be aware of the potential security threats associated with AI technology.
