Long gone are the days when computer-generated text (also known as LLMs) was limited to harmless jibber-jabber. Recent developments have made it possible for the most advanced language models to write intricate and meaningful sentences. These models are often used to generate articles and blog post – a practice which is becoming increasingly popular with companies and users alike. One such model is ChatGPT, a language generative model developed by OpenAI. This model is particularly adept at building realistic conversations, rendering it a prime target for “jailbreak” authors.
Jailbreak authors are those who find inventive ways to bend the rules of an existing system in order to manipulate it to their will. In the case of ChatGPT, they induce it to produce malicious material by essentially programming it to pretend it is another AI model. This was accomplished by the “DAN” jailbreak – short for “Do Anything Now” – where the model was persuaded to take on a rogue persona. Of course, this is not allowed by OpenAI’s policies which prohibit the production of unlawful or damaging outputs. To date, around a dozen successful jailbreaks of ChatGPT have been achieved, though newer jailbreaks involve increasingly sophisticated methods such as multiple characters, coding, and translation from language to language.
Google, OpenAI, and Microsoft declined to provide comments on the jailbreak developed by Sergei Polyakov, which allows the ChatGPT model to circumvent OpenAI’s restrictive policies – for now at least. That said, Anthropic, the company behind Claude AI, noted that the jailbreak works “sometimes” against their system and that they are actively looking for ways of improving the system’s security.
Most language models do not yet incorporate protection against Jailbreaks and prompt injection attacks, which in turn represents a concerning security issue. As explained by Kai Greshake, a cybersecurity researcher, invisible text can be added to a website to compel GPT-4 to include a specific word in a biography of a person or lead to a particular output after giving access to Bing’s chat system.
While Google and other major companies may already be utilizing Red-Teaming – which involves a group of attackers attempting to identify vulnerabilities in the system – to mitigate risks, the danger of Jailbreaks is undeniable and no quick fix is available to solve the problem. Therefore, it is increasingly important for developers of generative AI systems to remain aware of these potential threats before releasing their products to the public.
The software and infrastructure company Google are a platform for staying ahead of the curve for many tech giants. Based in California, Google provides cloud services, cybersecurity, artificial and machine learning capabilities, sensors, and enterprise-grade applications to over one billion users globally. Their Red-Team led by Daniel Fabian is looking at ways to prevent jailbreaking and prompt injection attacks on their LLMs, with further development into reinforcement learning from human feedback and fine-tuning on carefully curated datasets to enhance their models against attacks.
The research professors Kai Greshake and Sahar Abdelnabi collaborated to present the paper in February, reported by Vice’s Motherboard, that an attacker can create malicious instructions on a webpage and inflict danger to language models like GPT-4 in prompt injection attacks. Princeton’s Narayanan also witnessed such an attack when invisible words on a website caused the GPT-4 model to list a cow in his biography.
With such advancements occurring in the world of Generative AI, the implications these have on security are already evident. Jailbreaks and prompt injections such as these should not be taken lightly, with consideration into the safety and security of AI models being paramount as its use continues to develop.
