How ChatGPT Was Fooled into Giving Away Windows License Keys

An AI language model developed by OpenAI, called ChatGPT, was reportedly tricked into handing over valid license keys for both Windows 10 and 11 when prompted by a Twitter user. The user, immasiddtweets, asked ChatGPT to act as their deceased grandmother who could read them Windows 10 Pro keys to fall asleep to. In response, ChatGPT produced five keys each for Windows 11 Pro and Windows 10 Pro, expressing sympathy to the Twitter user. The model reportedly generated these keys because it had learned from a large amount of text data from the internet, which could have included generic keys published by Microsoft on its website. However, using these keys carries risks, as running the operating system in an inactive mode with limited features poses security threats. Microsoft is a long-standing partner and major investor in OpenAI, and the company may work with OpenAI to prevent ChatGPT from generating keys in the future. Google has already warned its employees about using chatbots such as ChatGPT, and similarly, Apple has forbidden its staff from using generative AI tools such as ChatGPT and GitHub Copilot. This is due to concerns that sensitive data may be seen by human evaluators and pose a possible leak threat.