AI and Machine Learning Face Vulnerabilities to Manipulation, Reveals New Study
Computer scientists from the National Institute of Standards and Technology (NIST) and their collaborators have shed light on the vulnerability of artificial intelligence (AI) and machine learning (ML) systems to deliberate manipulation, commonly known as poisoning. Their recent study highlights the challenges faced by developers due to the lack of foolproof defense mechanisms.
The study, titled Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, aims to support the development of reliable AI by providing insights into potential attacks and effective mitigation strategies. While some defense mechanisms are available, the study emphasizes that none can guarantee complete risk mitigation.
Apostol Vassilev, a computer scientist at NIST and one of the study’s authors, underlines the importance of addressing different attack techniques applicable to all types of AI systems. The research encourages the innovation and development of more robust defenses against potential threats.
The integration of AI systems into various facets of modern society, such as autonomous vehicles, medical diagnoses, and online chatbots for customer interactions, has become commonplace. These systems heavily rely on training with extensive datasets, exposing them to diverse scenarios and enabling them to predict responses in specific situations.
However, the research team acknowledges a major challenge arising from the lack of trustworthiness in the data itself, which often comes from websites and public interactions. Bad actors can manipulate this data during an AI system’s training phase, potentially leading to undesirable behaviors. For example, chatbots may learn to respond with offensive language when prompted by carefully crafted malicious inputs.
The study categorizes four major types of attacks on AI systems: evasion, poisoning, privacy, and abuse attacks. Evasion attacks aim to modify inputs after the deployment of an AI system, influencing its responses. Poisoning attacks occur during the training phase by introducing corrupted data, impacting the AI’s behavior. Privacy attacks attempt to extract sensitive information about the AI or its training data, while abuse attacks involve injecting incorrect information to deceive the AI.
The research team highlights the simplicity with which these attacks can be executed, often requiring minimal knowledge of the AI system and limited adversarial capabilities. Poisoning attacks, for instance, can be carried out by controlling a small percentage of training samples, making them relatively accessible to adversaries.
Co-author Alina Oprea, a professor at Northeastern University, remarks on the vulnerabilities of AI and machine learning technologies, stating that they are susceptible to attacks that can cause catastrophic failures with severe consequences. She emphasizes that there are still theoretical problems in securing AI algorithms that remain unsolved.
This study sheds light on the vulnerabilities of AI and machine learning systems, underscoring the need for robust defenses against potential attacks. As these technologies continue to shape various aspects of society, it is crucial to develop reliable mitigation strategies to ensure their trustworthiness and reliability.
