What is data poisoning? Attacks on the data AI models learn from, defined

Data poisoning is an attack in which an adversary alters the data an AI model is trained, fine-tuned or grounded on, so that the finished model misbehaves: it degrades, makes targeted errors, or carries a hidden backdoor that a later trigger activates. NIST classes it as a poisoning attack in which the adversary controls part of the training data.

How it works

NIST's taxonomy, NIST AI 100-2 E2025, sorts poisoning by effect. Availability poisoning degrades a model across the board, while targeted and backdoor poisoning cause wrong results only on chosen inputs, which makes them harder to notice in testing.

Generative models raise the exposure because they are trained on data scraped at scale. NIST gives the example of an attacker buying domains that serve URLs listed in a published dataset and replacing the content. OWASP's LLM04:2025 entry extends the target beyond pre-training to fine-tuning and embedding data, and warns that a backdoor can leave behaviour unchanged until its trigger appears.

MITRE ATLAS tracks the attack as technique AML.T0020, Training Data Poisoning. Its description covers adding, removing or modifying samples, altering labels, and tampering with feedback loops, and it notes that the learned behaviour can outlast the poisoned data itself.

What it is not

Data poisoning is distinct from model poisoning, which NIST describes as modifying the trained model directly to inject malicious functionality. It is also distinct from a malicious model file: OWASP notes that models shared through public repositories can carry malware, for example through malicious pickling, which is a supply chain risk beyond poisoning. And it is not prompt injection, which manipulates a model at the moment of use rather than during training.

Why it matters to a CISO

In the EU, resisting poisoning is a legal requirement for some systems. Article 15(5) of the EU Artificial Intelligence Act requires high-risk AI systems to include, where appropriate, measures against data poisoning, model poisoning, adversarial examples and confidentiality attacks.

The controls are mostly about provenance. OWASP recommends tracking data origins and transformations, for example with an ML bill of materials, and ATLAS lists dataset provenance, training data sanitization and model validation among its mitigations. Because a backdoor may stay dormant through ordinary testing, red-team exercises that plant controlled poisoned records are one way to check whether those controls catch tampering.

Where it appears on cisonews.ai

The explainer on protecting training data covers the controls in depth, and what the EU AI Act requires of a CISO covers Article 15. For the supply chain contrast, see the record of malicious models found on Hugging Face. The ATLAS technique index lists which logged incidents match which technique.

Related terms

High-risk AI system, MITRE ATLAS and AI red teaming.