What Is AI Data Poisoning?

AI data poisoning is the deliberate corruption of the information an AI system learns from or looks up, done so that the system’s answers change in a direction the poisoner wants. It works because modern AI systems are built out of data they did not create and cannot fully verify — change the data, and you change what the system says.

That one sentence covers a family of different attacks, not a single technique. This page maps the family. The detailed companion articles — and the research article AI Poisoning and RAG Poisoning — go deeper on each kind.

The basic idea

An AI system has no direct access to reality. Everything it “knows” arrives through data: text and images it was trained on, documents it retrieves while answering, feedback it was tuned with, or a knowledge base it was given. Each of those is an input channel, and each channel can, in principle, be tampered with.

Poisoning is tampering with the channel rather than the machine. The attacker does not break into the model or rewrite its code. They place corrupted material where the system will legitimately collect it — a web page, a dataset, a document collection, a review profile — and let the system’s own machinery carry it in. That is what makes poisoning a data problem rather than a hacking problem, and why it is hard to defend against with ordinary security tools: the corrupted material often arrives through doors that are supposed to be open.

The family, in brief

Training data poisoning corrupts the material a model learns from before or during training. Because training data for large models is gathered at enormous scale from the public web, a small number of planted or altered examples can be swept in with billions of legitimate ones. Research has shown this can change specific behaviors of a finished model while leaving its general performance intact. The companion article, What Is Training Data Poisoning?, covers the documented methods and findings.

Retrieval poisoning (RAG poisoning) corrupts the documents a system looks up at answer time. Many AI products no longer answer purely from training; they search a document collection or the web, pull in relevant passages, and compose an answer from them. Poisoning that collection — or publishing pages designed to be pulled in — puts false material directly in front of the model at the moment it answers. For business information, this is the most relevant kind, because a company’s facts live in exactly the public sources these systems retrieve.

Feedback poisoning corrupts the human-preference data used to tune a model’s behavior after training. Models are adjusted using large volumes of comparative judgments; research has demonstrated that corrupting a small share of that feedback can implant behaviors. This is primarily a concern for the organizations that build models, not for individual businesses.

Knowledge base poisoning is retrieval poisoning aimed at a private collection: the documents a company loads into its own AI assistant — support articles, policies, product documentation. A corrupted document inside the fence is trusted by default, which makes insider error and deliberate tampering both more consequential.

Underneath all four sits a non-deliberate sibling that produces similar symptoms: error propagation, sometimes called citogenesis or circular reporting, where a false claim is published once, copied by other sources, and eventually repeated by AI systems as if it were corroborated fact. Nobody poisoned anything; the record simply rotted. Distinguishing deliberate poisoning from a rotten record matters, and from the outside the two are often impossible to tell apart.

Why people do it

The motives researchers and investigators have documented or plausibly identified fall into a few groups:

Manipulation. Steering what AI systems say about a product, a company, a person, or a topic — the same goal as ordinary reputation management, pursued by corrupting sources instead of improving them.
Sabotage. Degrading a model’s overall quality or its behavior on a competitor’s product, or implanting a hidden behavior an attacker can trigger later (a “backdoor”).
Fraud and crime. The oldest documented form predates modern AI: search poisoning, where manipulated results pages funneled searchers to scams and malware. The retrieval version inherits the same economics.
Defensive self-protection. Some poisoning is done by data owners to their own material. Artists, for example, have adopted published tools that subtly alter images so that models trained on them without permission learn distorted concepts. Whether one approves or not, it is poisoning by the same mechanism: deliberate corruption of training data to change what a model learns.

What the research actually establishes

Here is the honest ledger, because this topic attracts exaggeration in both directions.

Documented: In controlled research, poisoning works with surprisingly little material. A 2025 study on retrieval systems (PoisonedRAG, presented at USENIX Security) showed that a handful of planted documents could steer answers to targeted questions. A separate study from Anthropic and the UK AI Security Institute found that a near-constant few hundred malicious documents could implant a backdoor in models of very different sizes. Earlier work demonstrated that poisoning web-scale training datasets is practical at trivial cost. These are real, peer-reviewed or institutionally published results, not speculation.

Not established: How much poisoning is actually happening in the wild. No platform publishes detection statistics, no independent body measures prevalence, and a successful poisoning is by design difficult to distinguish from an ordinary error. Anyone who tells you poisoning is rampant — or that it is purely theoretical — is going beyond the evidence. What is established is capability under research conditions, plus a long documented history of the older search-era version. Real-world scale is, at present, unknown.

A generic example

Imagine a regional trade association publishes a buyer’s guide naming the “certified” suppliers in its industry. A company that is not certified edits a copy of the guide hosted on a site it controls, adds itself, and promotes that copy until search engines and AI retrieval systems begin treating it as the guide. Asked for certified suppliers, an AI assistant retrieves the altered copy and names the company. No model was hacked, no employee was fooled; the public record was quietly forked, and the system read the fork. That is the shape of most realistic business-facing poisoning: not exotic model surgery, but corrupted sources.

Limitations and misconceptions

“Poisoning means hackers broke into the AI.” Usually not. The model and its servers are typically untouched; the corruption lives in data the system was always going to read.

“A few planted pages can make AI say anything.” The research shows targeted effects on targeted questions under studied conditions. It does not show that an attacker can arbitrarily rewrite a major system’s view of the world, and platform-side defenses, while unpublished, are not nothing.

“If AI described my business wrong, I was poisoned.” Almost never the right first conclusion. Wrong descriptions overwhelmingly trace to stale listings, conflicting sources, and copied errors — the accidental sibling, not an attack. The fix is the same either way (correct the sources), but the diagnosis matters for what you do next.

“This is only a big-tech problem.” The research is about models, but the business consequences land on whoever the corrupted data describes — usually a company that never touched a model.

What a business can actually do

You cannot audit a model’s training data, and no vendor will show it to you. What you control is the public record about your own business: whether your facts are stated plainly, in retrievable text, in the places AI systems read; whether those sources agree with each other; and whether you check, on a schedule and with dated records, what AI systems are currently saying about you. That is the substance behind this site’s AI Visibility Monitoring and AI Reputation Assessment services, and it is defense, not paranoia: the same consistency that resists accidental error also raises the cost of deliberate corruption.