What Is an AI Hallucination?

An AI hallucination is when a language model produces text that sounds fluent and confident but is factually wrong or entirely fabricated. The model is not lying and it is not malfunctioning in the ordinary sense — it is doing exactly what it was built to do, generating plausible language, in a case where plausible and true come apart.

This article is part of our pillar on how AI search works, because hallucination is the failure mode that connects how these models are built to what they sometimes say about real businesses.

What the term actually describes

“Hallucination” is an umbrella term for several related failures:

Fabricated references. Invented citations, paper titles, authors, URLs, court cases, and quotations — formatted exactly like real ones.
Invented facts. False dates, false founding stories, false product specifications, awards a company never won, services it never offered.
False attribution. A real statement or action assigned to the wrong person or organization.
Confident specificity. Precise numbers, names, and details supplied where the model has no actual information — the precision itself is generated.

What unites these is not randomness. The text is usually coherent, well-formed, and internally consistent. That is what makes the failure dangerous: nothing in the surface of the answer signals that it was invented. A misspelled or garbled answer invites checking; a hallucinated answer invites trust.

It also helps to separate hallucination from two neighboring problems. A model repeating a falsehood that is widely published is making a source error — the record itself is wrong. A model misreading a real document is making an interpretation error. Hallucination, strictly used, is the case where the content was never in any source at all: the model generated it.

Why the mechanism produces this

A large language model is trained to predict the next token — the next piece of text — given everything that came before it. Training rewards text that resembles the patterns of good, informative writing in the training data. Over billions of examples, that produces a system that is extraordinarily good at producing text that reads like a correct answer.

But plausible and verified are different targets, and only one of them is being optimized. Three consequences follow directly from the design:

There is no verification step in generation. The model does not look a fact up in an internal database, check it, and then write it down. Knowledge from training is stored distributed across the model’s parameters as statistical patterns, not as records that can be retrieved and confirmed. When generation proceeds word by word, each word is chosen for fit, not for confirmed truth.

Gaps get filled, not flagged. If a person is asked about a company they know nothing about, they can say “I don’t know.” A language model can produce those words too, but nothing in its basic mechanism forces the admission at the moment information runs out. The same fluency that carries a well-supported answer carries an unsupported one, because the generation process does not mark the boundary between “patterns strongly supported by training” and “patterns I am completing by shape.” Additional training can teach models to decline more often, and providers do apply it — but it is a behavioral adjustment layered on top of the mechanism, not a change to the mechanism itself.

Confidence is a style, not a measurement. The assured tone of an answer reflects the tone of the texts the model learned from — reference works, journalism, documentation — not a calibrated internal estimate of correctness. An answer can therefore be delivered at full confidence while being entirely invented, because the confidence and the content are generated by the same process, independently of any checking.

This is why no provider claims to have eliminated hallucination. Reducing it is an active, documented area of work; eliminating it would require a different relationship between generation and verification than current architectures have. Any claim that a platform has “solved” it should be treated as marketing, not description.

The word itself is contested

“Hallucination” is borrowed from human perception, and the borrowing is imperfect. A hallucinating person perceives something that is not there. A language model does not perceive anything; it has no sensory experience to distort. Critics of the term argue that it anthropomorphizes the system, invites the wrong mental model (a mind that occasionally malfunctions rather than a prediction process working as designed), and subtly excuses the output — as if a false citation were a perceptual slip rather than a predictable product of how the tool works.

Alternatives such as “fabrication” or “confabulation” are sometimes proposed; confabulation — filling gaps in memory with invented detail, without intent to deceive — is arguably the closest clinical analogy. None of the alternatives has displaced “hallucination” in general use, and this site uses the term because it is the one readers will encounter. But the critics have a point worth keeping: the failure is not a glitch in an otherwise fact-checking machine. It is what a plausibility machine occasionally produces.

A concrete example

A customer asks an AI assistant about a regional accounting firm: when was it founded, and does it handle nonprofit audits? The firm’s real website gives a founding year and a services list, but the firm’s name resembles a larger firm’s, and its online footprint is thin.

The assistant answers smoothly: founded in 1987 (invented — the real year is 2004), with offices in two cities (one is invented), and yes, it offers nonprofit audit services (it does not). Every element is plausible. The founding-year format matches how firms describe themselves; the second office matches the geography; the service matches the category. Nothing was retrieved badly — the details were never anywhere. They were completed, the way a sentence is completed.

If that answer is then copied — into a directory listing, a forum post, a “best firms” roundup — the fabrication begins to look like a sourced fact, and later systems may repeat it with a citation attached. How that propagation works is the subject of our pillar on AI citations.

Why this matters for businesses

Hallucination is usually discussed as a problem for researchers and lawyers — the fabricated citation in a brief is the famous case. For businesses, the same failure mode meets reputation directly. The fabricated content is not abstract: it is a founding date, a certification, a service, a price, a legal or safety claim, attached to a real company’s name, delivered in an authoritative tone to a customer who asked a direct question.

Two features make this hard to manage. First, you cannot see it happen: unlike your own website, an AI answer is generated per conversation, so a false claim about your business can circulate in answers you never see. Second, you cannot edit it: there is no page to correct. The leverage an information owner has is indirect — which is exactly why the quality and consistency of the published record matters, because that record is both the model’s background knowledge and the raw material retrieval systems draw on.

What reduces it

The main mitigation is grounding: connecting generation to retrieved sources so the answer is built from documents that can be inspected, rather than from parametric memory alone. Search-connected systems retrieve pages, draw on them, and increasingly cite them, which changes the failure profile — the answer can be wrong because the source is wrong, but it is far less likely to be invented from nothing, and a citation gives the reader (and the business) something checkable.

Grounding reduces hallucination; it does not abolish error. Retrieval can pull the wrong source, a good source can be misread, and many answers are still generated without retrieval at all. The honest summary: hallucination is a documented, structural tendency of language models, being actively reduced, not a solved problem — and the defense available to any business is a published record clear and consistent enough that systems working from sources get it right, and fabrications stand out against it.

Common misconceptions

“Hallucination means the AI is broken or lying.” Neither. Lying requires intent and knowledge of the truth; the model has neither. And the behavior arises from the normal operation of the system, not a defect in a particular unit.

“It only happens on obscure topics.” Obscure topics are higher-risk, because training support is thin — which is precisely the situation of most small businesses — but fabricated citations and false details occur on well-known subjects too.

“If the answer includes a citation, it can’t be hallucinated.” Citations can themselves be fabricated, and a real citation may not support the claim it is attached to. A citation is an opportunity to check, not a guarantee.

“Newer models have fixed this.” Capability improvements and grounding have reduced the rate in documented evaluations; the underlying mechanism is unchanged, and providers continue to warn users to verify important information.