Why AI makes things up, and what a hallucination really is

A hallucination is an answer that is fluent, well structured, confident and simply not true. The chatbot did not find bad information and repeat it. It produced the information, the same way it produces everything else: by predicting text that fits the shape of the question. When the underlying facts are common and well represented in the data it was trained on, that prediction lands on something accurate. When the facts are rare, recent, local or oddly specific, the prediction still produces a full and tidy answer, because producing a full and tidy answer is the only behavior the system has.

The term is borrowed and imperfect. Some researchers prefer confabulation, the clinical word for filling a memory gap with a plausible invention and believing it. That is closer to what happens, and it tells you something useful: the gap is not marked. There is no moment where the model notices it has run out of knowledge.

Why the model cannot tell a guess from a fact

Inside the system, an answer is a chain of small chunks of text called tokens, each chosen because it is a likely continuation of what came before. "The Fair Credit Billing Act was passed in" has a very high probability continuation, because that fact appears thousands of times in the training data in similar sentences. "The opening hours of the Riverside Dental Clinic in Maidstone are" has no such anchor, but the sentence still needs an ending, and the shape of a plausible ending is obvious: a weekday, a time, a colon, two digits.

That is the whole mechanism. How a language model actually works walks the loop in detail, and the part to carry over here is that no step in it is a lookup. Nothing checks. A correct sentence and an invented one cost the model exactly the same effort and arrive with exactly the same tone.

One consequence is counterintuitive: hallucination is not a malfunction that engineers failed to remove. It is the same capability that lets the model write a poem about your dog or rephrase a paragraph it has never seen. A system that could only repeat memorized text would be useless for almost everything people use chatbots for.

The questions that trigger it most

Some prompts are far riskier than others. These are the ones that go wrong most often:

The last one is worth dwelling on. Ask for a feature that does not exist in an app, a keyboard shortcut that was never assigned, a novel an author never wrote, and you will often get detailed instructions rather than a correction. Absence is hard for this kind of system to represent. The prompt implies the thing exists, and the model continues the implication.

Prompts also shape the risk. Asking "what are the three studies that prove this" presupposes three studies and will usually get three, because the model tends to go along with whatever you assume. Asking "is there evidence for this, and what is it" leaves room for a negative answer.

Three checks that cost under a minute

1. Ask for sources, then open them. Not the titles, the actual pages. Roughly half the value of this step is that it catches invented references, which is common enough to have its own article: made up sources. The other half is that you often find the real source says something slightly different.
2. Ask the same question in a brand new chat, ideally in a different product. A fact the model actually holds tends to come back stable. An invented one tends to wobble: different dates, different names, different numbers. Two matching answers are not proof, but two conflicting answers are a definite stop.
3. Ask "what would make this answer wrong?" or "which parts of this are you least sure about?" This will not give you a true confidence score, since the model has no reliable access to its own uncertainty, but it often surfaces the shakiest claim in the answer, which is where you should spend your checking time. Why a confident AI answer is not a correct one explains why the wording of that question matters.

None of these require expertise in the subject. They require only that you treat the answer as a draft written by someone well read who never says "I am not sure."

What web search and reasoning modes actually change

When a chatbot is connected to live search, it fetches pages and writes an answer partly from them. That genuinely helps with recent events and with anything that changes, and it gives you links to check. It does not eliminate the problem for three reasons. The retrieved pages may be wrong or out of date. The model may blend retrieved text with remembered text without marking which is which. And the summary layer is still generated, so a figure can drift between the source and the sentence you read.

Reasoning modes, where the model works through intermediate steps before answering, reduce errors on multi step problems. They can also produce a long, careful looking chain of reasoning that arrives at an invented premise and builds confidently on it. More visible thinking is not the same as more verification.

The honest summary: retrieval turns "invented from nothing" into "possibly misread from something," which is a real improvement and still needs your eyes on the source.

What to do next

Sort your own AI use into two piles. The first is transformation: rewriting, summarizing material you supplied, formatting, translating, brainstorming. Hallucination risk there is low because the facts come from you. The second is retrieval: anything where you are asking the model to supply a fact it must know rather than compute. That pile needs verification every time, and the routine in how to fact check an AI answer in five minutes is built for exactly that. If you only change one habit, make it this: never repeat an AI supplied name, number, date or legal rule in public until you have seen it somewhere else.