AI Hallucinations Explained: Why Language Models Make Things Up and How to Reduce It

A support chatbot tells a customer that refunds take 14 days. The real policy says 7. The answer was fluent, polite and wrong. This is an AI hallucination, and it is one of the main reasons AI features need careful design and testing.

This guide explains the causes in plain terms and focuses on practical prevention. The automated check near the end is a small Python script we ran against example answers we wrote ourselves. It was not run against a live AI model.

What is an AI hallucination?

A hallucination is when a generative AI system produces content that is factually wrong, made up, or not supported by the information it was given, while presenting it as true. The term is a metaphor. The model is not seeing things. It is generating text that fits the pattern of a good answer without being tied to facts.

The dangerous part is the confidence. A hallucinated answer usually looks exactly like a correct one, with the same tone, structure and detail.

Why language models hallucinate

A large language model is trained to predict the next piece of text, a token, based on patterns in a huge amount of training data. Several things follow from that:

• Plausible is not the same as true. The model learns what answers tend to look like. When it lacks the facts, the most likely-looking continuation can still be invented.
• Gaps and cut-offs. Training data has gaps and ends at a certain date. The model may fill in missing facts rather than say it does not know.
• Training rewards answering. Research, including a 2025 OpenAI paper on why models hallucinate, argues that common training and evaluation methods reward guessing over admitting uncertainty, much like a multiple-choice test with no penalty for wrong answers.
• Ambiguous prompts. Vague questions invite the model to assume details.
• Long or messy context. Important facts can be overlooked when buried in long inputs.

Common types of hallucination

Researchers often split these into factuality errors, which contradict the real world, and faithfulness errors, which contradict the source material the model was given. For business apps built on your own documents, faithfulness is usually what you can test most directly.

Made-up code deserves a special warning. If an assistant suggests installing a package that does not exist, an attacker can publish a malicious package under that name. Check that every suggested dependency is real and trusted before installing it.

Where it matters most

A hallucinated movie fact is harmless. A hallucinated drug dose, legal citation, refund policy or financial figure is not. There have been well-reported cases of lawyers submitting court filings with case citations invented by AI tools. Rank your use cases by the cost of a wrong answer, and apply the strongest controls where that cost is high.

How to reduce hallucinations

1. Ground the answer in trusted data. Give the model the relevant facts rather than relying on its memory.
2. Allow “I don’t know”. Tell the model explicitly to say when the information is not available, and treat that as a good outcome.
3. Be specific. Clear, narrow questions leave less room for invention. Our prompt engineering guide covers this in detail.
4. Ask for quotes or citations from the provided sources, then verify them automatically.
5. Use structured output, such as JSON with fixed fields, so answers are easier to validate.
6. Use tools for things models are weak at, like calculations, dates and live data lookups.
7. Keep a human in the loop for high-stakes decisions.

Grounding with your own data

Retrieval-augmented generation (RAG) is the most common grounding pattern. The system searches your documents for passages relevant to the question and includes them in the prompt, with instructions to answer only from those passages. It sharply reduces invented facts, but it is not a cure. The model can still misread a passage, combine passages wrongly, or fall back on its own knowledge, and the search step can return the wrong documents. So grounded systems still need checks.

A simple automated check

Here is one cheap guard we built: flag any number in an answer that does not appear in the source text. It is crude, but numbers like prices, days and limits are exactly where costly hallucinations show up.

import re

SOURCE = """Refunds are paid within 7 days to the original payment method.
Orders over 1000 rupees ship free. Standard delivery takes 3 to 5 working days."""

def unsupported_numbers(answer, source):
    found = set(re.findall(r"\d+", answer))
    allowed = set(re.findall(r"\d+", source))
    return sorted(found - allowed, key=int)

We tested three answers written to imitate typical model output:

grounded     -> no unsupported numbers
hallucinated -> ['14', '500']
partly       -> ['1']

The “hallucinated” answer claimed 14-day refunds and free shipping over 500 rupees, and both numbers were caught. The “partly” answer had correct delivery times but added “or 1 day with express”, which the policy never mentions, and that was caught too. A second check verifies that quoted phrases really appear in the source:

{'paid within 7 days': True, 'no questions asked': False}

These checks have obvious limits. They miss wrong words, units and unsupported claims without numbers, and they would wrongly flag a correct calculation. Many teams add an “LLM as a judge” step, where a second model checks each claim against the sources. That catches more, but it can be wrong too, so measure it.

Testing for hallucinations

Treat hallucination rate as a quality metric you measure, not a feeling. A practical approach:

• Build a test set of real questions with known correct answers and sources.
• Include questions the system should refuse, because the answer is not in the data.
• Score answers for correctness, faithfulness to sources and appropriate refusals.
• Run the set on every change to prompts, models or retrieval settings.
• Sample production conversations for human review.

The discipline of regression testing applies here too: a fixed set of cases, run on every change, with results compared over time. When choosing how to ground a system, our comparison of RAG vs fine-tuning vs prompt engineering explains the trade-offs.

Advice for everyday users

• Verify facts, figures, quotes and citations from the original source.
• Be especially careful with medical, legal and financial information.
• Paste the source text in and ask the assistant to answer only from it.
• Ask “how confident are you, and what is this based on?”, but remember the answer can also be wrong.

Can hallucinations be eliminated?

Not with current technology. Newer models and better training have reduced hallucination rates on many benchmarks, and grounding helps a great deal, but every system still makes mistakes. The realistic goal is to make errors rare, easy to detect and low in impact through system design, testing and clear signals to users.

Hallucinations in AI coding tools

Developers meet hallucinations daily in coding assistants: a method that does not exist on a class, an outdated configuration option, or a confident explanation of an error that is really caused by something else. The defences are the ones you already use for human code. Compile or type-check the result, run the tests, and read the official documentation for any API you have not used before. Take extra care with security-sensitive code, such as authentication, cryptography and input validation, where plausible-looking code can be subtly unsafe. Treat AI suggestions as a draft from a fast but unreliable colleague: often useful, never automatically correct.

Final thoughts

Hallucinations come from how language models work: they generate likely text, not checked facts. You cannot switch them off, but you can design around them. Ground answers in trusted data, let the model say “I don’t know”, add automatic checks for the details that matter, and measure the results continuously.

Frequently asked questions

Keep learning