AI hallucinations explained: why chatbots make things up and how to catch it

Why ChatGPT, Claude and Gemini sometimes invent facts, what published benchmarks show about how often, and practical checks a UK business can put in place.

An AI hallucination is when a chatbot states something false as if it were true: a case that does not exist, a price that is wrong, a clause that is not in the contract. It happens because these models are built to produce a plausible answer, not to check one. Every current model does it some of the time. The fix is not finding a model that never does, but designing your use of AI so that errors are caught before they reach a customer, a court or your accounts.

  • In an October 2025 study by the EBU and BBC, 45% of AI assistants’ answers about news had at least one significant issue, and 20% had major accuracy problems.
  • On Vectara’s public summarisation test, current models from Anthropic, OpenAI and Google added unsupported content to between about 6% and 12% of summaries.
  • In June 2025 the High Court in London warned that freely available AI tools “are not capable of conducting reliable legal research”, after fake case citations reached court.
  • Giving the model your own documents, letting it say “I don’t know” and checking claims against sources all reduce the risk. None removes it.
AI assistants and news accuracy
45%of answers had at least one significant issue
31%had serious sourcing problems
20%had major accuracy issues, such as invented or outdated details
3,000+answers reviewed by 22 broadcasters in 18 countries

Source: EBU and BBC, News Integrity in AI Assistants, October 2025

Why AI models make things up

A large language model generates text one small piece at a time, each time choosing a likely next piece based on patterns learned from huge amounts of writing. It has no built-in list of facts it can look up, and no separate step where it checks its answer against the world. When it knows something well, the likely answer and the true answer coincide. When it does not, it still produces something that reads like a confident answer.

OpenAI published research in September 2025 arguing that the way models are trained and tested makes this worse. Its summary says “language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty”. If a test only counts right answers, a model that guesses scores better than one that admits it does not know, in the same way a student gains marks by guessing on a multiple choice paper.

OpenAI’s own example makes the point. On its SimpleQA factual quiz, an older model (o4-mini) answered 24% correctly and got 75% wrong, abstaining only 1% of the time. A newer model (gpt-5-thinking-mini) answered 22% correctly but declined to answer 52% of the time, so only 26% of its answers were wrong. Slightly fewer right answers, far fewer wrong ones. For business use, that second behaviour is usually what you want.

What hallucinations look like in practice

  • Invented sources. References, case law, statistics or web links that look real and are not.
  • Wrong details in a real thing. The right regulation with the wrong threshold, or a real product with last year’s price.
  • Unsupported additions to a summary. A summary of your contract that includes an obligation the contract does not contain.
  • Out-of-date facts stated as current. Every model has a knowledge cutoff. Without web search or your documents, it does not know what has changed since.
  • Misattribution. A real quote or figure credited to the wrong organisation. The EBU and BBC study found sourcing was the biggest single problem, in 31% of answers.

How often does it happen?

There is no single hallucination rate. It depends on the model, the task and how the question is asked. Two published sources give a sense of scale, and both explain their methods.

News questions: the EBU and BBC study

Twenty-two public service broadcasters in 18 countries, including the BBC, asked ChatGPT, Copilot, Gemini and Perplexity news questions in 14 languages, and journalists reviewed more than 3,000 answers. The results, published in October 2025, found 45% of answers had at least one significant issue. Gemini performed worst, with significant issues in 76% of its responses, which the EBU put down largely to poor sourcing. The models tested have since been replaced, so treat this as evidence of the problem, not a current ranking.

Summarising documents: the Vectara leaderboard

Vectara, an AI search company, runs a public leaderboard that asks each model to summarise more than 7,700 articles using only facts in the text, then uses its own evaluation model (HHEM-2.3) to judge whether each summary adds anything unsupported. The dataset is kept private so models cannot be trained on it. Its update of 22 October 2026 included these results for recent models from the three big providers:

Share of summaries containing unsupported content, selected models

Measures one task, summarising a supplied article. Judged by Vectara's own model. The newest releases, such as Claude Fable 5.2, GPT-6.2 and Gemini 4 Carbon, were not yet listed.

Source: Vectara hallucination leaderboard, HHEM-2.3, updated 22 October 2026

View data
 Share of summaries containing unsupported content, selected models
GPT-6 Sol6.5%
GPT-6 Astra8.7%
Claude Haiku 4.59.8%
Claude Opus 5.59.9%
Gemini 3.1 Pro preview10.4%
Gemini 4 Argon12.1%
GPT-5.6 Sol12.4%

Three cautions about that chart. It measures one narrow task, so a model that summarises well may still invent facts when asked open questions. The judge is itself an AI model, which can be wrong. And the lowest score on the whole leaderboard, 1.8%, is still not zero, while the highest was 24.2%. Even the best case means roughly one summary in fifty contains something the source did not say.

Model makers also publish their own figures. Google, for example, said on 10 November 2026 that Gemini 4 Carbon passed 92% of tasks on an internal agentic honesty test. Vendor results can be useful, but they are not independent, and they rarely tell you how a model will do on your documents.

The UK cases that made this real

The clearest warning for UK businesses came from the courts. In Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin), decided on 6 June 2025, the Divisional Court dealt with two cases where lawyers had put fictitious authorities before the court. In one, 18 of the 45 citations relied on did not exist. In the other, five fake cases were cited.

The judgment, given by Dame Victoria Sharp, President of the King’s Bench Division, and Mr Justice Johnson, said that freely available generative AI tools “are not capable of conducting reliable legal research”. It said they can produce “apparently coherent and plausible responses” that “may turn out to be entirely incorrect” and “make confident assertions that are simply untrue”.

The lesson applies well beyond law. If you send a client a report, a tender response or a compliance statement, you are responsible for what is in it, whoever or whatever drafted it. You are also generally responsible for what you tell customers, and a chatbot on your website speaks for your business. Our guide to AI chatbots for small businesses covers that risk.

Why people trust it anyway

AI answers are fluent and well organised, which makes them feel reliable. The Reuters Institute’s Generative AI and News Report 2026 found that 75% of people who used AI to answer factual questions said they trusted the answers. Ofcom’s April 2026 Adults’ Media Use and Attitudes report found 45% of UK AI users had used it to find factual information. Plenty of those answers will be right. The problem is that the wrong ones look exactly the same.

How to reduce hallucinations

You cannot switch hallucinations off, but you can make them much less likely and much easier to catch.

  • Give it the source. Ask questions about documents you supply (your policy, the contract, the price list), not about the world in general. Models are far more accurate when the answer is in front of them. This is the idea behind document chat tools, which we explain in RAG explained.
  • Ask for quotes first. Anthropic’s prompting guidance suggests asking the model to pull out the relevant quotes from a long document before answering. You can then check each quote exists.
  • Allow “I don’t know”. Say explicitly that it should say when the answer is not in the material, rather than guess.
  • Turn on web search for current facts, and click through to the sources it cites. A citation is only useful if you open it.
  • Keep the task narrow. “Extract the notice period from this lease” is safer than “tell me everything I need to know about this lease”.
  • Have a person sign off anything external. Especially figures, legal or regulatory statements, and anything with a name, date or price in it.

Most of these come down to giving the model better context. We go into that in our guide to context engineering for business prompts.

Checking an AI answer

Step 1 of 5

Spot claims

Highlight every name, number, date, quote and citation in the answer.

Step 2 of 5

Trace each

Find the original source for each claim, not another AI summary.

Step 3 of 5

Check fit

Confirm the source says what the answer claims, in context.

Step 4 of 5

Fix or cut

Correct anything wrong and remove what you cannot verify.

Step 5 of 5

Record it

Note recurring errors and adjust your prompts or process.

Where the risk matters most

Match the amount of checking to the cost of being wrong. A first draft of a social post needs a quick read. A figure in a funding application, an answer to a customer about their refund rights, a statement in a tender or anything a regulator might read needs a proper source check. Your AI policy should say which is which, and the UK GDPR checklist for AI agents covers the extra care needed when personal data is involved, including the right to have inaccurate personal data corrected.

AI agents that take actions, such as sending emails or updating records, raise the stakes further, because a wrong belief turns straight into a wrong action. Give them narrow permissions and a human approval step for anything that leaves the business.

Common questions

Is one AI assistant much more accurate than the others?

Not consistently. Rankings change with each release and differ by task. On Vectara’s October 2026 update, the current models from Anthropic, OpenAI and Google were within a few percentage points of each other. Our comparison of Claude, ChatGPT and Gemini covers the other differences.

Does web search stop hallucinations?

It helps with recent facts, but it does not guarantee accuracy. In the EBU and BBC study the most common problem was sourcing, with missing, misleading or incorrect attribution in 31% of answers. The model can still misread or misattribute what it finds.

Will future models stop doing this?

Newer models are generally better at declining to answer when unsure, as OpenAI’s SimpleQA figures show. But the underlying method, predicting plausible text, has not changed, so plan for errors to continue at some rate.

Are we liable for what our chatbot says?

In general, a business is responsible for the information it gives customers, however it was produced. This is general information, not legal advice. If your chatbot gives answers about prices, refunds or contracts, take advice and keep a person in the loop.

Where to start

List the places your business already uses AI output, mark the ones where a wrong answer would cost money or trust, and put a named person and a simple check in front of each. If you want help designing AI tools that are grounded in your own documents and built with checks from the start, our AI and automation team can help.

Sources

→Keep reading

Related guides.

All insights
AI explained

Claude vs ChatGPT vs Gemini: which AI assistant suits a UK business?

Claude, ChatGPT and Gemini compared for UK businesses: current plans and prices, data training settings, team features, and which one fits which kind of… Read
AI explained

Claude for business: plans, prices and what a UK team can do with it

What Anthropic's Claude offers a UK business: Pro, Team and Enterprise prices, data settings, Claude Code and Cowork, and where it fits or falls… Read
AI explained

ChatGPT for business: which plan a UK team needs and what it costs

ChatGPT Plus, Business and Enterprise explained for UK firms: current seat prices, data and privacy settings, ChatGPT Work agents, and which plan to choose. Read

→Get in touch

Got a question? Ask us.

Tell us what you’re trying to do. Every project starts with a free audit: someone who does the work replies with a few ideas, an honest view and a written plan with prices.

What happens next

  1. 1
    You send the formIt takes about two minutes.
  2. 2
    We do our homeworkWe look at your website and your market.
  3. 3
    You get a short planWhat we’d do first, with prices. Yours to keep.
  • Free, with no obligation
  • We won’t add you to a mailing list

We only use your details to reply to you. See our privacy policy.