An AI hallucination is when a chatbot states something false as if it were true: a case that does not exist, a price that is wrong, a clause that is not in the contract. It happens because these models are built to produce a plausible answer, not to check one. Every current model does it some of the time. The fix is not finding a model that never does, but designing your use of AI so that errors are caught before they reach a customer, a court or your accounts.
- In an October 2025 study by the EBU and BBC, 45% of AI assistants’ answers about news had at least one significant issue, and 20% had major accuracy problems.
- On Vectara’s public summarisation test, current models from Anthropic, OpenAI and Google added unsupported content to between about 6% and 12% of summaries.
- In June 2025 the High Court in London warned that freely available AI tools “are not capable of conducting reliable legal research”, after fake case citations reached court.
- Giving the model your own documents, letting it say “I don’t know” and checking claims against sources all reduce the risk. None removes it.
Source: EBU and BBC, News Integrity in AI Assistants, October 2025
Why AI models make things up
A large language model generates text one small piece at a time, each time choosing a likely next piece based on patterns learned from huge amounts of writing. It has no built-in list of facts it can look up, and no separate step where it checks its answer against the world. When it knows something well, the likely answer and the true answer coincide. When it does not, it still produces something that reads like a confident answer.
OpenAI published research in September 2025 arguing that the way models are trained and tested makes this worse. Its summary says “language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty”. If a test only counts right answers, a model that guesses scores better than one that admits it does not know, in the same way a student gains marks by guessing on a multiple choice paper.
OpenAI’s own example makes the point. On its SimpleQA factual quiz, an older model (o4-mini) answered 24% correctly and got 75% wrong, abstaining only 1% of the time. A newer model (gpt-5-thinking-mini) answered 22% correctly but declined to answer 52% of the time, so only 26% of its answers were wrong. Slightly fewer right answers, far fewer wrong ones. For business use, that second behaviour is usually what you want.
What hallucinations look like in practice
- Invented sources. References, case law, statistics or web links that look real and are not.
- Wrong details in a real thing. The right regulation with the wrong threshold, or a real product with last year’s price.
- Unsupported additions to a summary. A summary of your contract that includes an obligation the contract does not contain.
- Out-of-date facts stated as current. Every model has a knowledge cutoff. Without web search or your documents, it does not know what has changed since.
- Misattribution. A real quote or figure credited to the wrong organisation. The EBU and BBC study found sourcing was the biggest single problem, in 31% of answers.
How often does it happen?
There is no single hallucination rate. It depends on the model, the task and how the question is asked. Two published sources give a sense of scale, and both explain their methods.
News questions: the EBU and BBC study
Twenty-two public service broadcasters in 18 countries, including the BBC, asked ChatGPT, Copilot, Gemini and Perplexity news questions in 14 languages, and journalists reviewed more than 3,000 answers. The results, published in October 2025, found 45% of answers had at least one significant issue. Gemini performed worst, with significant issues in 76% of its responses, which the EBU put down largely to poor sourcing. The models tested have since been replaced, so treat this as evidence of the problem, not a current ranking.
Summarising documents: the Vectara leaderboard
Vectara, an AI search company, runs a public leaderboard that asks each model to summarise more than 7,700 articles using only facts in the text, then uses its own evaluation model (HHEM-2.3) to judge whether each summary adds anything unsupported. The dataset is kept private so models cannot be trained on it. Its update of 22 October 2026 included these results for recent models from the three big providers:
Three cautions about that chart. It measures one narrow task, so a model that summarises well may still invent facts when asked open questions. The judge is itself an AI model, which can be wrong. And the lowest score on the whole leaderboard, 1.8%, is still not zero, while the highest was 24.2%. Even the best case means roughly one summary in fifty contains something the source did not say.
Model makers also publish their own figures. Google, for example, said on 10 November 2026 that Gemini 4 Carbon passed 92% of tasks on an internal agentic honesty test. Vendor results can be useful, but they are not independent, and they rarely tell you how a model will do on your documents.
The UK cases that made this real
The clearest warning for UK businesses came from the courts. In Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin), decided on 6 June 2025, the Divisional Court dealt with two cases where lawyers had put fictitious authorities before the court. In one, 18 of the 45 citations relied on did not exist. In the other, five fake cases were cited.
The judgment, given by Dame Victoria Sharp, President of the King’s Bench Division, and Mr Justice Johnson, said that freely available generative AI tools “are not capable of conducting reliable legal research”. It said they can produce “apparently coherent and plausible responses” that “may turn out to be entirely incorrect” and “make confident assertions that are simply untrue”.
The lesson applies well beyond law. If you send a client a report, a tender response or a compliance statement, you are responsible for what is in it, whoever or whatever drafted it. You are also generally responsible for what you tell customers, and a chatbot on your website speaks for your business. Our guide to AI chatbots for small businesses covers that risk.
Why people trust it anyway
AI answers are fluent and well organised, which makes them feel reliable. The Reuters Institute’s Generative AI and News Report 2026 found that 75% of people who used AI to answer factual questions said they trusted the answers. Ofcom’s April 2026 Adults’ Media Use and Attitudes report found 45% of UK AI users had used it to find factual information. Plenty of those answers will be right. The problem is that the wrong ones look exactly the same.
How to reduce hallucinations
You cannot switch hallucinations off, but you can make them much less likely and much easier to catch.
- Give it the source. Ask questions about documents you supply (your policy, the contract, the price list), not about the world in general. Models are far more accurate when the answer is in front of them. This is the idea behind document chat tools, which we explain in RAG explained.
- Ask for quotes first. Anthropic’s prompting guidance suggests asking the model to pull out the relevant quotes from a long document before answering. You can then check each quote exists.
- Allow “I don’t know”. Say explicitly that it should say when the answer is not in the material, rather than guess.
- Turn on web search for current facts, and click through to the sources it cites. A citation is only useful if you open it.
- Keep the task narrow. “Extract the notice period from this lease” is safer than “tell me everything I need to know about this lease”.
- Have a person sign off anything external. Especially figures, legal or regulatory statements, and anything with a name, date or price in it.
Most of these come down to giving the model better context. We go into that in our guide to context engineering for business prompts.
Step 1 of 5
Spot claims
Highlight every name, number, date, quote and citation in the answer.
Step 2 of 5
Trace each
Find the original source for each claim, not another AI summary.
Step 3 of 5
Check fit
Confirm the source says what the answer claims, in context.
Step 4 of 5
Fix or cut
Correct anything wrong and remove what you cannot verify.
Step 5 of 5
Record it
Note recurring errors and adjust your prompts or process.
Where the risk matters most
Match the amount of checking to the cost of being wrong. A first draft of a social post needs a quick read. A figure in a funding application, an answer to a customer about their refund rights, a statement in a tender or anything a regulator might read needs a proper source check. Your AI policy should say which is which, and the UK GDPR checklist for AI agents covers the extra care needed when personal data is involved, including the right to have inaccurate personal data corrected.
AI agents that take actions, such as sending emails or updating records, raise the stakes further, because a wrong belief turns straight into a wrong action. Give them narrow permissions and a human approval step for anything that leaves the business.
Common questions
Is one AI assistant much more accurate than the others?
Not consistently. Rankings change with each release and differ by task. On Vectara’s October 2026 update, the current models from Anthropic, OpenAI and Google were within a few percentage points of each other. Our comparison of Claude, ChatGPT and Gemini covers the other differences.
Does web search stop hallucinations?
It helps with recent facts, but it does not guarantee accuracy. In the EBU and BBC study the most common problem was sourcing, with missing, misleading or incorrect attribution in 31% of answers. The model can still misread or misattribute what it finds.
Will future models stop doing this?
Newer models are generally better at declining to answer when unsure, as OpenAI’s SimpleQA figures show. But the underlying method, predicting plausible text, has not changed, so plan for errors to continue at some rate.
Are we liable for what our chatbot says?
In general, a business is responsible for the information it gives customers, however it was produced. This is general information, not legal advice. If your chatbot gives answers about prices, refunds or contracts, take advice and keep a person in the loop.
Where to start
List the places your business already uses AI output, mark the ones where a wrong answer would cost money or trust, and put a named person and a simple check in front of each. If you want help designing AI tools that are grounded in your own documents and built with checks from the start, our AI and automation team can help.
Sources
- EBU: AI’s systemic distortion of news is consistent across languages and territories (News Integrity in AI Assistants, with the BBC) (October 2025)
- Vectara: Hallucination leaderboard (HHEM-2.3) (updated 22 October 2026, checked 22 November 2026)
- The National Archives: Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin) (6 June 2025)
- OpenAI: Why language models hallucinate (5 September 2025)
- Google: Gemini 4 Carbon is now available (10 November 2026)
- Anthropic: Prompting best practices, Claude documentation (checked 22 November 2026)
- Reuters Institute: Generative AI and News Report 2026 (29 October 2026)
- Ofcom: Adults’ Media Use and Attitudes report (2 April 2026)