In brief
- Hallucinations arise from how generative models work and cannot be removed completely through prompting.
- Sources, RAG, structured data and explicit answer boundaries can reduce the risk.
- The required control depends on the consequence and use of an error, not on how convincing the answer sounds.
Why do false answers occur?
A language model calculates a likely continuation; it does not automatically perform a fact check. If information is missing, a question is ambiguous or the supplied context is contradictory, the model may fill the gap with a linguistically fitting pattern.
Even a correct source does not guarantee correct interpretation. The model can transpose numbers, miss exceptions or combine content from separate documents incorrectly. Hallucination therefore covers more than entirely invented facts.
| Cause | Visible symptom | Suitable measure |
|---|---|---|
| Current information is missing | The assistant states a plausible but invented price or contract status. | Fetch the authoritative value through an API or current source; do not answer without evidence. |
| The question or task is ambiguous | The model silently assumes a country, product or customer group. | Detect missing details and ask a targeted question instead of hiding assumptions. |
| Sources conflict | An answer combines old and new travel expense rules. | Resolve validity and precedence; expose the conflict and escalate it. |
| A relevant exception is lost in context | The general rule is correct, but a contract or safety exception is missing. | Retrieve more precisely, check mandatory claims and add edge cases to evals. |
| A tool or search result is misinterpreted | A missing inventory value is described as zero stock or available stock. | Validate structured status values and handle error states separately. |
Not every error has the same significance
An awkward wording suggestion is easy to spot and correct. A false dosage, contract statement or payment instruction can have serious consequences. Quality must therefore be assessed in the context of the task.
- Which type of error can occur?
- How likely is it to be detected before use?
- Can its effect be reversed?
- Who must review or approve the output?
Which measures reduce the risk?
There is no single safeguard. Good solutions combine appropriate technical architecture with organisational controls and clear user experience.
- Step 1
Constrain
Define the task, permitted sources and output format narrowly.
- Step 2
Ground
Provide current facts through RAG or structured interfaces and show supporting passages.
- Step 3
Evaluate
Test representative cases, including exceptions and deliberately unanswerable questions.
- Step 4
Control
Require human approval for high-impact use and technically limit actions.
- Step 5
Monitor
Systematically review errors, changes and user feedback in operation.
How does good user experience help?
The interface should present sources, freshness and uncertainty in an understandable way. It must not imply guaranteed correctness. When evidence is missing, a clear 'There is not enough information to answer this' is more valuable than a complete-sounding response.
Staff also need concrete guidance: Which content may be entered, how should results be checked, and where should an error be reported? AI literacy is part of quality control.
How is quality measured robustly?
A demo with a few successful questions is not enough. An evaluation set should include common cases, edge cases, different languages, outdated sources and deliberately unanswerable questions. Domain experts define expected statements or passages and assess errors by impact.
Relevant tests must run again after changes to models, prompts, data or processes. Quality is an operating process, not a one-time acceptance event.
Example: guidance on internal travel policy
Staff ask an assistant about permitted hotel costs. The application answers only when it finds a relevant current section, shows country, validity date and source, and directs conference exceptions to Finance. Evaluation covers normal and unknown countries, old policies and the exception. A plausible number without a source counts as an error.
What to remember
First define which errors are acceptable for the use case. Architecture, testing and approvals can then make the remaining risk transparent and responsible.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed