In brief
- Sources need a domain-defined hierarchy and visible validity.
- A higher similarity score does not make a passage more authoritative.
- The system must distinguish no evidence from a merely weak match.
- Conflicts and knowledge gaps are operational data that should drive source improvement.
Which kinds of conflict occur?
Conflicts are not limited to two directly opposing sentences. Often an old and new version coexist, a general rule meets an exception or two departments use the same term differently. The application needs metadata to recognise these situations.
- Expired and current versions of the same rule
- A global policy and a permitted local exception
- A draft and approved document without clear status
- Different definitions or scopes
- A database value and document statement with different freshness
How is source priority defined?
Priority is a domain decision and cannot be delegated to the language model. Each topic should define which source is authoritative, how exceptions are marked and when a version takes effect.
- Approval status and accountable owner
- Validity period and affected location or audience
- Source type: system of record, policy, guidance or informal note
- Explicit rule for a specific case over a general rule
How should the assistant respond?
If approved rules resolve the conflict unambiguously, the application uses the valid source and cites it. If uncertainty remains, the answer must show the disagreement. Without sufficient evidence, a bounded non-answer is better than an invented resolution.
| Source state | May the system resolve it? | Response behaviour | Next route |
|---|---|---|---|
| Approved current version and clearly superseded old version | Yes, using the documented version rule | State the current rule with validity and citation | Remove the old version from active retrieval and retain a regression test |
| General rule and approved local exception | Yes, when audience, place and time clearly match | Explain the applicable exception and label the general rule as context | Ask for missing applicability information first |
| Two approved sources with unclear priority | No | Expose the contradiction with both citations and do not invent a decision | Escalate to the domain owner and limit the answer until resolved |
| Only a weak result or no supporting evidence | No | State the knowledge gap instead of producing a plausible answer | Ask a clarifying question or name the responsible team |
| The authoritative source is technically unavailable | No | Show the outage and last known timestamp without claiming freshness | Trigger the defined outage process and monitoring |
How are knowledge gaps fixed permanently?
Unanswerable questions, recurring conflicts and overridden answers should be categorised. The domain owner decides whether to clean sources, add metadata or publish a new authoritative statement. The case then becomes a regression test.
- Log the conflict with the query, results and versions
- Route it to the responsible content owner with appropriate privacy controls
- Correct the source rather than only the prompt or response wording
- Add the corrected case to retrieval and answer evals
Example: conflicting travel expense limits
An old travel policy lists a hotel allowance of CHF 180, while a new country rule lists CHF 220. Metadata shows that the new rule applies in Germany from January. For a February trip the application uses the country rule and displays both applicability details. If the destination is missing, it asks rather than choosing a limit.
What to remember
Define authority outside the model. A good assistant resolves only documented conflicts and turns all remaining uncertainty into something people can act on.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed