In brief
- RAG combines information retrieval with text generation.
- Business knowledge remains in its sources and can be updated continuously.
- Good results depend as much on document quality, permissions and retrieval as on the model.
Before the first question: prepare sources for RAG
RAG does not begin with a user question. A controlled knowledge collection must be prepared first. This determines which versions can be found, which roles may see them and whether a later result retains enough domain context to be useful.
- Step 1
Select sources
Include only approved collections with named ownership and validity.
- Step 2
Divide content
Split documents so that each retrievable passage retains enough domain meaning.
- Step 3
Add metadata and rights
Carry version, product, language, validity and existing access rules with the content.
- Step 4
Build and refresh the index
Create search representations and propagate content changes or revoked rights promptly.
For every question: retrieve, constrain and answer
At runtime, the entire knowledge collection is not sent to the model. The application first checks identity and filters, retrieves relevant passages and supplies only a bounded selection as context. The answer and its evidence must then remain reviewable together.
- Step 1
Capture the question and permissions
Determine user context, product, language and any other permitted filters.
- Step 2
Retrieve passages
Combine semantic search, keywords and metadata filters, then rank the results.
- Step 3
Ground the answer
Give the question and selected passages to the LLM and connect claims to citations.
- Step 4
Decline or escalate
When evidence is insufficient, ask for clarification or hand over rather than guessing.
Why not simply train the model?
RAG separates general language capability from changing business knowledge. When a policy changes, the index can be refreshed without retraining a model. Sources remain visible, and content can be filtered by role.
Fine-tuning can be useful for behaviour, style or a repeated specialist task. For frequently changing facts and traceable evidence, RAG is usually the more direct approach.
Common RAG failures and suitable controls
RAG does not repair contradictory, outdated or poorly structured documents. If several versions of a policy are equally searchable, the answer may also become inconsistent. Sources need an owner, validity, clear structure and explicit access rules.
| Failure | Common cause | Suitable control |
|---|---|---|
| Outdated answer | Several versions are indexed equally or an update is missing from the index. | Maintain validity and ownership; monitor updates and deletion. |
| Wrong product variant | Passages lose product, language or version context when content is divided. | Use meaningful sections and mandatory metadata filters. |
| Unauthorised result | Source permissions are not enforced in retrieval or are updated too slowly. | Check access during indexing and retrieval; explicitly test revocation. |
| Plausible answer without evidence | Weak results are treated as sufficient context. | Define minimum evidence, visible citations and an explicit refusal path. |
How is a RAG system evaluated?
A useful evaluation separates retrieval from answer generation. First, check whether the correct passage was found at all. Then assess whether the answer reflects it correctly and completely. Include questions that should deliberately remain unanswered.
- Collect representative questions with expected passages
- Measure retrieval, faithfulness and citations separately
- Test permissions and outdated content
- Turn feedback into specific improvements rather than merely storing it
When is RAG not the best solution?
For exact balances, prices or status values, an application should usually call a structured interface directly. For simple navigation, classic search may be enough. RAG is strong when several unstructured sources need to be understood, combined and explained in natural language.
Example: internal quality knowledge
A manufacturer stores work instructions, inspection requirements and defect catalogues in several repositories. A RAG system indexes only approved current documents, respects existing roles and shows the passages used for every answer. Unsupported questions are handed to the responsible domain team. Before rollout, real questions verify that the correct version and product variant are retrieved, not merely that the answer sounds good.
What to remember
Start with a bounded set of sources, real user questions and defined permissions. Retrieval is one part; content maintenance and domain ownership determine long-term value.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed