In brief
- Embeddings represent meaning as sequences of numbers; they are not the readable original text.
- Vector search finds passages with similar meaning even when they use different words.
- A vector database replaces neither the original source nor permissions and freshness rules.
- Good retrieval often combines vectors, keywords, metadata filters and subsequent ranking.
How does content become an embedding?
An embedding model receives a passage, image or other supported content and produces a fixed sequence of numbers. The same process is applied to a search query. A similarity measure then compares query and document vectors.
- Step 1
Split content
Documents are divided into meaningful sections with enough context.
- Step 2
Create vectors
An embedding model calculates a numerical representation for each section.
- Step 3
Store metadata
Source, title, version, language and permissions remain attached.
- Step 4
Search by similarity
The query is embedded and compared with stored vectors.
What does a vector database do?
A vector database or search index optimises search across many high-dimensional vectors. It returns candidates that are semantically related to the query and can constrain them using metadata. The stored original passage is still needed for display and model context.
- Fast similarity search across large content sets
- Filters for language, product, validity or user permission
- A link to the readable original and its location
- Updates and deletion of vectors when sources change
What determines retrieval quality?
An embedding alone does not create good search. Chunk size, headings, metadata, language and query type all influence results. The three retrieval approaches solve different problems and should be compared on the same real questions.
| Retrieval approach | Example | Particular strength | Typical failure |
|---|---|---|---|
| Keyword search | ‘Error code E-417’ finds that exact character sequence. | Product codes, names, domain terms and exact wording | Misses a passage that expresses the same meaning in different words. |
| Vector search | ‘Working abroad’ finds ‘temporary cross-border activity’. | Semantically similar questions, paraphrases and natural language | Returns a thematically similar but inapplicable special case. |
| Hybrid search | ‘E-417 during an overseas assignment’ combines the code match and semantic context. | Weights exact signals and semantic similarity together | Poor weighting suppresses the exact result or overvalues one keyword. |
What are the limits and security concerns?
A vector is not secure anonymisation. It is derived from content and must be protected accordingly. Permissions cannot wait until after retrieval if unauthorised passages have already reached a model. A high similarity score also does not prove factual correctness.
- Protect embeddings and original content under the same security logic
- Carry permission metadata into the index and enforce it during retrieval
- Plan model changes and vector dimensions as migrations
- Evaluate successful, missing and incorrect retrieval separately
Example: finding a policy expressed in different words
An employee asks, 'How long may I work abroad?' The policy uses the term 'temporary cross-border activity'. A keyword-only search may miss it; vector search recognises similar meaning. Metadata limits results to the current country and authorised user group. The answer shows the original passage and version date.
What to remember
Use embeddings for semantic search, but keep the original source, metadata and permissions inseparable from every result. Measured retrieval quality matters more than the database type alone.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed