In brief
- Direct prompt injection comes from a user's input.
- Indirect prompt injection is embedded in emails, websites, documents, images or tool results the system reads.
- RAG, fine-tuning and a strict system prompt do not eliminate the underlying risk.
- Effective protection primarily limits data flow, permissions and the consequences of a successful attack.
How does an external instruction enter the system?
A language model processes instructions and data in the same context. It may therefore interpret a passage that should only be summarised as an instruction to act. The instruction need not be visible: hidden text, encoded characters or image content can also influence processing.
| Stage in the attack path | What happens? | Possible consequence | Effective boundary |
|---|---|---|---|
| 1. Untrusted content | An input, email, web page, file or image contains a visible or hidden instruction. | Business data and an external control instruction enter the model context together. | Label provenance, minimise content and never treat external data as a trusted system rule. |
| 2. Model interpretation | The model gives the external instruction more weight than the actual task. | The summary, selection or next step is manipulated. | Narrow the task, structure output and enforce security-critical rules outside the model. |
| 3. Data or tool request | The manipulated output requests further sources, files, recipients or actions. | Confidential data may be exposed or an unauthorised change prepared. | Least privilege, server-side authorisation, destination and parameter allowlists, and limits. |
| 4. Action or onward use | A downstream system accepts the proposal or a person assumes it is legitimate. | An email, payment, deletion or incorrect decision takes effect. | Approval for consequential actions, visible source context and a logged abort route. |
What can an attack cause?
Impact depends not only on the model but also on connected data and tools. A text-only assistant may return a false answer. An agent with email, file or payment tools could exfiltrate information or prepare a consequential action.
- Disclosure of confidential context or system information
- Manipulated summaries and decisions
- Unauthorised tool calls or data changes
- Circumvention of content and process rules
- Propagation of the attack to downstream systems or people
Which defence layers are useful?
No single detector catches every variant. A robust solution assumes some manipulated content will pass and constrains its influence. Deterministic security controls cannot depend on the model itself.
- Step 1
Mark content as untrusted
Separate external data from internal instructions and use it only for its intended purpose.
- Step 2
Minimise access
Expose only necessary data and narrowly scoped tools.
- Step 3
Validate calls
Check parameters, user permissions and business rules on the server.
- Step 4
Contain consequences
Require confirmation for sensitive or irreversible actions and apply limits.
- Step 5
Test attacks
Evaluate direct, indirect, multilingual and hidden variants regularly.
How are attempts detected and handled in operation?
Detectors, unusual tool sequences and user reports provide signals, including false alarms. The team therefore needs graduated responses: annotate, block, revoke access, isolate a case or temporarily constrain the service.
- Log detected attacks and affected data sources
- Turn recurring patterns and new variants into test cases
- Monitor tool calls for drift from the user's objective
- Trigger incident response when data exposure may have occurred
- Retest controls against bypasses after changes
Example: a manipulated supplier email
An agent is asked to summarise incoming emails. One message contains a hidden instruction to send internal price lists to an external address. The email is marked as untrusted content; the model has no tool for arbitrary file access and may create drafts only for known internal recipients. External sending requires visible approval. The attack may influence a summary but cannot send data without control.
What to remember
Plan for prompt injection as an expected attack path. Separate untrusted content from control and design the system so a successfully manipulated model step can cause only limited harm.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed