In brief
- Monitoring requires predefined baselines, thresholds and owners for response.
- A traceable case connects user context, model and prompt version, sources, tools, guardrails and result.
- Prompts and responses may contain sensitive data and cannot be copied into logs without assessment.
- Production signals complement fixed evals but do not replace them.
What should be monitored?
An AI application is more than a model call. Good observability connects infrastructure, cost, retrieval, model output, guardrails, tool actions and domain impact. Selection depends on the risks of the use case.
| Signal | Normal range or baseline | Alert and automated response | Owner and permitted log data |
|---|---|---|---|
| Availability, latency and errors | Known response time and technical error rate by user group and process. | Sharp increase: route to a safe fallback, limit new requests or disable the service. | Operations; timestamp, endpoint, status code and correlation ID rather than full content. |
| Quality and domain corrections | Approved evaluation scores and expected correction and refusal rates. | Threshold missed: stop release, roll back model route or expand human review. | Product and domain owners; test-case ID, score and error class, with sensitive content controlled. |
| Retrieval, sources and access | Hit rate, source age and permission revocation within defined deadlines. | Stale source or access failure: block the source and refuse unsupported answers. | Data owner; source ID, version, access decision and timestamp. |
| Tool actions and approvals | Expected action types, ranges, recipients and rejection rate. | Unknown sequence or limit breach: block the call, constrain credentials and assess an incident. | Operations and security; tool, validated parameter class, approval and result without unnecessary payload. |
| Cost and usage | Budget per case, team and model plus expected volume. | Spike or misuse pattern: apply a rate limit, cheaper route or account suspension. | Service owner; token or call volume, cost centre, model and aggregated user group. |
How should logging protect data?
Full inputs help diagnose failures but may contain personal data, secrets or untrusted content. Every log category therefore needs a purpose, minimum fields, access protection and limited retention.
- Use correlation identifiers instead of unnecessary direct identifiers
- Remove or mask values that are not needed; use tokenisation or keyed pseudonyms when records must be linked
- Protect the key or mapping separately and continue to treat pseudonymised values as personal data
- Apply role-based access and tamper-resistant storage
- Set retention according to analysis, security and legal needs
- Inform users transparently about relevant logging
Which signals should trigger an alert?
An alert is useful only if an owner knows what to do. Thresholds are based on expected normal behaviour and distinguish technical disruption, quality degradation, potential attack and disallowed action.
- Step 1
Set a baseline
Document normal use, quality, latency and tool sequences.
- Step 2
Define thresholds
Separate warning, blocking and incident by impact and urgency.
- Step 3
Provide context
Enrich alerts with the affected version, function and safe diagnostic data.
- Step 4
Test the response
Exercise ownership, availability and technical containment regularly.
How does monitoring drive improvement?
Production data reveals new wording and failure patterns. Domain-confirmed cases can be anonymised or recreated synthetically and added to regression tests. This creates a learning loop without treating every user interaction as unreviewed training data.
- Analyse new error classes and affected user groups
- Use expert samples rather than relying only on automated scores
- Compare changes using the same evals before and after release
- Treat near misses and overrides as important risk signals
- Remove metrics and excessive logs that no longer support decisions
Example: a service assistant after a model change
A new model version improves response time but domain corrections increase for French queries. The dashboard separates results by language and version and triggers a quality alert. The team rolls the affected route back, adds the cases to its eval set and reassesses the change. Prompts containing customer data remain tightly restricted and protected.
What to remember
Design monitoring from risk and response backwards. Log enough for traceability and incidents while consistently minimising and protecting sensitive content.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed