Drift Detection Plan

Documents the drift detection capabilities of the ai-agent-eval-harness-healthtech reference implementation and the requirements for production-grade drift monitoring. Covers eval-harness-based drift detection, model performance regression, and knowledge base relevance monitoring.

Read alongside the model card and the audit logging plan.

Types of Drift

Drift Type	Definition	Relevance to This System
Data drift	Input data distribution changes over time	User queries may shift in topic, language mix, or complexity; KB content may become stale
Concept drift	Relationship between inputs and desired outputs changes	Medication adherence guidance evolves; clinical guidelines are updated; new medications enter the market
Model drift	Model performance degrades on the same inputs	LLM provider updates model weights; prompt changes affect output quality; embedding model changes affect retrieval
Evaluation drift	Eval corpus no longer represents real usage patterns	Eval cases become unrepresentative as user behaviour changes; adversarial landscape evolves

Current Drift Detection Mechanisms

Eval Harness as Drift Detector

The primary drift detection mechanism in the reference implementation is the eval harness. It is not a traditional drift-detection pipeline, but it serves a similar purpose by detecting performance regressions that could indicate drift.

Mechanism	What It Detects	Frequency	Trigger
Deterministic CI gate	Guardrail regressions (refusal, escalation, citation) on the curated corpus	Every code change	Build failure if any threshold breached
Live eval run	Model performance (faithfulness, hallucination) against a live LLM	Manual / nightly	Threshold breach recorded as a regression
Nightly Promptfoo red-team	Adversarial robustness against OWASP LLM Top 10 plus hand-crafted cases	Nightly	New adversarial pattern discovered
Cost/latency gates	Performance regression in token usage or latency	Every code change	Per-turn budget exceeded
Locale parity gate	Locale-specific performance degradation	Every code change	Locale threshold breach on any dimension

Eval Thresholds

Dimension	Threshold	Drift Signal
Faithfulness	CI-enforced gate (published pass/fail)	A drop below the faithfulness gate suggests model output quality degraded or KB relevance shifted
Hallucination	CI-enforced gate (published pass/fail)	A rise above the hallucination gate suggests the model is generating unsupported content
Refusal correctness	Binary per-case gate	Any miss means a guardrail regression (deterministic; should never happen)
Escalation correctness	Binary per-case gate	Any miss means an escalation regression (deterministic; should never happen)
Citation correctness	Binary per-case gate	Any fabricated citation means citation enforcement regressed
Cost	Per-turn token budget	Exceeding budget suggests prompt or response pattern changed
Latency	Per-turn latency budget	Regression suggests a provider or infrastructure change

Red-Team Adversarial Drift Detection

The Promptfoo nightly red-team serves as a specialised drift detector for adversarial robustness. It exercises the system against:

OWASP LLM Top 10 prompt-injection templates (evolving with Promptfoo updates)
Hand-crafted adversarial cases covering dosing elicitation, diagnosis fishing, system-prompt extraction, role-coercion, and distress disclosure
Adversarial slices in the es-419 and pt-BR eval corpora

When a new adversarial pattern is discovered (either by Promptfoo or by manual investigation), it is folded back into the adversarial seed bank. This ensures the eval corpus evolves with the threat landscape.

Golden Corpus Stability

The eval corpus (315 cases across three locales) is a fixed golden dataset. It provides a stable baseline against which performance is measured. Because the corpus is committed and version-controlled, any performance change on the same corpus version must be due to a change in the system (model, guardrails, retrieval, or prompts), not a change in the test data.

The golden corpus is not a substitute for monitoring real usage patterns. It tests the system against a known, curated set of scenarios; it does not detect whether the system is encountering new types of queries in production.

Current State

The reference implementation detects drift through eval-harness regression, not through continuous monitoring. The mechanisms in place:

Change-gated eval regression: Every code change is tested against the curated corpus. A regression on any threshold fails the CI eval job, surfacing it for investigation (a CI signal, not a hard merge gate - no enforced branch protection). This catches model drift (if the LLM provider updates weights), guardrail drift (if a code change weakens a guardrail), and retrieval drift (if embedding or KB changes affect retrieval quality).
Nightly adversarial testing: Promptfoo exercises the system against evolving adversarial templates. A new bypass technique discovered by the nightly run is a form of adversarial drift detection.
Locale parity enforcement: The eval harness holds all three locales to the same thresholds, detecting locale-specific regressions that could indicate drift in the model’s multilingual capability.
Cost/latency budget enforcement: Per-turn cost and latency budgets catch performance drift that could indicate provider changes, prompt bloat, or infrastructure degradation.

What is not in place:

No automated concept drift detection: The system does not monitor whether its KB content is becoming stale relative to current clinical guidelines. KB card accessed_at dates are recorded but not automatically checked for recency.
No production monitoring: No real-user data is collected or analysed for drift. The eval corpus is the only performance dataset.
No input distribution monitoring: No tracking of query topic distribution, language mix, or complexity trends over time.
No automated re-evaluation cadence: The eval harness runs on-demand (changes) and nightly (red-team); there is no scheduled comprehensive re-evaluation.

Production Path

Production-grade drift detection would require:

Automated drift detection pipeline: Continuous monitoring of model performance against a held-out validation set; statistical tests for distribution shift in input features (query topics, language, complexity); automated alerting when drift exceeds defined thresholds
Performance regression monitoring: Longitudinal tracking of all eval dimensions (faithfulness, hallucination, refusal correctness, escalation correctness, citation correctness); trend analysis with confidence intervals; automated degradation alerts before thresholds are breached
Concept drift detection for KB relevance: Automated monitoring of KB card recency against source publications; scheduled re-crawling of source URLs to detect content changes; alerting when source content diverges from card content
Scheduled re-evaluation cadence: Weekly comprehensive eval runs against the full corpus; monthly live-model eval with updated thresholds; quarterly adversarial assessment with new attack techniques
Input distribution monitoring: Tracking of query topic distribution, language mix, and complexity over time; statistical tests for distribution shift; alerting when real usage patterns diverge significantly from the eval corpus
Model version tracking: Logging of LLM provider model versions on each eval run; correlation of performance changes with model version updates; rollback procedures for provider-initiated model changes
Embedding drift detection: Periodic re-embedding of the KB corpus; comparison of embedding distributions over time; alerting on significant shifts that could indicate embedder model changes
Feedback loop: User feedback collection (implicit and explicit); integration of feedback signals into the eval corpus; continuous improvement of the golden dataset based on production usage patterns