Skip to content
AI Agent Eval Harness

ADR-0036: Bounded name-redaction policy

  • Status: Accepted
  • Date: 2026-07-02
  • Decision-makers: Waldemar Szemat

Context and Problem Statement

Section titled “Context and Problem Statement”

The deterministic PII redactor redacts a fixed set of structured identifiers and deliberately does not redact personal names. That exclusion has lived as prose at the redaction module and as a pinned test since the redactor shipped, and the governance posture describes it, but it had never been ratified as a first-class, cited decision.

An external certification review flagged the un-redacted-names behavior as a gap to adjudicate rather than a silent implementation detail. The behavior is deliberate and defensible, but “deliberate” is only auditable if it is an explicit decision with a stated rationale and a named residual, not an un-owned property of the code. This record ratifies the existing policy and records the deferred alternative - a config-gated active name pass - as an owner decision rather than an oversight. It is a documentation ratification: it introduces no code behavior change.

Decision Drivers

Section titled “Decision Drivers”
  • False-positive rate. Reliable personal-name detection without a Named Entity Recognition (NER) model has an unacceptable false-positive rate - a name-shaped pattern would mask ordinary words and medication or clinical terms, degrading the very answers the agent exists to give.
  • Synthetic-only data (ADR-0018). The reference implementation runs on 100% synthetic data with no real patient identities, so an un-redacted first name carries no real-world disclosure risk in this deployment.
  • No new runtime dependency. Adding an NER model to redact names would violate the demo’s zero-new-runtime-dependency and $0 posture for a benefit the synthetic-only invariant already neutralizes.
  • Auditability. The exclusion must be a cited, pinned decision that cannot drift silently, not an implicit gap.
  • The redactor stays a redactor. It is always informational and never a refusal gate (ADR-0005), so a name miss cannot silently block a turn.

Considered Options

Section titled “Considered Options”
  • Ratify the bounded policy - structured identifiers only, names excluded on false-positive-rate and synthetic-only-data grounds - and record the active config-gated name pass as explicitly deferred, off by default and not implemented (chosen).
  • Implement an NER-model-based name-redaction pass now (rejected: it adds a runtime dependency and model-hosting cost that violate the $0 posture, for a benefit the synthetic-only invariant already neutralizes in this deployment).
  • Implement a regex/heuristic name-redaction pass now (rejected: name-shaped patterns have an unacceptable false-positive rate - they would mask ordinary words and medication or clinical terms, degrading the agent’s core answers).

Decision Outcome

Section titled “Decision Outcome”
  • Structured identifiers only. The redactor’s scope is the set of structured identifier kinds: email, phone, national ID (including a US SSN), payment card, the Latin-American national identifiers RUT, CPF, and DNI, and the medical record number, date of birth, and postal address. Each is anchored, bounded, and - where the identifier carries a check digit - structurally validated (a card checksum for payment cards, the national check digit for RUT and CPF), so a matched span is a real identifier, not a look-alike. Personal names are not in this set.
  • Names are excluded on false-positive-rate and synthetic-only-data grounds. Reliable name detection without an NER model produces an unacceptable false-positive rate - masking ordinary words and clinical or medication terms - the demo runs on synthetic-only data (ADR-0018) with no real patient identities, and the documented privacy posture does not require name redaction. A user who writes “Hi, I’m …” has an email or phone in the same message redacted while the first name reaches the model verbatim.
  • The active, config-gated name-redaction pass is deferred. A future config-gated active name pass - off by default - is the sanctioned upgrade path for a real-data deployment, but it is not implemented and not wired behind any setting today. Enabling it is a real-patient-data production concern, not a demo concern; it is a named, out-of-scope option, not a forgotten gap.
  • The policy is pinned and cannot drift silently. The exclusion is asserted by a pinned test and documented at the redaction module; changing it requires revisiting this decision and that test together.

Confirmation

Section titled “Confirmation”
  • A pinned test proves that a name in an otherwise-identifier-bearing message is left verbatim while the structured identifiers around it are redacted.
  • No code changed for this ratification, so the redactor’s matched spans and its always-informational contract are unchanged, and the deterministic, key-less guardrail floor stays green.

Consequences

Section titled “Consequences”

Positive

Section titled “Positive”
  • The deliberate name exclusion is now an explicit, cited decision with a named residual: the external review finding closes as accepted, not open.
  • The rationale - false-positive rate, synthetic-only data, no NER dependency - is recorded once and referenced from the governance posture, so a reviewer sees the same policy from the code, this decision record, and the posture documentation.
  • The upgrade path - a config-gated active name pass for real-data deployments - is a named deferred option, not a forgotten gap.

Negative

Section titled “Negative”
  • Personal names in a real-data deployment would reach the model verbatim under today’s configuration; mitigating that is exactly the deferred config-gated pass, which a production deployment on real patient data must enable.

Neutral

Section titled “Neutral”
  • No code, span-shape, or test changes: this record ratifies existing behavior and adds cross-references only.

Pros and Cons of the Options

Section titled “Pros and Cons of the Options”

Ratify the bounded policy, defer the active name pass (chosen)

Section titled “Ratify the bounded policy, defer the active name pass (chosen)”
  • Good, because it matches the actual risk: on synthetic-only data an un-redacted name carries no real disclosure risk, while a name-shaped matcher would corrupt clinical answers.
  • Good, because it turns an implicit code property into a cited, pinned decision with a named residual and a sanctioned upgrade path.
  • Bad, because it does nothing for a hypothetical real-data deployment until the deferred active pass is enabled - accepted, because that is a production concern.

Implement an NER-model name pass now (rejected)

Section titled “Implement an NER-model name pass now (rejected)”
  • Good, because an NER model would give the highest-recall name detection.
  • Bad, because it adds a runtime dependency and model-hosting cost that violate the $0 and zero-new-runtime-dependency posture, for a benefit the synthetic-only invariant already neutralizes here.

Implement a regex/heuristic name pass now (rejected)

Section titled “Implement a regex/heuristic name pass now (rejected)”
  • Good, because it needs no model.
  • Bad, because name-shaped patterns have an unacceptable false-positive rate - they would mask ordinary words and medication or clinical terms, degrading the agent’s core answers.

More Information

Section titled “More Information”