Data-readiness scorecard

Reference documentation: not a medical device

This documentation describes a public reference implementation evaluated on 100% synthetic data. It is a capability and readiness reference, not a compliance certification or legal advice, and it is not a medical device. It is not clinically validated and handles no production PHI.

Data readiness is one of the four causes that sink AI pilots. This page grades the demo’s own data - the synthetic eval corpus and knowledge base - on the dimensions that decide whether data can support a project, and doubles as the scorecard to run on your data before you build.

The demo’s corpus, graded

Every grade below is a real, checkable property of the shipped corpus (see the dataset card), not an aspiration.

Dimension	Grade	Evidence
Provenance	Strong	Every KB card carries a source and an access date; content is paraphrased from public sources (DailyMed, MedlinePlus, WHO EML), never copied.
Licensing	Strong	Redistributable under MIT with a per-source audit; no Data-Use-Agreement corpus is included, by policy.
PII / PHI posture	Strong	100% synthetic; no PHI, no PII, no real record. An identifiability review gate rejects any non-conforming data file.
Locale coverage	Strong	Three locales (en, es-419, pt-BR) held to the same bar, parity-enforced.
Labeling quality	Strong	Gold labels on every case; 100% human-curated after a producer-critic generation loop.
Coverage / volume	Partial	An intentionally narrow eval set - sized to exercise the eval dimensions, not to train. A broader corpus is roadmap.
Freshness	Partial	Source access dates are frozen; refreshing provenance is a manual maintenance task, not an automated gate.
Known bias	Documented	A known lean toward US-English clinical vocabulary; the producer-critic loop partially corrects it and the residual is documented, not claimed solved.

The honest reading: the corpus is strong where it must be (provenance, licensing, privacy, labeling, locale parity) and deliberately limited where a portfolio eval set should be (raw volume), with the limits documented rather than hidden.

Run it on your data

The same dimensions are the checklist for your own project. Grade each one honestly against your real data before building:

• Provenance and licensing - can you use it, and can you prove where each field came from?
• PII / PHI - what is in it, and what must be redacted or synthesized before it touches a model?
• Coverage - does it span the cases the agent will actually meet, including the adversarial and the out-of-scope?
• Labeling - do you have a trustworthy gold standard to measure against?
• Locale and bias - does it represent the population you serve?
• Freshness - how does it stay current, and who owns that?

A dimension graded “gap” here is a pilot-killer surfaced early - far cheaper than discovering it after the build.

Links

• Dataset card - the full corpus documentation.
• PII redaction - the redaction posture.
• Regulatory posture - the compliance boundary.
• Value model - the business case this data supports.