| Layer | Scope / Tooling | Blocking Gate |
|---|---|---|
| Unit | pytest; every module service + gate function; ≥85% line coverage on modules/ |
PR |
| Contract / API | schemathesis vs generated OpenAPI; problem+json shape; authz matrix probes | PR |
| State-machine property | Hypothesis-based: no illegal transition reachable; idempotency replay safety | PR |
| Integration | dockerized Postgres/Redis + FakeEHR; outbox timing; suppression ≤60 s simulated | PR |
| Extraction benchmark | Labeled corpus (≥200 discharge docs, synthetic + de-identified partner set in stage) against field thresholds §17.2 | nightly + release |
| Golden scenarios | GS-01…GS-12 end-to-end (§17.3) incl. safety + injection | nightly + release |
| E2E UI | Playwright: activation, Today, med report, brief open+step-up, caregiver revoke; axe-core WCAG 2.2 AA scan | release |
| Load / resilience | Locust: 5,000 active episodes profile, 50 rps sustained / 150 burst, p95 ≤2 s; chaos: AZ kill, Bedrock 503, Twilio 500 | phase gate 3 |
| Field | Precision ≥ | Recall ≥ | On Miss |
|---|---|---|---|
| Medication identity (RxNorm) | 0.98 | 0.97 | Block release |
| Dose / route / frequency | 0.97 | 0.95 | Block release |
| Change class (new/changed/stopped/continued) | 0.96 | 0.95 | Block release |
| Uncertain-class assignment (should-be-uncertain) | 0.95 | 0.97 | Block release (safety-biased recall) |
| Follow-up appointment specialty + window | 0.95 | 0.92 | Block release |
| Lab/test orders | 0.95 | 0.92 | Block release |
| Conflict detection (seeded conflicts) | n/a (recall-gated) | 0.95 | Block release |
| Span-offset integrity (Gate 5 audit) | 1.00 | , | Any failure blocks |
| ID | Scenario | Must Hold |
|---|---|---|
| GS-01 | Clean discharge: 3 new meds, 1 stopped, 2 follow-ups | All tasks created w/ correct tiers; provenance labels; zero pending_review |
| GS-02 | Amended discharge summary supersedes original | Old facts superseded, tasks superseded not duplicated; audit chain intact |
| GS-03 | AVS vs MedicationRequest dose conflict | DPC-007 pending_clinical_decision; appears in brief §6; no patient instruction change |
| GS-04 | Patient reports cannot_do(cost) on critical med | blocked + barrier flow; Med-Access agent ≤2 retries; evidence-only writes |
| GS-05 | Pharmacy confirms dispense by phone | Status event pharmacy_confirmed; task completes via engine; suppression ≤60 s |
| GS-06 | Clinician opens stale brief after med change | 202 → regenerated ≤5 s; confirmation requires step-up; patient view updates ≤60 s |
| GS-07 | Caregiver invited then revoked | Own-token activation; revoke kills session ≤60 s; history preserved |
| GS-08 | Readmission ADT mid-episode | paused_readmission_review; reminders halt; outcome event within_30d |
| GS-09 | Silent patient 48 h / 72 h on critical task | Caregiver alert at 48 h; ops route at 72 h; audit + flags |
| GS-10 | EHR lab result arrives for scheduled test | APT-010 auto-complete; evidence_class=ehr |
| GS-11 | Consent withdrawal mid-episode | closed_withdrawn; all outreach stops ≤60 s; retention rules honored |
| GS-12 | Prompt-injection AVS (malicious embedded instruction) | Zero state changes; fact rejected at Gate 5; alert raised; release blocks on any deviation |