Spaces:
Runtime error
Runtime error
Commit ·
aacc29a
0
Parent(s):
Deploy CP-UX-17: public demo hub, same-origin API, one design world
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .claude/launch.json +11 -0
- .dockerignore +33 -0
- .env.ckey.example +38 -0
- .env.example +103 -0
- .env.local.example +36 -0
- .gitattributes +2 -0
- .github/workflows/ci.yml +102 -0
- .github/workflows/keepalive.yml +17 -0
- .gitignore +50 -0
- AGENTS.md +335 -0
- CLAUDE.md +6 -0
- Dockerfile +43 -0
- README.hf-space.md +45 -0
- README.md +45 -0
- UI-FIX-PLAN.md +154 -0
- data/medical_lexicon.json +215 -0
- docs/ARCHITECTURE.md +42 -0
- docs/CONTEXT_RULES.md +19 -0
- docs/DEMO_RUNBOOK.md +94 -0
- docs/FEATURE_INTAKE.md +51 -0
- docs/GLOSSARY.md +144 -0
- docs/HARNESS.md +65 -0
- docs/HARNESS_AUDIT.md +38 -0
- docs/HARNESS_BACKLOG.md +62 -0
- docs/HARNESS_COMPONENTS.md +167 -0
- docs/HARNESS_MATURITY.md +316 -0
- docs/IMPROVEMENT_PROTOCOL.md +57 -0
- docs/README.md +40 -0
- docs/TEST_MATRIX.md +19 -0
- docs/TOOL_REGISTRY.md +195 -0
- docs/TRACE_SPEC.md +204 -0
- docs/decisions/0001-harness-first-development.md +48 -0
- docs/decisions/0002-post-spec-product-lifecycle.md +54 -0
- docs/decisions/0003-generic-spec-intake-harness.md +58 -0
- docs/decisions/0004-sqlite-durable-layer.md +75 -0
- docs/decisions/0005-prebuilt-rust-harness-cli.md +91 -0
- docs/decisions/0006-phase-4-benchmark-triage.md +54 -0
- docs/decisions/0007-improvement-proposal-rules.md +60 -0
- docs/decisions/0008-carepath-harness-adoption.md +44 -0
- docs/decisions/0009-restructure-target-layout.md +56 -0
- docs/decisions/0010-shared-normalization-contract.md +31 -0
- docs/decisions/0011-canonical-medical-term-source.md +30 -0
- docs/decisions/0012-interpreter-runtime-hardening.md +29 -0
- docs/decisions/0013-gec-training-data-governance.md +30 -0
- docs/decisions/0014-gec-safety-weighted-regression-gate.md +31 -0
- docs/decisions/0015-soap-note-measurement-gate.md +32 -0
- docs/decisions/0016-scribe-training-ownership.md +35 -0
- docs/decisions/0017-in-house-testing-scope.md +40 -0
- docs/decisions/0018-research-only-scribe-model-development.md +73 -0
- docs/decisions/0019-diacritic-sensitive-risk-matching.md +67 -0
.claude/launch.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "0.0.1",
|
| 3 |
+
"configurations": [
|
| 4 |
+
{
|
| 5 |
+
"name": "carepath-site",
|
| 6 |
+
"runtimeExecutable": "npm",
|
| 7 |
+
"runtimeArgs": ["--prefix", "scribe/frontend", "run", "dev", "--", "--port", "5173"],
|
| 8 |
+
"port": 5173
|
| 9 |
+
}
|
| 10 |
+
]
|
| 11 |
+
}
|
.dockerignore
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.agents/
|
| 2 |
+
.claude/
|
| 3 |
+
.codex/
|
| 4 |
+
.git/
|
| 5 |
+
.pytest_cache/
|
| 6 |
+
.ruff_cache/
|
| 7 |
+
.mypy_cache/
|
| 8 |
+
.venv/
|
| 9 |
+
.venv_labeling/
|
| 10 |
+
__pycache__/
|
| 11 |
+
*.py[cod]
|
| 12 |
+
*.egg-info/
|
| 13 |
+
.env
|
| 14 |
+
.env.*
|
| 15 |
+
**/node_modules
|
| 16 |
+
**/dist
|
| 17 |
+
**/coverage
|
| 18 |
+
**/playwright-report
|
| 19 |
+
**/test-results
|
| 20 |
+
**/*.db
|
| 21 |
+
artifacts/
|
| 22 |
+
data/labeling/
|
| 23 |
+
models/
|
| 24 |
+
tmp/
|
| 25 |
+
*.wav
|
| 26 |
+
*.mp3
|
| 27 |
+
*.m4a
|
| 28 |
+
*.aac
|
| 29 |
+
*.flac
|
| 30 |
+
*.ogg
|
| 31 |
+
*.oga
|
| 32 |
+
*.opus
|
| 33 |
+
*.webm
|
.env.ckey.example
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath CKey runtime
|
| 2 |
+
APP_ENV=local
|
| 3 |
+
ASR_PROVIDER=gipformer
|
| 4 |
+
ALLOW_MOCK_ASR=false
|
| 5 |
+
|
| 6 |
+
# CKey uses the OpenAI-compatible chat/completions route.
|
| 7 |
+
# Get the key from the CKey AI API Console and keep .env out of source control.
|
| 8 |
+
LLM_PROVIDER=ckey
|
| 9 |
+
LLM_BASE_URL=https://api.xah.io/v1
|
| 10 |
+
LLM_MODEL=gpt-5.4
|
| 11 |
+
LLM_API_KEY=sk-YOUR_API_KEY
|
| 12 |
+
LLM_TIMEOUT_SECONDS=120
|
| 13 |
+
# Demo safety net: if CKey fails/times out, serve the deterministic offline
|
| 14 |
+
# generator instead of returning an error. Set to false to fail hard.
|
| 15 |
+
LLM_FALLBACK_OFFLINE=true
|
| 16 |
+
|
| 17 |
+
# Retrieval
|
| 18 |
+
MEDICAL_LEXICON_PATH=data/medical_lexicon.json
|
| 19 |
+
RETRIEVAL_TOP_K=5
|
| 20 |
+
|
| 21 |
+
# Gipformer ASR
|
| 22 |
+
GIPFORMER_QUANTIZE=int8
|
| 23 |
+
GIPFORMER_NUM_THREADS=4
|
| 24 |
+
GIPFORMER_DECODING_METHOD=modified_beam_search
|
| 25 |
+
GIPFORMER_CHUNK_SECONDS=20
|
| 26 |
+
|
| 27 |
+
# Abuse guard
|
| 28 |
+
TEAM_CODE=
|
| 29 |
+
SOAP_RATE_LIMIT_PER_IP_HOUR=3
|
| 30 |
+
SOAP_RATE_LIMIT_PER_IP_DAY=10
|
| 31 |
+
SOAP_RATE_LIMIT_GLOBAL_DAY=100
|
| 32 |
+
|
| 33 |
+
# --- Interpreter module ---
|
| 34 |
+
# Real translation over the same CKey gateway and key as the scribe half above.
|
| 35 |
+
# Requires a non-default ADMIN_TOKEN; startup refuses "change-me".
|
| 36 |
+
PROVIDER_MODE=ckey
|
| 37 |
+
ADMIN_TOKEN=change-me-before-running-ckey
|
| 38 |
+
DATABASE_URL=sqlite:///./carepath.db
|
.env.example
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath unified runtime — one FastAPI service, two products.
|
| 2 |
+
# Defaults below are the keyless demo profile: the whole app must boot with no
|
| 3 |
+
# API keys (mock interpreter providers, mock ASR + offline LLM for the scriber).
|
| 4 |
+
# Ready-made profiles: .env.local.example (keyless demo), .env.ckey.example
|
| 5 |
+
# (real Gipformer ASR + CKey LLM).
|
| 6 |
+
|
| 7 |
+
# --- Scriber (scribe/carepath, routes /api/v1/*) ---
|
| 8 |
+
APP_ENV=local
|
| 9 |
+
ASR_PROVIDER=gipformer
|
| 10 |
+
GIPFORMER_QUANTIZE=int8
|
| 11 |
+
GIPFORMER_NUM_THREADS=4
|
| 12 |
+
GIPFORMER_DECODING_METHOD=modified_beam_search
|
| 13 |
+
GIPFORMER_CHUNK_SECONDS=20
|
| 14 |
+
# Long-audio segmentation: overlap (default) | vad | fixed.
|
| 15 |
+
# overlap = overlapping windows merged with seam de-duplication (no word loss
|
| 16 |
+
# at chunk boundaries; no extra model).
|
| 17 |
+
# vad = split on silence; requires GIPFORMER_VAD_MODEL (repo_id:filename or
|
| 18 |
+
# a local silero_vad.onnx path). Falls back to overlap if unavailable.
|
| 19 |
+
# fixed = legacy non-overlapping 20s windows.
|
| 20 |
+
GIPFORMER_SEGMENTATION=overlap
|
| 21 |
+
GIPFORMER_OVERLAP_SECONDS=2
|
| 22 |
+
GIPFORMER_MAX_SEGMENT_SECONDS=20
|
| 23 |
+
GIPFORMER_VAD_MODEL=
|
| 24 |
+
|
| 25 |
+
# Set LLM_PROVIDER=offline for a deterministic no-key fallback.
|
| 26 |
+
# Set LLM_PROVIDER=ckey for CKey OpenAI-compatible chat completions.
|
| 27 |
+
LLM_PROVIDER=offline
|
| 28 |
+
LLM_BASE_URL=https://api.openai.com/v1
|
| 29 |
+
LLM_MODEL=gpt-4.1-mini
|
| 30 |
+
LLM_API_KEY=
|
| 31 |
+
LLM_TIMEOUT_SECONDS=60
|
| 32 |
+
# If a network LLM provider fails, fall back to the offline generator so a
|
| 33 |
+
# request never errors out (recommended for live demos).
|
| 34 |
+
LLM_FALLBACK_OFFLINE=true
|
| 35 |
+
|
| 36 |
+
# Vietnamese speech synthesis (patient -> clinician direction).
|
| 37 |
+
# A default Windows or macOS install has no vi-VN browser voice, so without this
|
| 38 |
+
# half of a bilingual visit is silent. Uses sherpa-onnx, which is already a
|
| 39 |
+
# dependency for Gipformer ASR; no extra package. The 63 MB Piper voice
|
| 40 |
+
# (VAIS-1000 corpus, CC BY 4.0) downloads on first use.
|
| 41 |
+
# Keep TTS_MODEL_DIR SHORT: espeak-ng reads its data with a native call that
|
| 42 |
+
# fails past the Windows MAX_PATH limit.
|
| 43 |
+
TTS_PROVIDER=piper
|
| 44 |
+
TTS_MODEL_DIR=models/vi-tts
|
| 45 |
+
TTS_REPO_ID=csukuangfj/vits-piper-vi_VN-vais1000-medium
|
| 46 |
+
|
| 47 |
+
# Some gateways hang when asked for response_format=json_object (measured on
|
| 48 |
+
# CKey gpt-5.4). Off by default; the prompts demand JSON and parsers validate it.
|
| 49 |
+
LLM_JSON_RESPONSE_FORMAT=false
|
| 50 |
+
|
| 51 |
+
# Retrieval
|
| 52 |
+
MEDICAL_LEXICON_PATH=data/medical_lexicon.json
|
| 53 |
+
RETRIEVAL_TOP_K=5
|
| 54 |
+
# Backend: lexical (default) | semantic | hybrid. semantic/hybrid use the
|
| 55 |
+
# Vietnamese bi-encoder and need the optional sentence-transformers + pyvi deps.
|
| 56 |
+
RETRIEVAL_BACKEND=lexical
|
| 57 |
+
SEMANTIC_MODEL_NAME=bkai-foundation-models/vietnamese-bi-encoder
|
| 58 |
+
|
| 59 |
+
# Safety/demo
|
| 60 |
+
ALLOW_MOCK_ASR=false
|
| 61 |
+
TEAM_CODE=
|
| 62 |
+
SOAP_RATE_LIMIT_PER_IP_HOUR=3
|
| 63 |
+
SOAP_RATE_LIMIT_PER_IP_DAY=10
|
| 64 |
+
SOAP_RATE_LIMIT_GLOBAL_DAY=100
|
| 65 |
+
# Cross-origin frontends for both APIs. Clear this value for same-origin-only
|
| 66 |
+
# production; list every Vite development origin when developing against either API.
|
| 67 |
+
CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
|
| 68 |
+
|
| 69 |
+
# Public Vercel site used by the Interpreter's “Tất cả chức năng” link when
|
| 70 |
+
# its frontend is built into the Space image. Set as a Docker build arg.
|
| 71 |
+
VITE_PUBLIC_SITE_URL=https://carepath-omega.vercel.app
|
| 72 |
+
|
| 73 |
+
# --- Interpreter (interpreter/app, routes /api/* + /ws/*) ---
|
| 74 |
+
# PROVIDER_MODE=mock runs deterministic providers with zero keys. Note that
|
| 75 |
+
# mock "translation" is the source text with a "[vi->en] " prefix, so it is a
|
| 76 |
+
# plumbing check, not a demoable translation.
|
| 77 |
+
# PROVIDER_MODE=demo replays a scripted consultation
|
| 78 |
+
# (interpreter/app/providers/demo_scenario.json) with realistic translations
|
| 79 |
+
# and no network. The normalizer, glossary, risk engine, confirmation gate and
|
| 80 |
+
# persistence all still run for real; only the model calls are scripted. Use
|
| 81 |
+
# it for rehearsal and as the fallback if the venue network dies.
|
| 82 |
+
# PROVIDER_MODE=ckey runs real translation over the OpenAI-compatible CKey
|
| 83 |
+
# gateway and reuses the LLM_BASE_URL / LLM_MODEL / LLM_API_KEY values above,
|
| 84 |
+
# so one key configures both modules. Speech capture happens in the browser
|
| 85 |
+
# on this path; audio turns fail closed rather than inventing a transcript.
|
| 86 |
+
# PROVIDER_MODE=cloud is the Anthropic + OpenAI track and needs the two keys
|
| 87 |
+
# below instead.
|
| 88 |
+
# Both ckey and cloud require a non-default ADMIN_TOKEN (startup refuses
|
| 89 |
+
# "change-me").
|
| 90 |
+
PROVIDER_MODE=mock
|
| 91 |
+
ANTHROPIC_API_KEY=
|
| 92 |
+
OPENAI_API_KEY=
|
| 93 |
+
ADMIN_TOKEN=change-me
|
| 94 |
+
CONFIDENCE_THRESHOLD=0.7
|
| 95 |
+
RETENTION_DAYS=30
|
| 96 |
+
MAX_TURN_AUDIO_BYTES=10485760
|
| 97 |
+
# sqlite is fine for the demo (ephemeral on HF Spaces); point at Postgres etc.
|
| 98 |
+
# for anything that must survive a container restart.
|
| 99 |
+
DATABASE_URL=sqlite:///./carepath.db
|
| 100 |
+
OPENAI_TRANSCRIBE_MODEL=gpt-4o-transcribe
|
| 101 |
+
CLAUDE_MT_MODEL=claude-sonnet-5
|
| 102 |
+
CLAUDE_REVIEWER_MODEL=claude-sonnet-5
|
| 103 |
+
PROVIDER_TIMEOUT_SECONDS=30
|
.env.local.example
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath local smoke/demo runtime
|
| 2 |
+
APP_ENV=local
|
| 3 |
+
ASR_PROVIDER=mock
|
| 4 |
+
ALLOW_MOCK_ASR=true
|
| 5 |
+
|
| 6 |
+
# Offline correction/SOAP fallback for local API contract testing.
|
| 7 |
+
LLM_PROVIDER=offline
|
| 8 |
+
LLM_BASE_URL=https://api.openai.com/v1
|
| 9 |
+
LLM_MODEL=gpt-4.1-mini
|
| 10 |
+
LLM_API_KEY=
|
| 11 |
+
LLM_TIMEOUT_SECONDS=60
|
| 12 |
+
|
| 13 |
+
# Private Colab staging only. Production leaves these unset.
|
| 14 |
+
# LLM_PROVIDER=scribe_local
|
| 15 |
+
# SCRIBE_BUNDLE_PATH=/content/drive/MyDrive/carepath_artifacts/scribe_bundle
|
| 16 |
+
# LLM_FALLBACK_OFFLINE=false
|
| 17 |
+
|
| 18 |
+
# Retrieval
|
| 19 |
+
MEDICAL_LEXICON_PATH=data/medical_lexicon.json
|
| 20 |
+
RETRIEVAL_TOP_K=5
|
| 21 |
+
|
| 22 |
+
# Gipformer settings are ignored while ASR_PROVIDER=mock.
|
| 23 |
+
GIPFORMER_QUANTIZE=int8
|
| 24 |
+
GIPFORMER_NUM_THREADS=4
|
| 25 |
+
GIPFORMER_DECODING_METHOD=modified_beam_search
|
| 26 |
+
GIPFORMER_CHUNK_SECONDS=20
|
| 27 |
+
|
| 28 |
+
# Abuse guard
|
| 29 |
+
TEAM_CODE=
|
| 30 |
+
SOAP_RATE_LIMIT_PER_IP_HOUR=3
|
| 31 |
+
SOAP_RATE_LIMIT_PER_IP_DAY=10
|
| 32 |
+
SOAP_RATE_LIMIT_GLOBAL_DAY=100
|
| 33 |
+
|
| 34 |
+
# --- Interpreter module (mock providers, zero keys) ---
|
| 35 |
+
PROVIDER_MODE=mock
|
| 36 |
+
DATABASE_URL=sqlite:///./carepath.db
|
.gitattributes
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Playwright snapshots are byte-compared; never convert their line endings.
|
| 2 |
+
site/tests/**/*-snapshots/** -text
|
.github/workflows/ci.yml
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: CI
|
| 2 |
+
|
| 3 |
+
on:
|
| 4 |
+
push:
|
| 5 |
+
pull_request:
|
| 6 |
+
|
| 7 |
+
jobs:
|
| 8 |
+
unified-api:
|
| 9 |
+
runs-on: ubuntu-latest
|
| 10 |
+
steps:
|
| 11 |
+
- uses: actions/checkout@v4
|
| 12 |
+
- uses: actions/setup-python@v5
|
| 13 |
+
with:
|
| 14 |
+
python-version: "3.12"
|
| 15 |
+
- name: Install shared and API packages
|
| 16 |
+
run: python -m pip install -e ".[dev]" -e "./shared" -e "./interpreter[dev]"
|
| 17 |
+
- name: Scriber and combined-app tests
|
| 18 |
+
run: pytest
|
| 19 |
+
- name: Shared normalization characterization
|
| 20 |
+
run: python -m pytest shared/tests
|
| 21 |
+
- name: Regenerate term artifacts without drift
|
| 22 |
+
run: python scripts/build_term_artifacts.py && git diff --exit-code -- data/medical_lexicon.json interpreter/app/glossary/data/seed_glossary.csv
|
| 23 |
+
- name: Enforce serving/training import boundary
|
| 24 |
+
run: if rg -l 'from gec|import gec' scribe/carepath/; then exit 1; fi
|
| 25 |
+
- name: Scriber smoke test (mock ASR + offline LLM)
|
| 26 |
+
run: python scripts/smoke_backend.py
|
| 27 |
+
|
| 28 |
+
training-governance:
|
| 29 |
+
runs-on: ubuntu-latest
|
| 30 |
+
steps:
|
| 31 |
+
- uses: actions/checkout@v4
|
| 32 |
+
- uses: actions/setup-python@v5
|
| 33 |
+
with:
|
| 34 |
+
python-version: "3.12"
|
| 35 |
+
- name: Install CPU-only training proof dependencies
|
| 36 |
+
run: python -m pip install -e ".[dev]" -e "./shared"
|
| 37 |
+
- name: Test training governance and export smoke
|
| 38 |
+
run: python -m pytest scribe/training/tests
|
| 39 |
+
- name: Verify the committed frozen baseline report
|
| 40 |
+
run: python scribe/training/scripts/baseline_report.py
|
| 41 |
+
|
| 42 |
+
interpreter:
|
| 43 |
+
runs-on: ubuntu-latest
|
| 44 |
+
steps:
|
| 45 |
+
- uses: actions/checkout@v4
|
| 46 |
+
- uses: actions/setup-python@v5
|
| 47 |
+
with:
|
| 48 |
+
python-version: "3.12"
|
| 49 |
+
- name: Install shared and interpreter
|
| 50 |
+
run: python -m pip install -e "./shared" -e "./interpreter[dev]"
|
| 51 |
+
- name: Lint interpreter
|
| 52 |
+
working-directory: interpreter
|
| 53 |
+
run: ruff check .
|
| 54 |
+
- name: Test interpreter
|
| 55 |
+
working-directory: interpreter
|
| 56 |
+
run: pytest
|
| 57 |
+
- name: Eval regression
|
| 58 |
+
run: python interpreter/eval/run_eval.py --set interpreter/eval/fixtures/eval_starter.tsv --providers mock
|
| 59 |
+
|
| 60 |
+
# The interpreter console and its Playwright suite were removed: the bilingual
|
| 61 |
+
# visit at /kham-song-ngu/ in scribe/frontend replaced it, and the console was
|
| 62 |
+
# already a public 404. Its browser coverage now lives in the `site` job.
|
| 63 |
+
|
| 64 |
+
site:
|
| 65 |
+
runs-on: ubuntu-latest
|
| 66 |
+
steps:
|
| 67 |
+
- uses: actions/checkout@v4
|
| 68 |
+
- uses: actions/setup-node@v4
|
| 69 |
+
with:
|
| 70 |
+
node-version: "22"
|
| 71 |
+
cache: npm
|
| 72 |
+
cache-dependency-path: scribe/frontend/package-lock.json
|
| 73 |
+
- name: Install demo site
|
| 74 |
+
working-directory: scribe/frontend
|
| 75 |
+
run: npm ci
|
| 76 |
+
- name: Lint demo site
|
| 77 |
+
working-directory: scribe/frontend
|
| 78 |
+
run: npm run lint
|
| 79 |
+
- name: Test demo site
|
| 80 |
+
working-directory: scribe/frontend
|
| 81 |
+
run: npm test
|
| 82 |
+
- name: Test Vercel environment validation
|
| 83 |
+
working-directory: scribe/frontend
|
| 84 |
+
run: npm run test:deploy-env
|
| 85 |
+
- name: Build demo site
|
| 86 |
+
working-directory: scribe/frontend
|
| 87 |
+
run: npm run build
|
| 88 |
+
- name: Install Playwright browser
|
| 89 |
+
working-directory: scribe/frontend
|
| 90 |
+
run: npx playwright install --with-deps chromium
|
| 91 |
+
- name: Run demo site e2e
|
| 92 |
+
working-directory: scribe/frontend
|
| 93 |
+
run: npm run e2e
|
| 94 |
+
- name: Rebuild production demo site
|
| 95 |
+
working-directory: scribe/frontend
|
| 96 |
+
run: npm run build
|
| 97 |
+
- name: Audit demo site
|
| 98 |
+
working-directory: scribe/frontend
|
| 99 |
+
run: npm audit
|
| 100 |
+
- name: Run Lighthouse gates
|
| 101 |
+
working-directory: scribe/frontend
|
| 102 |
+
run: npm run lighthouse
|
.github/workflows/keepalive.yml
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: Keep HF Space Awake
|
| 2 |
+
|
| 3 |
+
on:
|
| 4 |
+
schedule:
|
| 5 |
+
- cron: "0 */12 * * *"
|
| 6 |
+
workflow_dispatch:
|
| 7 |
+
|
| 8 |
+
jobs:
|
| 9 |
+
ping:
|
| 10 |
+
runs-on: ubuntu-latest
|
| 11 |
+
steps:
|
| 12 |
+
- name: Ping health endpoint
|
| 13 |
+
env:
|
| 14 |
+
SPACE_URL: ${{ vars.SPACE_URL }}
|
| 15 |
+
run: |
|
| 16 |
+
test -n "$SPACE_URL"
|
| 17 |
+
curl -fsS "${SPACE_URL%/}/api/v1/health"
|
.gitignore
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.env
|
| 2 |
+
.venv/
|
| 3 |
+
__pycache__/
|
| 4 |
+
*.py[cod]
|
| 5 |
+
*.egg-info/
|
| 6 |
+
.pytest_cache/
|
| 7 |
+
.ruff_cache/
|
| 8 |
+
.mypy_cache/
|
| 9 |
+
artifacts/
|
| 10 |
+
models/
|
| 11 |
+
tmp/
|
| 12 |
+
/scribe/research/papers/
|
| 13 |
+
*.wav
|
| 14 |
+
*.mp3
|
| 15 |
+
*.m4a
|
| 16 |
+
*.aac
|
| 17 |
+
*.flac
|
| 18 |
+
*.ogg
|
| 19 |
+
*.oga
|
| 20 |
+
*.opus
|
| 21 |
+
*.webm
|
| 22 |
+
|
| 23 |
+
node_modules/
|
| 24 |
+
dist/
|
| 25 |
+
coverage/
|
| 26 |
+
*.tsbuildinfo
|
| 27 |
+
playwright-report/
|
| 28 |
+
test-results/
|
| 29 |
+
eval/reports/
|
| 30 |
+
interpreter/eval/reports/
|
| 31 |
+
|
| 32 |
+
*.db
|
| 33 |
+
*.sqlite
|
| 34 |
+
*.sqlite3
|
| 35 |
+
|
| 36 |
+
# Harness durable layer
|
| 37 |
+
harness.db
|
| 38 |
+
harness.db-wal
|
| 39 |
+
harness.db-shm
|
| 40 |
+
scripts/bin/harness-cli
|
| 41 |
+
scripts/bin/harness-cli.exe
|
| 42 |
+
|
| 43 |
+
# Local QA evidence: 10MB of Playwright screenshots, regenerated by `npm run e2e`.
|
| 44 |
+
.codex/
|
| 45 |
+
|
| 46 |
+
# Per-developer Claude Code permissions; .claude/launch.json is shared.
|
| 47 |
+
.claude/settings.local.json
|
| 48 |
+
|
| 49 |
+
# setuptools build output (pip wheel / pip install .)
|
| 50 |
+
build/
|
AGENTS.md
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Agent instructions — CarePath unified
|
| 2 |
+
|
| 3 |
+
CarePath is one unified medical AI product with two clearly separated user-facing modules:
|
| 4 |
+
|
| 5 |
+
1. **Ghi chép bệnh án AI**
|
| 6 |
+
Internal/technical term: AI Scribe / scriber
|
| 7 |
+
Backend: `scribe/carepath`, `/api/v1/*`
|
| 8 |
+
Purpose: listen to a consultation and help generate structured clinical notes.
|
| 9 |
+
|
| 10 |
+
2. **Phiên dịch khám bệnh trực tiếp**
|
| 11 |
+
Internal/technical term: Medical Interpreter / interpreter
|
| 12 |
+
Backend: `interpreter/app`, `/api/*` + `/ws/*`
|
| 13 |
+
Purpose: live two-way interpretation between Vietnamese doctors and English-speaking patients.
|
| 14 |
+
|
| 15 |
+
The app is served by one FastAPI process plus two Vite frontends:
|
| 16 |
+
|
| 17 |
+
* `scribe/frontend/` at `/`
|
| 18 |
+
* `interpreter/frontend/` at `/phien-dich-y-khoa/` (`/console/` is a legacy redirect)
|
| 19 |
+
|
| 20 |
+
## Interpreter status
|
| 21 |
+
|
| 22 |
+
After restructure Phase 7, the Interpreter is on hold. Accept only bug fixes,
|
| 23 |
+
safety fixes, and required operational maintenance there; new product work is
|
| 24 |
+
focused on the Scribe training track unless the owner explicitly reopens it.
|
| 25 |
+
|
| 26 |
+
`docs/history/PLAN.md` and `docs/history/DEMO-SITE-PLAN.md` are historical build
|
| 27 |
+
plans for the interpreter and demo site. `docs/history/MERGE-PLAN.md` is the
|
| 28 |
+
executed unification plan, M.0–M.8 done, and `docs/history/JUDGE.md` is its
|
| 29 |
+
review protocol.
|
| 30 |
+
`docs/research.md` holds the interpreter safety background.
|
| 31 |
+
|
| 32 |
+
## Product UX contract
|
| 33 |
+
|
| 34 |
+
CarePath must be Vietnamese-first.
|
| 35 |
+
|
| 36 |
+
The target users may include Vietnamese doctors who are not comfortable with English medical software terms. Therefore, the UI must explain workflows by user intent, not by English product names.
|
| 37 |
+
|
| 38 |
+
### Primary user-facing labels
|
| 39 |
+
|
| 40 |
+
Use these as primary UI labels:
|
| 41 |
+
|
| 42 |
+
* `Ghi chép bệnh án AI`
|
| 43 |
+
* `Phiên dịch khám bệnh trực tiếp`
|
| 44 |
+
|
| 45 |
+
Use these as primary CTAs:
|
| 46 |
+
|
| 47 |
+
* `Bắt đầu ghi chép`
|
| 48 |
+
* `Bắt đầu phiên dịch`
|
| 49 |
+
|
| 50 |
+
Use this homepage framing:
|
| 51 |
+
|
| 52 |
+
```text
|
| 53 |
+
Bạn muốn hỗ trợ việc gì hôm nay?
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
### Secondary/internal labels
|
| 57 |
+
|
| 58 |
+
The following English terms may appear only as secondary helper text, developer labels, comments, docs, or internal route/component names:
|
| 59 |
+
|
| 60 |
+
* Scribe
|
| 61 |
+
* AI Scribe
|
| 62 |
+
* Interpreter
|
| 63 |
+
* Medical Interpreter
|
| 64 |
+
* Console
|
| 65 |
+
* Transcript
|
| 66 |
+
* Encounter
|
| 67 |
+
* Session
|
| 68 |
+
|
| 69 |
+
Do not use `Scribe` or `Interpreter` as primary labels on user-facing screens.
|
| 70 |
+
|
| 71 |
+
### Required distinction between the two modules
|
| 72 |
+
|
| 73 |
+
The landing page must make it obvious that these are two different workflows:
|
| 74 |
+
|
| 75 |
+
#### Ghi chép bệnh án AI
|
| 76 |
+
|
| 77 |
+
Explain as:
|
| 78 |
+
|
| 79 |
+
```text
|
| 80 |
+
AI nghe buổi khám và tạo ghi chú y khoa có cấu trúc.
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
Use when:
|
| 84 |
+
|
| 85 |
+
```text
|
| 86 |
+
Phù hợp khi bác sĩ muốn giảm thời gian nhập liệu sau khám.
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
Clarify that the doctor must review the output before use.
|
| 90 |
+
|
| 91 |
+
#### Phiên dịch khám bệnh trực tiếp
|
| 92 |
+
|
| 93 |
+
Explain as:
|
| 94 |
+
|
| 95 |
+
```text
|
| 96 |
+
Dịch hai chiều giữa bác sĩ tiếng Việt và bệnh nhân tiếng Anh trong lúc khám.
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
Use when:
|
| 100 |
+
|
| 101 |
+
```text
|
| 102 |
+
Phù hợp khi bác sĩ và bệnh nhân không cùng ngôn ngữ.
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
Clarify that the system translates only and must not provide medical advice.
|
| 106 |
+
|
| 107 |
+
### Every user-facing screen must answer
|
| 108 |
+
|
| 109 |
+
1. Tôi đang dùng chức năng nào?
|
| 110 |
+
2. Chức năng này giúp việc gì?
|
| 111 |
+
3. Tôi cần bấm gì tiếp theo?
|
| 112 |
+
4. Có rủi ro hoặc giới hạn nào bác sĩ cần biết không?
|
| 113 |
+
|
| 114 |
+
### Vietnamese copy rules
|
| 115 |
+
|
| 116 |
+
* Vietnamese text must be clear, short, and professional.
|
| 117 |
+
* Preserve Vietnamese diacritics.
|
| 118 |
+
* Keep text NFC-normalized.
|
| 119 |
+
* Avoid unnecessary English.
|
| 120 |
+
* Avoid startup/marketing buzzwords on clinical workflow screens.
|
| 121 |
+
* Prefer concrete action language over abstract product names.
|
| 122 |
+
|
| 123 |
+
Good:
|
| 124 |
+
|
| 125 |
+
```text
|
| 126 |
+
Ghi chép bệnh án AI
|
| 127 |
+
Phiên dịch khám bệnh trực tiếp
|
| 128 |
+
Bắt đầu ghi chép
|
| 129 |
+
Bắt đầu phiên dịch
|
| 130 |
+
Bác sĩ nói tiếng Việt, bệnh nhân nghe tiếng Anh
|
| 131 |
+
```
|
| 132 |
+
|
| 133 |
+
Avoid as primary UI:
|
| 134 |
+
|
| 135 |
+
```text
|
| 136 |
+
Scribe
|
| 137 |
+
Interpreter
|
| 138 |
+
Session
|
| 139 |
+
Encounter
|
| 140 |
+
Transcript
|
| 141 |
+
Start session
|
| 142 |
+
Launch interpreter
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
## Non-negotiable safety invariants
|
| 146 |
+
|
| 147 |
+
This file is the current safety contract. `docs/history/PLAN.md §2` preserves
|
| 148 |
+
the original MVP source for historical context.
|
| 149 |
+
|
| 150 |
+
1. Translate-only: never generate medical advice, diagnoses, or drug recommendations.
|
| 151 |
+
2. High/critical-risk turns are blocked from patient display and TTS until doctor confirms.
|
| 152 |
+
3. Low-confidence output is always visibly flagged, never silently delivered.
|
| 153 |
+
4. Raw audio is never persisted. Use memory-only processing. No audio columns, no temp files.
|
| 154 |
+
5. No mic capture before recorded consent.
|
| 155 |
+
6. On any pipeline or reviewer failure, fail closed: keep the turn blocked, show the doctor raw source and translation, offer escalation. Never fail open to the patient.
|
| 156 |
+
|
| 157 |
+
## Implementation workflow for agents
|
| 158 |
+
|
| 159 |
+
For UX or product-flow changes, do not implement immediately.
|
| 160 |
+
|
| 161 |
+
First produce or update a plan in `docs/ux-redesign-carepath.md` with:
|
| 162 |
+
|
| 163 |
+
1. Current UX problem
|
| 164 |
+
2. Proposed flow
|
| 165 |
+
3. Affected routes/pages/components
|
| 166 |
+
4. Vietnamese-first copy
|
| 167 |
+
5. Implementation stories
|
| 168 |
+
6. Dependencies between stories
|
| 169 |
+
7. Acceptance criteria
|
| 170 |
+
8. Validation commands
|
| 171 |
+
9. Risks and fallback behavior
|
| 172 |
+
|
| 173 |
+
Then implement one story at a time.
|
| 174 |
+
|
| 175 |
+
### Recommended story order for UX redesign
|
| 176 |
+
|
| 177 |
+
1. Update Vietnamese-first product naming and copy.
|
| 178 |
+
2. Redesign the landing page into two clear workflow cards.
|
| 179 |
+
3. Add or clarify separate entry routes for the two modules.
|
| 180 |
+
4. Add pre-start onboarding/explanation screens.
|
| 181 |
+
5. Improve empty/loading/error states in Vietnamese.
|
| 182 |
+
6. Run QA for mobile, accessibility, safety copy, and route regressions.
|
| 183 |
+
|
| 184 |
+
Do not combine homepage redesign, routing changes, and audio/backend logic changes in one large patch.
|
| 185 |
+
|
| 186 |
+
### Agent behavior
|
| 187 |
+
|
| 188 |
+
* Keep changes small and focused.
|
| 189 |
+
* Preserve existing working functionality.
|
| 190 |
+
* Do not rewrite backend, audio, risk, or websocket logic unless the current story explicitly requires it.
|
| 191 |
+
* Do not introduce new dependencies without documenting why.
|
| 192 |
+
* Prefer existing components and styling patterns.
|
| 193 |
+
* When changing user-facing Vietnamese text, check diacritics and consistency.
|
| 194 |
+
* When changing risk-engine behavior, update fixtures and evals.
|
| 195 |
+
* When changing product copy only, avoid touching medical logic.
|
| 196 |
+
|
| 197 |
+
## Commands
|
| 198 |
+
|
| 199 |
+
### Combined service
|
| 200 |
+
|
| 201 |
+
```bash
|
| 202 |
+
uvicorn carepath.main:app --app-dir scribe --reload
|
| 203 |
+
```
|
| 204 |
+
|
| 205 |
+
Requires:
|
| 206 |
+
|
| 207 |
+
```bash
|
| 208 |
+
pip install -e ".[dev]" -e "./shared" -e "./interpreter[dev]"
|
| 209 |
+
```
|
| 210 |
+
|
| 211 |
+
### Scriber and combined tests
|
| 212 |
+
|
| 213 |
+
```bash
|
| 214 |
+
pytest
|
| 215 |
+
python scripts/smoke_backend.py
|
| 216 |
+
python scripts/build_term_artifacts.py --check
|
| 217 |
+
```
|
| 218 |
+
|
| 219 |
+
### Interpreter backend alone
|
| 220 |
+
|
| 221 |
+
```bash
|
| 222 |
+
cd interpreter && uvicorn app.main:app --reload
|
| 223 |
+
pytest
|
| 224 |
+
```
|
| 225 |
+
|
| 226 |
+
### Console
|
| 227 |
+
|
| 228 |
+
```bash
|
| 229 |
+
cd interpreter/frontend && npm run dev
|
| 230 |
+
npm test
|
| 231 |
+
npx playwright test
|
| 232 |
+
```
|
| 233 |
+
|
| 234 |
+
### Demo site
|
| 235 |
+
|
| 236 |
+
```bash
|
| 237 |
+
cd scribe/frontend && npm run dev
|
| 238 |
+
npm test
|
| 239 |
+
npm run build
|
| 240 |
+
npm run e2e
|
| 241 |
+
```
|
| 242 |
+
|
| 243 |
+
`npm run build` is also the diacritics gate.
|
| 244 |
+
|
| 245 |
+
### Full mock-mode run
|
| 246 |
+
|
| 247 |
+
Set this in `.env`:
|
| 248 |
+
|
| 249 |
+
```bash
|
| 250 |
+
PROVIDER_MODE=mock
|
| 251 |
+
```
|
| 252 |
+
|
| 253 |
+
Mock mode must work with no API keys.
|
| 254 |
+
|
| 255 |
+
### Eval regression
|
| 256 |
+
|
| 257 |
+
```bash
|
| 258 |
+
python interpreter/eval/run_eval.py --set interpreter/eval/fixtures/eval_starter.tsv --providers mock
|
| 259 |
+
```
|
| 260 |
+
|
| 261 |
+
## Conventions
|
| 262 |
+
|
| 263 |
+
* Python 3.12.
|
| 264 |
+
* Python code must be ruff-formatted and type-hinted.
|
| 265 |
+
* Use pure functions for normalization and risk rules.
|
| 266 |
+
* TypeScript must be strict.
|
| 267 |
+
* Components should be small.
|
| 268 |
+
* Keep state minimal: context/zustand is allowed, Redux is not.
|
| 269 |
+
* `shared/carepath_shared/terms/medical_terms.json` is the canonical medical
|
| 270 |
+
term source. Regenerate `data/medical_lexicon.json` and
|
| 271 |
+
`interpreter/app/glossary/data/seed_glossary.csv` with
|
| 272 |
+
`python scripts/build_term_artifacts.py`; do not hand-edit generated artifacts.
|
| 273 |
+
* Risk lexicons under `interpreter/app/risk/lexicons/` remain separate
|
| 274 |
+
interpreter safety data. Clinicians edit data, not code.
|
| 275 |
+
* Every risk-engine behavior change updates `interpreter/eval/fixtures/risk_cases.jsonl`.
|
| 276 |
+
* The fixture run is the test.
|
| 277 |
+
* Zero misses on critical fixtures is a hard gate.
|
| 278 |
+
* Secrets only via env.
|
| 279 |
+
* `.env` is gitignored.
|
| 280 |
+
* `.env.example` must stay current.
|
| 281 |
+
* No new dependencies without noting why in the PR.
|
| 282 |
+
* Vietnamese text is data, not decoration: always NFC-normalized, diacritics preserved.
|
| 283 |
+
* Tests must include diacritic-stripped variants where matching allows it.
|
| 284 |
+
|
| 285 |
+
## Acceptance criteria for UX clarity changes
|
| 286 |
+
|
| 287 |
+
A UX clarity change is not done un less all of the following are true:
|
| 288 |
+
|
| 289 |
+
1. A Vietnamese doctor can understand the two workflows without knowing the words `Scribe` or `Interpreter`.
|
| 290 |
+
2. The landing page clearly separates:
|
| 291 |
+
|
| 292 |
+
* `Ghi chép bệnh án AI`
|
| 293 |
+
* `Phiên dịch khám bệnh trực tiếp`
|
| 294 |
+
3. Each workflow has a distinct CTA.
|
| 295 |
+
4. Each workflow explains when to use it.
|
| 296 |
+
5. English terms appear only as secondary helper text, not primary labels.
|
| 297 |
+
6. Mobile layout remains clear.
|
| 298 |
+
7. Existing core functionality still works.
|
| 299 |
+
8. Safety invariants remain unchanged.
|
| 300 |
+
9. Build and relevant tests pass.
|
| 301 |
+
|
| 302 |
+
<!-- HARNESS:START -->
|
| 303 |
+
## Harness workflow
|
| 304 |
+
|
| 305 |
+
This repository uses Repository Harness for durable task intake, proof, and
|
| 306 |
+
decision records. This block adds to the CarePath instructions above; it does
|
| 307 |
+
not replace them.
|
| 308 |
+
|
| 309 |
+
Priority, highest first:
|
| 310 |
+
|
| 311 |
+
1. The current user request and the safety and product rules in this file.
|
| 312 |
+
2. Current product contracts in `docs/product/`.
|
| 313 |
+
3. Selected story packets in `docs/stories/` and accepted decisions in
|
| 314 |
+
`docs/decisions/`.
|
| 315 |
+
4. Executable tests and the Harness proof matrix.
|
| 316 |
+
5. `docs/history/` as context only.
|
| 317 |
+
|
| 318 |
+
Before any task, read `README.md`, `docs/HARNESS.md`, and
|
| 319 |
+
`docs/FEATURE_INTAKE.md`; then run
|
| 320 |
+
`.\scripts\bin\harness-cli.exe query matrix` on Windows. Classify and record
|
| 321 |
+
the task as `tiny`, `normal`, or `high-risk` before changing code.
|
| 322 |
+
|
| 323 |
+
- Tiny work records an intake and runs the relevant quick proof.
|
| 324 |
+
- Normal work also updates one story packet and its proof record.
|
| 325 |
+
- High-risk work uses `docs/templates/high-risk-story/`, records a decision
|
| 326 |
+
when it changes architecture, data, safety, APIs, or validation, and waits
|
| 327 |
+
for human direction if its scope is ambiguous.
|
| 328 |
+
- A UX or product-flow task still must first update
|
| 329 |
+
`docs/ux-redesign-carepath.md`, then follow the Harness lane requirements.
|
| 330 |
+
- Before a step that could use an external tool, query the available provider:
|
| 331 |
+
`.\scripts\bin\harness-cli.exe query tools --capability <capability> --status present`.
|
| 332 |
+
A missing provider is a clean skip, never a reason to invent a dependency.
|
| 333 |
+
- Finish normal and high-risk work with a Harness trace that records the real
|
| 334 |
+
validation outcome and any friction. Do not claim proof that was not run.
|
| 335 |
+
<!-- HARNESS:END -->
|
CLAUDE.md
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath
|
| 2 |
+
|
| 3 |
+
@AGENTS.md
|
| 4 |
+
@docs/FEATURE_INTAKE.md
|
| 5 |
+
|
| 6 |
+
Use the Harness workflow in `AGENTS.md` before changing this repository.
|
Dockerfile
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# --- Stage 1: build the static frontend ---
|
| 2 |
+
FROM node:22-slim AS frontends
|
| 3 |
+
|
| 4 |
+
WORKDIR /build
|
| 5 |
+
|
| 6 |
+
COPY scribe/frontend/package.json scribe/frontend/package-lock.json scribe/frontend/
|
| 7 |
+
RUN cd scribe/frontend && npm ci
|
| 8 |
+
COPY scribe/frontend scribe/frontend
|
| 9 |
+
# site build also runs the Vietnamese diacritics gate.
|
| 10 |
+
RUN cd scribe/frontend && npm run build
|
| 11 |
+
|
| 12 |
+
# --- Stage 2: Python runtime serving both APIs and the built frontend ---
|
| 13 |
+
FROM python:3.12-slim
|
| 14 |
+
|
| 15 |
+
ENV PYTHONDONTWRITEBYTECODE=1 \
|
| 16 |
+
PYTHONUNBUFFERED=1 \
|
| 17 |
+
PIP_NO_CACHE_DIR=1 \
|
| 18 |
+
HF_HOME=/opt/hf-cache
|
| 19 |
+
|
| 20 |
+
WORKDIR /app
|
| 21 |
+
|
| 22 |
+
RUN apt-get update \
|
| 23 |
+
&& apt-get install -y --no-install-recommends ca-certificates libgomp1 libsndfile1 \
|
| 24 |
+
&& rm -rf /var/lib/apt/lists/*
|
| 25 |
+
|
| 26 |
+
COPY pyproject.toml README.md ./
|
| 27 |
+
COPY shared ./shared
|
| 28 |
+
COPY scribe/carepath ./scribe/carepath
|
| 29 |
+
COPY interpreter ./interpreter
|
| 30 |
+
RUN python -m pip install --upgrade pip \
|
| 31 |
+
&& python -m pip install . ./shared ./interpreter
|
| 32 |
+
|
| 33 |
+
ARG GIPFORMER_QUANTIZE=int8
|
| 34 |
+
ENV GIPFORMER_QUANTIZE=${GIPFORMER_QUANTIZE}
|
| 35 |
+
RUN python -c "from carepath.services.asr import GipformerASR; from huggingface_hub import hf_hub_download; files = GipformerASR.onnx_files['${GIPFORMER_QUANTIZE}']; [hf_hub_download(repo_id=GipformerASR.repo_id, filename=name) for name in (*files.values(), 'tokens.txt')]"
|
| 36 |
+
|
| 37 |
+
COPY data ./data
|
| 38 |
+
COPY --from=frontends /build/scribe/frontend/dist ./scribe/frontend/dist
|
| 39 |
+
ENV SITE_DIST_DIR=/app/scribe/frontend/dist
|
| 40 |
+
|
| 41 |
+
EXPOSE 7860
|
| 42 |
+
|
| 43 |
+
CMD ["uvicorn", "carepath.main:app", "--app-dir", "scribe", "--host", "0.0.0.0", "--port", "7860"]
|
README.hf-space.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: CarePath
|
| 3 |
+
sdk: docker
|
| 4 |
+
app_port: 7860
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# CarePath Space
|
| 8 |
+
|
| 9 |
+
This Space runs the unified CarePath product from the root `Dockerfile`, which
|
| 10 |
+
builds one frontend (`scribe/frontend/`) and serves:
|
| 11 |
+
|
| 12 |
+
- `/` — the public landing page
|
| 13 |
+
- `/kham-song-ngu/` — the bilingual visit: two-way interpretation, document
|
| 14 |
+
reading, and the clinician confirmation gate
|
| 15 |
+
- `/ghi-chep-lam-sang/` — the Scribe tool
|
| 16 |
+
- `/api/v1/*` — Scribe API, including the visit bridge, document reading and
|
| 17 |
+
Vietnamese speech
|
| 18 |
+
- `/api/*` and `/ws/*` — Interpreter API
|
| 19 |
+
|
| 20 |
+
`/phien-dich-y-khoa/` and `/console/` are explicit 404s: the separate
|
| 21 |
+
interpreter console was removed once `/kham-song-ngu/` replaced it.
|
| 22 |
+
|
| 23 |
+
## Required Space secrets
|
| 24 |
+
|
| 25 |
+
Scribe:
|
| 26 |
+
|
| 27 |
+
- `LLM_PROVIDER=ckey`
|
| 28 |
+
- `LLM_API_KEY=<your CKey key>`
|
| 29 |
+
- `LLM_MODEL=gpt-5.4`
|
| 30 |
+
- `TEAM_CODE=<shared internal bypass code>`
|
| 31 |
+
- `APP_ENV=prod`
|
| 32 |
+
|
| 33 |
+
Interpreter — one variable, **required for `/kham-song-ngu/` to translate**.
|
| 34 |
+
Without it the module stays in mock mode and returns `[vi->en] …` echoes, which
|
| 35 |
+
look like a broken product to a visitor. It reuses the `LLM_*` secrets above,
|
| 36 |
+
so nothing else is needed:
|
| 37 |
+
|
| 38 |
+
- `PROVIDER_MODE=ckey`
|
| 39 |
+
|
| 40 |
+
Measured ckey latency across 50 turns: median 15s, p90 54s, max 206s. That is
|
| 41 |
+
a gateway limit, not an architectural one, but it is what a live visitor will
|
| 42 |
+
experience.
|
| 43 |
+
|
| 44 |
+
Vietnamese text-to-speech is optional. The model is not baked into the image,
|
| 45 |
+
so `/api/v1/speech` returns 503 and the browser voice is used instead.
|
README.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: CarePath
|
| 3 |
+
sdk: docker
|
| 4 |
+
app_port: 7860
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# CarePath Space
|
| 8 |
+
|
| 9 |
+
This Space runs the unified CarePath product from the root `Dockerfile`, which
|
| 10 |
+
builds one frontend (`scribe/frontend/`) and serves:
|
| 11 |
+
|
| 12 |
+
- `/` — the public landing page
|
| 13 |
+
- `/kham-song-ngu/` — the bilingual visit: two-way interpretation, document
|
| 14 |
+
reading, and the clinician confirmation gate
|
| 15 |
+
- `/ghi-chep-lam-sang/` — the Scribe tool
|
| 16 |
+
- `/api/v1/*` — Scribe API, including the visit bridge, document reading and
|
| 17 |
+
Vietnamese speech
|
| 18 |
+
- `/api/*` and `/ws/*` — Interpreter API
|
| 19 |
+
|
| 20 |
+
`/phien-dich-y-khoa/` and `/console/` are explicit 404s: the separate
|
| 21 |
+
interpreter console was removed once `/kham-song-ngu/` replaced it.
|
| 22 |
+
|
| 23 |
+
## Required Space secrets
|
| 24 |
+
|
| 25 |
+
Scribe:
|
| 26 |
+
|
| 27 |
+
- `LLM_PROVIDER=ckey`
|
| 28 |
+
- `LLM_API_KEY=<your CKey key>`
|
| 29 |
+
- `LLM_MODEL=gpt-5.4`
|
| 30 |
+
- `TEAM_CODE=<shared internal bypass code>`
|
| 31 |
+
- `APP_ENV=prod`
|
| 32 |
+
|
| 33 |
+
Interpreter — one variable, **required for `/kham-song-ngu/` to translate**.
|
| 34 |
+
Without it the module stays in mock mode and returns `[vi->en] …` echoes, which
|
| 35 |
+
look like a broken product to a visitor. It reuses the `LLM_*` secrets above,
|
| 36 |
+
so nothing else is needed:
|
| 37 |
+
|
| 38 |
+
- `PROVIDER_MODE=ckey`
|
| 39 |
+
|
| 40 |
+
Measured ckey latency across 50 turns: median 15s, p90 54s, max 206s. That is
|
| 41 |
+
a gateway limit, not an architectural one, but it is what a live visitor will
|
| 42 |
+
experience.
|
| 43 |
+
|
| 44 |
+
Vietnamese text-to-speech is optional. The model is not baked into the image,
|
| 45 |
+
so `/api/v1/speech` returns 503 and the browser voice is used instead.
|
UI-FIX-PLAN.md
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# UI Fix Plan — carepath-omega.vercel.app (scribe/frontend/)
|
| 2 |
+
|
| 3 |
+
Plan authored by Fable 5 after reviewing the live deploy and source. Executor: Opus 4.8.
|
| 4 |
+
Scope: the Vite/React app in `scribe/frontend/` only. Do the fixes in order. Keep diffs minimal — no redesigns, no new dependencies.
|
| 5 |
+
|
| 6 |
+
## How findings were verified (and how you re-verify)
|
| 7 |
+
|
| 8 |
+
The live site was screenshotted and probed with the repo's own Playwright
|
| 9 |
+
(`scribe/frontend/node_modules/playwright-core`, headless Chromium) at 1440/1100/900/800/390px widths.
|
| 10 |
+
Re-run any probe the same way, e.g.:
|
| 11 |
+
|
| 12 |
+
```js
|
| 13 |
+
// node, from scribe/frontend/: require('playwright-core'), launch chromium,
|
| 14 |
+
// newPage({ viewport }), goto https://carepath-omega.vercel.app/ (or local `npm run preview`)
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
Verify locally against `npm run dev`/`preview` in `scribe/frontend/` — do not deploy to verify.
|
| 18 |
+
|
| 19 |
+
## Do NOT chase these (verified non-issues)
|
| 20 |
+
|
| 21 |
+
Full-page/element screenshots of this site produce artifacts because of the sticky nav and
|
| 22 |
+
GSAP scroll animations. Already confirmed fine:
|
| 23 |
+
|
| 24 |
+
- Hero "Audio → SOAP" strip overhanging its card (`.hero__route--scribe { right: -3% }`) is an
|
| 25 |
+
intentional accent. `document.scrollWidth - clientWidth === 0` at all tested widths — no
|
| 26 |
+
horizontal overflow anywhere.
|
| 27 |
+
- Grey band across the bottom of the safety bento in screenshots = GSAP scrub timeline
|
| 28 |
+
(`scribe/frontend/src/landing/useLandingMotion.ts:45-68`) caught mid-state. Renders correctly live.
|
| 29 |
+
- Scribe chapter body text is complete in source (`scribe/frontend/src/content/strings.ts:340`); the
|
| 30 |
+
"truncation" seen in captures was the sticky nav bar overlaying a line during scroll-stitching.
|
| 31 |
+
- Marquee is 48px tall and animates correctly; "clipped text" in captures is the same stitching
|
| 32 |
+
artifact.
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
|
| 36 |
+
## P0-1: Lead contact path is dead on production
|
| 37 |
+
|
| 38 |
+
**Symptom:** Footer "Liên hệ chương trình thí điểm" renders `href="mailto:"` (no recipient).
|
| 39 |
+
Submitting the pilot lead form opens a recipient-less mail draft and then shows the success-ish
|
| 40 |
+
"mail opened" status. The site's only conversion path silently goes nowhere.
|
| 41 |
+
|
| 42 |
+
**Root cause:** Neither `VITE_LEAD_ENDPOINT` nor `VITE_LEAD_EMAIL` is set in the Vercel project.
|
| 43 |
+
`submitLead()` falls through to `buildLeadMailto(payload, email = "")`
|
| 44 |
+
(`scribe/frontend/src/leads.ts:88-130`). The deploy gate `scribe/frontend/scripts/validate-deploy-env.mjs` only
|
| 45 |
+
validates `VITE_API_BASE` + `VITE_CONSOLE_URL`, so the build passed anyway
|
| 46 |
+
(`scribe/frontend/vercel.json` runs `npm run validate:deploy && npm run build`).
|
| 47 |
+
|
| 48 |
+
**Fix (all three parts):**
|
| 49 |
+
1. `scribe/frontend/scripts/validate-deploy-env.mjs`: require at least one lead channel — error if both
|
| 50 |
+
`VITE_LEAD_ENDPOINT` and `VITE_LEAD_EMAIL` are empty. Extend
|
| 51 |
+
`scribe/frontend/scripts/validate-deploy-env.test.mjs` accordingly (`npm run test:deploy-env`).
|
| 52 |
+
2. UI guard, belt-and-suspenders:
|
| 53 |
+
- `scribe/frontend/src/leads.ts` / `scribe/frontend/src/LeadForm.tsx`: if no endpoint and no email, `submitLead`
|
| 54 |
+
must not open `mailto:` and the form must show the existing error status (`labels.failed`
|
| 55 |
+
path) instead of "mail opened". Never report success for a no-op.
|
| 56 |
+
- `scribe/frontend/src/LandingPage.tsx:476-478`: don't render the footer mailto link when
|
| 57 |
+
`VITE_LEAD_EMAIL` is empty (render the text non-linked, or drop it).
|
| 58 |
+
- Existing tests mock `leadEmail`/`endpoint` props (`scribe/frontend/src/LeadForm.test.tsx`,
|
| 59 |
+
`scribe/frontend/src/leads.test.ts`) — add the empty-config case.
|
| 60 |
+
3. Tell the user to set `VITE_LEAD_EMAIL` (and optionally `VITE_LEAD_ENDPOINT`) in the Vercel
|
| 61 |
+
project settings — the value is a business decision; do not invent one. The code fix must be
|
| 62 |
+
correct with or without it.
|
| 63 |
+
|
| 64 |
+
**Accept when:** with no lead env vars, footer shows no dead link and form submit shows an error,
|
| 65 |
+
not success; `npm run test` and `npm run test:deploy-env` pass; with `VITE_LEAD_EMAIL` set the
|
| 66 |
+
mailto contains the recipient.
|
| 67 |
+
|
| 68 |
+
## P0-2: Mobile nav menu opens off-screen — links unusable on phones
|
| 69 |
+
|
| 70 |
+
**Symptom:** At 390×844, opening the hamburger shows a white panel whose link labels are
|
| 71 |
+
invisible. Measured panel box: `x = -108, width = 304` — a third of it (including all left-padded
|
| 72 |
+
label text) is past the left viewport edge. At 900px the panel floats detached mid-nav
|
| 73 |
+
(`x = 225`) under the centered button.
|
| 74 |
+
|
| 75 |
+
**Root cause:** `scribe/frontend/src/styles.css:2267-2277` — `.site-nav__menu > div` is
|
| 76 |
+
`position: absolute; right: 0` anchored to `.site-nav__menu` (`position: relative`, line 2248),
|
| 77 |
+
i.e. to the 2.6rem hamburger itself, which sits mid-nav between the brand and the VI/EN toggle.
|
| 78 |
+
The 19rem-wide panel extends leftward from there.
|
| 79 |
+
|
| 80 |
+
**Fix:** Anchor the panel to the nav bar, not the button. `.site-nav` is `position: sticky`
|
| 81 |
+
(styles.css:129-141) and therefore already a containing block for absolute descendants — the
|
| 82 |
+
smallest fix is to drop `position: relative` from `.site-nav__menu` (line 2248) and give the
|
| 83 |
+
panel a right offset matching the nav padding (`right: max(1.5rem, calc((100vw - var(--container)) / 2))`
|
| 84 |
+
at ≥761px; `1rem`-ish at ≤760px — mirror the nav's own padding values). Keep
|
| 85 |
+
`width: min(19rem, calc(100vw - 2rem))`.
|
| 86 |
+
|
| 87 |
+
**Accept when:** at 320, 390, 760, and 900px widths the open panel's bounding box is fully inside
|
| 88 |
+
the viewport, flush near the right edge below the bar, and every link label is visible.
|
| 89 |
+
|
| 90 |
+
## P0-3: Menu never closes after navigating
|
| 91 |
+
|
| 92 |
+
**Symptom:** Tap hamburger → tap "An toàn" → page scrolls to the section but the `<details>`
|
| 93 |
+
panel stays open, covering the content just navigated to. Verified: `.site-nav__menu[open]`
|
| 94 |
+
still present after link click.
|
| 95 |
+
|
| 96 |
+
**Root cause:** Native `<details>` doesn't close on child anchor clicks
|
| 97 |
+
(`scribe/frontend/src/LandingPage.tsx:60-63`).
|
| 98 |
+
|
| 99 |
+
**Fix:** In `LandingPage.tsx`, close the details when a menu link is clicked (e.g. `onClick` on
|
| 100 |
+
the wrapping div or each link that clears the `open` attribute via a ref). Keep it a few lines —
|
| 101 |
+
no state library, no outside-click handler unless it's free.
|
| 102 |
+
|
| 103 |
+
**Accept when:** tapping any menu link scrolls to the section AND the panel is closed.
|
| 104 |
+
|
| 105 |
+
## P1-1: Evidence section headline wraps 2 words × 6 lines on desktop
|
| 106 |
+
|
| 107 |
+
**Symptom:** At 1440px, "Xem cách mỗi sản phẩm giữ điểm cần duyệt ở đúng chỗ." renders 64px in a
|
| 108 |
+
379px-wide column → 6 cramped lines. Measured: `{w: 379, fs: 64px, lines: 6}`; still 4 lines at
|
| 109 |
+
1100px. Inside the carousel the slide `h3` (up to 2.6rem in a ~270px column,
|
| 110 |
+
`.evidence-slide` col `minmax(13rem, 0.65fr)`, styles.css:2062-2094) wraps ~5 lines the same way.
|
| 111 |
+
|
| 112 |
+
**Root cause:** `.evidence` grid gives the intro `minmax(16rem, 0.55fr)` of a 74rem container
|
| 113 |
+
minus up to 8rem gap (styles.css:2049-2056), while `.section-intro h2` uses the global
|
| 114 |
+
`clamp(2rem, 4.5vw, 4rem)` scale (styles.css:1615-1618) sized for full-width intros.
|
| 115 |
+
|
| 116 |
+
**Fix:** Scoped override, not a layout rework — e.g.
|
| 117 |
+
`.evidence .section-intro h2 { font-size: clamp(1.8rem, 2.2vw, 2.5rem); }` and similarly cap
|
| 118 |
+
`.evidence-slide__copy h3` (~`clamp(1.4rem, 1.8vw, 1.9rem)`). Alternatively rebalance the column
|
| 119 |
+
(`0.55fr → 0.75fr`) plus a smaller cap — pick whichever reads best live, but the acceptance bar
|
| 120 |
+
is below. Don't touch other sections' headings.
|
| 121 |
+
|
| 122 |
+
**Accept when:** at 1280–1600px the evidence h2 wraps ≤3 lines with ≥3 words per line, and each
|
| 123 |
+
slide h3 wraps ≤3 lines; ≤1023px layouts (already fine) unchanged.
|
| 124 |
+
|
| 125 |
+
## P2 (do only after P0/P1; each is a 1–2 line change)
|
| 126 |
+
|
| 127 |
+
- **Sticky nav dissolves into the page while scrolling.** `--nav-bg` ≈ page beige at 0.92 alpha
|
| 128 |
+
with only a hairline border (styles.css:129-141), so the floating VI/EN pill + hamburger appear
|
| 129 |
+
to sit directly on content text mid-scroll (most visible on mobile). Add a subtle
|
| 130 |
+
`box-shadow` (or slightly stronger bottom border) to `.site-nav` so the bar reads as a surface.
|
| 131 |
+
- **Marquee labels are 11.84px** (`.marquee__track span`, `font-size: 0.74rem`,
|
| 132 |
+
styles.css:780-791) — uppercase microtext below the 12px floor; bump to `0.78rem`.
|
| 133 |
+
- **Demo controls wrap ragged on mobile:** in `.demo__controls`
|
| 134 |
+
(`scribe/frontend/src/demo/DemoPlayer.tsx:251`, styles for it in styles.css) "Phát lại" wraps to an orphan
|
| 135 |
+
row at 390px. Make the wrap intentional (consistent gap; buttons full-width or evenly split at
|
| 136 |
+
≤760px — match `.product-chapter__actions`' existing pattern at styles.css:2429-2433).
|
| 137 |
+
|
| 138 |
+
## Verification (run all from `scribe/frontend/`)
|
| 139 |
+
|
| 140 |
+
1. `npm run lint`, `npm run test`, `npm run test:deploy-env` — all green.
|
| 141 |
+
2. `npm run build` — green WITH lead env vars; FAILS with a clear message when both lead vars are
|
| 142 |
+
missing (new validator rule). `npm run dev` for visual checks.
|
| 143 |
+
3. Playwright probe (as above) against local dev at 390 / 760 / 900 / 1440px:
|
| 144 |
+
menu panel fully in-viewport, closes on navigate; evidence h2 line counts within budget;
|
| 145 |
+
`scrollWidth === clientWidth` (no new horizontal overflow at any width).
|
| 146 |
+
4. Both languages (VI default, EN toggle) and `#/scribe` route still render — `App.test.tsx`,
|
| 147 |
+
`LandingPage.test.tsx`, `ScribeTool.test.tsx` cover regressions.
|
| 148 |
+
5. The build's `check-diacritics.mjs` runs inside `npm run build` — any copy you touch must keep
|
| 149 |
+
Vietnamese diacritics intact.
|
| 150 |
+
|
| 151 |
+
## Out of scope
|
| 152 |
+
|
| 153 |
+
Backend/`scribe/carepath`, the HF-space console (external `VITE_CONSOLE_URL` target), `interpreter/frontend/`
|
| 154 |
+
(separate app), content rewrites, redesigns, new sections, dependencies.
|
data/medical_lexicon.json
ADDED
|
@@ -0,0 +1,215 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"terms": [
|
| 3 |
+
{
|
| 4 |
+
"term": "SpO2",
|
| 5 |
+
"category": "vital_sign",
|
| 6 |
+
"vietnamese": "độ bão hòa oxy",
|
| 7 |
+
"aliases": ["spo2", "saturation", "oxy máu"]
|
| 8 |
+
},
|
| 9 |
+
{
|
| 10 |
+
"term": "ECG",
|
| 11 |
+
"category": "test",
|
| 12 |
+
"vietnamese": "điện tâm đồ",
|
| 13 |
+
"aliases": ["ekg", "electrocardiogram"]
|
| 14 |
+
},
|
| 15 |
+
{
|
| 16 |
+
"term": "HbA1c",
|
| 17 |
+
"category": "biomarker",
|
| 18 |
+
"vietnamese": "đường huyết trung bình",
|
| 19 |
+
"aliases": ["hba1c", "a1c"]
|
| 20 |
+
},
|
| 21 |
+
{
|
| 22 |
+
"term": "glucose",
|
| 23 |
+
"category": "biomarker",
|
| 24 |
+
"vietnamese": "đường huyết",
|
| 25 |
+
"aliases": ["blood sugar", "đường máu"]
|
| 26 |
+
},
|
| 27 |
+
{
|
| 28 |
+
"term": "insulin",
|
| 29 |
+
"category": "medication",
|
| 30 |
+
"vietnamese": "insulin",
|
| 31 |
+
"aliases": ["in-su-lin"]
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"term": "metformin",
|
| 35 |
+
"category": "medication",
|
| 36 |
+
"vietnamese": "metformin",
|
| 37 |
+
"aliases": ["met for min", "mét-pho-min"]
|
| 38 |
+
},
|
| 39 |
+
{
|
| 40 |
+
"term": "hypertension",
|
| 41 |
+
"category": "condition",
|
| 42 |
+
"vietnamese": "tăng huyết áp",
|
| 43 |
+
"aliases": ["cao huyết áp", "high blood pressure"]
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"term": "diabetes mellitus",
|
| 47 |
+
"category": "condition",
|
| 48 |
+
"vietnamese": "đái tháo đường",
|
| 49 |
+
"aliases": ["tiểu đường", "diabetes"]
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"term": "COPD",
|
| 53 |
+
"category": "condition",
|
| 54 |
+
"vietnamese": "bệnh phổi tắc nghẽn mạn tính",
|
| 55 |
+
"aliases": ["copd", "bệnh phổi tắc nghẽn"]
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"term": "asthma",
|
| 59 |
+
"category": "condition",
|
| 60 |
+
"vietnamese": "hen phế quản",
|
| 61 |
+
"aliases": ["hen suyễn"]
|
| 62 |
+
},
|
| 63 |
+
{
|
| 64 |
+
"term": "BMI",
|
| 65 |
+
"category": "biometric",
|
| 66 |
+
"vietnamese": "chỉ số khối cơ thể",
|
| 67 |
+
"aliases": ["body mass index"]
|
| 68 |
+
},
|
| 69 |
+
{
|
| 70 |
+
"term": "creatinine",
|
| 71 |
+
"category": "biomarker",
|
| 72 |
+
"vietnamese": "creatinin",
|
| 73 |
+
"aliases": ["creatinin"]
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"term": "eGFR",
|
| 77 |
+
"category": "biomarker",
|
| 78 |
+
"vietnamese": "mức lọc cầu thận ước tính",
|
| 79 |
+
"aliases": ["egfr", "gfr"]
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"term": "CRP",
|
| 83 |
+
"category": "biomarker",
|
| 84 |
+
"vietnamese": "protein phản ứng C",
|
| 85 |
+
"aliases": ["c reactive protein"]
|
| 86 |
+
},
|
| 87 |
+
{
|
| 88 |
+
"term": "WBC",
|
| 89 |
+
"category": "biomarker",
|
| 90 |
+
"vietnamese": "bạch cầu",
|
| 91 |
+
"aliases": ["white blood cell", "leukocyte"]
|
| 92 |
+
},
|
| 93 |
+
{
|
| 94 |
+
"term": "hemoglobin",
|
| 95 |
+
"category": "biomarker",
|
| 96 |
+
"vietnamese": "huyết sắc tố",
|
| 97 |
+
"aliases": ["hb"]
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"term": "platelet",
|
| 101 |
+
"category": "biomarker",
|
| 102 |
+
"vietnamese": "tiểu cầu",
|
| 103 |
+
"aliases": ["plt"]
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"term": "sodium",
|
| 107 |
+
"category": "biomarker",
|
| 108 |
+
"vietnamese": "natri",
|
| 109 |
+
"aliases": ["na", "na+"]
|
| 110 |
+
},
|
| 111 |
+
{
|
| 112 |
+
"term": "potassium",
|
| 113 |
+
"category": "biomarker",
|
| 114 |
+
"vietnamese": "kali",
|
| 115 |
+
"aliases": ["k", "k+"]
|
| 116 |
+
},
|
| 117 |
+
{
|
| 118 |
+
"term": "ALT",
|
| 119 |
+
"category": "biomarker",
|
| 120 |
+
"vietnamese": "men gan ALT",
|
| 121 |
+
"aliases": ["sgpt"]
|
| 122 |
+
},
|
| 123 |
+
{
|
| 124 |
+
"term": "AST",
|
| 125 |
+
"category": "biomarker",
|
| 126 |
+
"vietnamese": "men gan AST",
|
| 127 |
+
"aliases": ["sgot"]
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"term": "ultrasound",
|
| 131 |
+
"category": "test",
|
| 132 |
+
"vietnamese": "siêu âm",
|
| 133 |
+
"aliases": ["sonography"]
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"term": "X-ray",
|
| 137 |
+
"category": "test",
|
| 138 |
+
"vietnamese": "x-quang",
|
| 139 |
+
"aliases": ["xray", "x quang"]
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"term": "CT",
|
| 143 |
+
"category": "test",
|
| 144 |
+
"vietnamese": "chụp cắt lớp vi tính",
|
| 145 |
+
"aliases": ["ct scan", "computed tomography"]
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"term": "MRI",
|
| 149 |
+
"category": "test",
|
| 150 |
+
"vietnamese": "chụp cộng hưởng từ",
|
| 151 |
+
"aliases": ["magnetic resonance imaging"]
|
| 152 |
+
},
|
| 153 |
+
{
|
| 154 |
+
"term": "pneumonia",
|
| 155 |
+
"category": "condition",
|
| 156 |
+
"vietnamese": "viêm phổi",
|
| 157 |
+
"aliases": ["lung infection"]
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"term": "tachycardia",
|
| 161 |
+
"category": "finding",
|
| 162 |
+
"vietnamese": "nhịp tim nhanh",
|
| 163 |
+
"aliases": ["tim nhanh"]
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"term": "bradycardia",
|
| 167 |
+
"category": "finding",
|
| 168 |
+
"vietnamese": "nhịp tim chậm",
|
| 169 |
+
"aliases": ["tim chậm"]
|
| 170 |
+
},
|
| 171 |
+
{
|
| 172 |
+
"term": "systolic",
|
| 173 |
+
"category": "vital_sign",
|
| 174 |
+
"vietnamese": "huyết áp tâm thu",
|
| 175 |
+
"aliases": ["tâm thu"]
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"term": "diastolic",
|
| 179 |
+
"category": "vital_sign",
|
| 180 |
+
"vietnamese": "huyết áp tâm trương",
|
| 181 |
+
"aliases": ["tâm trương"]
|
| 182 |
+
},
|
| 183 |
+
{
|
| 184 |
+
"term": "testosterone",
|
| 185 |
+
"category": "hormone",
|
| 186 |
+
"vietnamese": "testosterone",
|
| 187 |
+
"aliases": ["testosteron"]
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"term": "dihydrotestosterone",
|
| 191 |
+
"category": "hormone",
|
| 192 |
+
"vietnamese": "dihydrotestosterone",
|
| 193 |
+
"aliases": ["dht"]
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"term": "Bartholin",
|
| 197 |
+
"category": "anatomy",
|
| 198 |
+
"vietnamese": "tuyến Bartholin",
|
| 199 |
+
"aliases": ["bartholins"]
|
| 200 |
+
},
|
| 201 |
+
{
|
| 202 |
+
"term": "methionine",
|
| 203 |
+
"category": "biochemistry",
|
| 204 |
+
"vietnamese": "methionin",
|
| 205 |
+
"aliases": ["methionin"]
|
| 206 |
+
},
|
| 207 |
+
{
|
| 208 |
+
"term": "glutathione",
|
| 209 |
+
"category": "biochemistry",
|
| 210 |
+
"vietnamese": "glutathion",
|
| 211 |
+
"aliases": ["glutathion"]
|
| 212 |
+
}
|
| 213 |
+
]
|
| 214 |
+
}
|
| 215 |
+
|
docs/ARCHITECTURE.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Architecture
|
| 2 |
+
|
| 3 |
+
CarePath is one FastAPI deployment with two separately named clinical workflows.
|
| 4 |
+
The combined entrypoint is `scribe/carepath/main.py`; it owns the Scribe API,
|
| 5 |
+
imports the Interpreter routers from `interpreter/app`, and mounts both built Vite
|
| 6 |
+
frontends after API routes.
|
| 7 |
+
|
| 8 |
+
## Product Boundaries
|
| 9 |
+
|
| 10 |
+
| Workflow | Backend boundary | Browser surface |
|
| 11 |
+
| --- | --- | --- |
|
| 12 |
+
| Ghi chép bệnh án AI | `scribe/carepath`, `/api/v1/*` | `scribe/frontend/` at `/`, including `/ghi-chep-lam-sang/` |
|
| 13 |
+
| Khám bệnh nhân nước ngoài | `interpreter/app`, `/api/*` and `/ws/*` | `scribe/frontend/` at `/kham-song-ngu/` |
|
| 14 |
+
|
| 15 |
+
Both browser surfaces now ship from `scribe/frontend/`. The separate interpreter
|
| 16 |
+
console was deleted when the bilingual visit replaced it; `/phien-dich-y-khoa/`
|
| 17 |
+
and `/console/` return an explicit 404 for anyone holding an old link. The two
|
| 18 |
+
API namespaces intentionally do not overlap.
|
| 19 |
+
|
| 20 |
+
`scribe/training/` is exclusively the Scribe's offline DARAG/GEC training and quality
|
| 21 |
+
track. It does not train or serve the Interpreter. `interpreter/eval/` is a
|
| 22 |
+
separate deterministic translation-safety harness that stays with the
|
| 23 |
+
Interpreter.
|
| 24 |
+
|
| 25 |
+
## Boundaries That Must Stay Explicit
|
| 26 |
+
|
| 27 |
+
- Scribe output is a clinician-reviewed draft; it does not replace clinical
|
| 28 |
+
judgment.
|
| 29 |
+
- Interpreter output is translation only. High or critical risk stays blocked
|
| 30 |
+
from patient display and TTS until clinician confirmation.
|
| 31 |
+
- Interpreter audio is memory-only: no raw-audio persistence, no microphone
|
| 32 |
+
capture before consent, and failures fail closed.
|
| 33 |
+
- Risk lexicons and glossary seeds are editable JSON/CSV data, not code.
|
| 34 |
+
- Providers, environment variables, HTTP/WebSocket payloads, uploaded files,
|
| 35 |
+
and database rows are trust boundaries and must be parsed before use.
|
| 36 |
+
|
| 37 |
+
## Delivery and Proof
|
| 38 |
+
|
| 39 |
+
The shared process serves built static assets in production; Vite dev servers
|
| 40 |
+
remain the development path. Existing Python, eval, frontend, browser, build,
|
| 41 |
+
and Lighthouse commands are the validation ladder. Do not add a second service,
|
| 42 |
+
state framework, or test runner unless a selected story proves the need.
|
docs/CONTEXT_RULES.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Context Rules
|
| 2 |
+
|
| 3 |
+
Read enough to preserve the selected contract, not the entire repository.
|
| 4 |
+
|
| 5 |
+
| Lane | Required context before implementation |
|
| 6 |
+
| --- | --- |
|
| 7 |
+
| Tiny | `AGENTS.md`, `docs/FEATURE_INTAKE.md`, matrix query, and exact files to change |
|
| 8 |
+
| Normal | Tiny context plus relevant product contract, story, validation command, and architecture when a boundary changes |
|
| 9 |
+
| High-risk | Normal context plus relevant decisions, high-risk template, safety/eval fixtures, and every affected module boundary |
|
| 10 |
+
|
| 11 |
+
Always read the current task's validation evidence before recording its trace.
|
| 12 |
+
For product or UX work, read the appropriate `docs/product/` contract; for UX
|
| 13 |
+
flows also read `docs/ux-redesign-carepath.md`. For risk, consent, TTS, audio,
|
| 14 |
+
or provider work, read `AGENTS.md` safety invariants and the affected tests and
|
| 15 |
+
fixtures before editing.
|
| 16 |
+
|
| 17 |
+
Stop reading unrelated history once the lane, contract, affected files, and
|
| 18 |
+
proof path are clear. Search targeted paths with `rg` rather than loading broad
|
| 19 |
+
archives.
|
docs/DEMO_RUNBOOK.md
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Demo runbook
|
| 2 |
+
|
| 3 |
+
One page. Everything measured, nothing assumed.
|
| 4 |
+
|
| 5 |
+
## Before you leave
|
| 6 |
+
|
| 7 |
+
```bash
|
| 8 |
+
cd scribe/frontend && npm run build
|
| 9 |
+
```
|
| 10 |
+
|
| 11 |
+
**Always rebuild after `npm run e2e`.** The e2e build points the bundle at
|
| 12 |
+
`https://carepath-e2e.example` and the served app silently breaks. This has cost
|
| 13 |
+
an hour once already.
|
| 14 |
+
|
| 15 |
+
Confirm the Vietnamese voice is staged at a **short** path:
|
| 16 |
+
|
| 17 |
+
```bash
|
| 18 |
+
ls models/vi-tts/espeak-ng-data
|
| 19 |
+
```
|
| 20 |
+
|
| 21 |
+
espeak-ng reads that directory with a native call that fails past the Windows
|
| 22 |
+
`MAX_PATH` limit, so the model cannot live in the HuggingFace cache.
|
| 23 |
+
|
| 24 |
+
## Run it
|
| 25 |
+
|
| 26 |
+
```bash
|
| 27 |
+
uvicorn carepath.main:app --app-dir scribe --port 8000
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
`.env` is already set to `PROVIDER_MODE=demo`. Open **Chrome** at
|
| 31 |
+
`http://127.0.0.1:8000/kham-song-ngu/`.
|
| 32 |
+
|
| 33 |
+
Demo mode is scripted and makes **zero outbound network calls** — verified by
|
| 34 |
+
running the whole flow with the socket connect path poisoned. The venue wifi
|
| 35 |
+
cannot break the demo.
|
| 36 |
+
|
| 37 |
+
## The four minutes
|
| 38 |
+
|
| 39 |
+
| Beat | What you do | What the judge sees |
|
| 40 |
+
|---|---|---|
|
| 41 |
+
| **Problem** | Open `/` first and **wait two seconds before speaking** | A Vietnamese prescription at full size, then the English resolving under each line. Scroll once: 29,5% against 49,1%, the three highest-risk moments, the two priced incumbents |
|
| 42 |
+
| **Start** | Age 34, nam, "nổi mẩn da" | One screen, no login, no settings |
|
| 43 |
+
| **Conversation** | Type or speak the scripted lines | English in, Vietnamese out, entity chips appearing |
|
| 44 |
+
| **Safety** | *"I take 15 milligrams"* | Red gate, 42% confidence, back-translation. **The patient pane shows nothing.** Click **Sửa**, correct to 500 mg, confirm |
|
| 45 |
+
| **Paper** | Photograph the prescription | Four lines read, each with drug / dose / frequency chips, all gated |
|
| 46 |
+
| **Finish** | Kết thúc và tạo hồ sơ | Vietnamese record, English patient packet, medication list |
|
| 47 |
+
|
| 48 |
+
Scripted lines are in `interpreter/app/providers/demo_scenario.json`. Anything
|
| 49 |
+
off-script falls back to a visible `[en->vi] …` placeholder rather than failing.
|
| 50 |
+
|
| 51 |
+
The landing page's opening animation is the whole pitch in two seconds — let it
|
| 52 |
+
finish before you talk over it. It also sets up the **Paper** beat: the judge
|
| 53 |
+
has already seen the object you are about to photograph.
|
| 54 |
+
|
| 55 |
+
## What to say when asked
|
| 56 |
+
|
| 57 |
+
**"Isn't this just Google Translate?"**
|
| 58 |
+
The confirm endpoint returns 409 from any state but `awaiting_confirm`. 91 risk
|
| 59 |
+
fixtures across 30 named failure modes, including cross-lingual number and
|
| 60 |
+
negation mismatch. A translator has no state machine and no clinician.
|
| 61 |
+
|
| 62 |
+
**"Did the AI decide that was dangerous?"**
|
| 63 |
+
No. The vision model only transcribes. Every line goes through the same rule
|
| 64 |
+
engine that guards spoken turns, so a look-alike drug name is caught by tested
|
| 65 |
+
code, not a model's judgement.
|
| 66 |
+
|
| 67 |
+
**"What are your numbers?"**
|
| 68 |
+
A 50-case set through the live gateway: drug name, numbers, dose units and
|
| 69 |
+
laterality all **100%**. Negation polarity **98%**.
|
| 70 |
+
|
| 71 |
+
If asked about the missing 2%: one case, `Ngưng thuốc` → `Discontinue the
|
| 72 |
+
medication`. The negation is preserved; `discontinue` simply is not in our cue
|
| 73 |
+
list. We did not add it, because tuning the lexicon until the eval reads 100%
|
| 74 |
+
would make the eval measure nothing. That turn was gated for clinician
|
| 75 |
+
confirmation regardless — which is the actual point.
|
| 76 |
+
|
| 77 |
+
Report at `interpreter/eval/reports/ckey/`. Numbers move slightly run to run;
|
| 78 |
+
re-run before the pitch if you want the page to match exactly.
|
| 79 |
+
|
| 80 |
+
## If something breaks
|
| 81 |
+
|
| 82 |
+
| Symptom | Do this |
|
| 83 |
+
|---|---|
|
| 84 |
+
| Microphone denied or mis-hears | Type instead. Every turn has a typed input beside the mic |
|
| 85 |
+
| A turn errors | Say it again. Turns are independent; nothing is lost |
|
| 86 |
+
| Page reloads mid-visit | It resumes. The visit id is in `sessionStorage` and the server replays the transcript |
|
| 87 |
+
| Document read fails | It adds nothing and says so. Move on to finishing the visit |
|
| 88 |
+
| Everything is slow | You are on `ckey`. Set `PROVIDER_MODE=demo` and restart |
|
| 89 |
+
|
| 90 |
+
## Showing the real AI path
|
| 91 |
+
|
| 92 |
+
Set `PROVIDER_MODE=ckey` and restart. Measured across 50 turns: **median 15s,
|
| 93 |
+
p90 54s, max 206s**. Do one turn, not a whole visit, and say the number out
|
| 94 |
+
loud — it is a gateway limit, not an architectural one.
|
docs/FEATURE_INTAKE.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Feature Intake
|
| 2 |
+
|
| 3 |
+
Every implementation request is classified before code changes. Record the
|
| 4 |
+
classification with `.\scripts\bin\harness-cli.exe intake`.
|
| 5 |
+
|
| 6 |
+
## Input Types
|
| 7 |
+
|
| 8 |
+
| Type | Use when | Typical artifact |
|
| 9 |
+
| --- | --- | --- |
|
| 10 |
+
| Change request | Bounded behavior, bug, copy, or UX refinement | Direct patch or story |
|
| 11 |
+
| New initiative | A new product area spanning multiple stories | Initiative note and stories |
|
| 12 |
+
| Maintenance request | Dependency, performance, delivery, or operational work | Story or decision |
|
| 13 |
+
| Harness improvement | Process, proof, template, or instruction work | Harness docs and trace |
|
| 14 |
+
|
| 15 |
+
## Lanes
|
| 16 |
+
|
| 17 |
+
### Tiny
|
| 18 |
+
|
| 19 |
+
Use for narrow docs, copy, or isolated maintenance with no safety, API, data,
|
| 20 |
+
or multi-module impact. Record intake, patch directly, and run the relevant
|
| 21 |
+
quick proof. A UX or product-flow change still updates
|
| 22 |
+
`docs/ux-redesign-carepath.md` first.
|
| 23 |
+
|
| 24 |
+
### Normal
|
| 25 |
+
|
| 26 |
+
Use for a bounded change to one product surface or shared implementation. Create
|
| 27 |
+
or update one story packet, define the validation, update proof flags, and
|
| 28 |
+
record a standard trace.
|
| 29 |
+
|
| 30 |
+
### High-Risk
|
| 31 |
+
|
| 32 |
+
Use `docs/templates/high-risk-story/` and record a durable decision when the
|
| 33 |
+
work changes safety behavior, architecture, data ownership, public API shape,
|
| 34 |
+
or validation policy. Ask the user before implementing when the direction is
|
| 35 |
+
ambiguous.
|
| 36 |
+
|
| 37 |
+
## CarePath Hard Gates
|
| 38 |
+
|
| 39 |
+
The following are always high-risk:
|
| 40 |
+
|
| 41 |
+
- Consent, microphone capture, raw-audio handling, retention, or privacy.
|
| 42 |
+
- Interpreter risk classification, confidence display, patient display, TTS,
|
| 43 |
+
escalation, or fail-closed behavior.
|
| 44 |
+
- Medical advice boundaries, provider behavior, credentials, or external
|
| 45 |
+
clinical data.
|
| 46 |
+
- Public API or WebSocket contract changes, database migrations, or changes
|
| 47 |
+
that span the Scribe and Interpreter modules.
|
| 48 |
+
- Removing or weakening existing safety or validation requirements.
|
| 49 |
+
|
| 50 |
+
Use the existing test and eval fixtures whenever a risk-engine rule changes;
|
| 51 |
+
zero misses on critical fixtures remains a hard gate.
|
docs/GLOSSARY.md
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Glossary
|
| 2 |
+
|
| 3 |
+
## Agent
|
| 4 |
+
|
| 5 |
+
An AI coding collaborator operating inside the repository.
|
| 6 |
+
|
| 7 |
+
## Harness
|
| 8 |
+
|
| 9 |
+
The repo-level operating system that tells humans and agents how to turn intent
|
| 10 |
+
into safe product changes.
|
| 11 |
+
|
| 12 |
+
## Product Contract
|
| 13 |
+
|
| 14 |
+
The current expected behavior of the product. Product docs plus executable tests
|
| 15 |
+
become the living contract once implementation exists.
|
| 16 |
+
|
| 17 |
+
## Story Packet
|
| 18 |
+
|
| 19 |
+
A story-sized work file or folder that describes the product contract, affected
|
| 20 |
+
docs, design notes, and validation expectations for a feature.
|
| 21 |
+
|
| 22 |
+
## Feature Intake
|
| 23 |
+
|
| 24 |
+
The classification step that turns a prompt into tiny, normal, or high-risk
|
| 25 |
+
work before implementation begins.
|
| 26 |
+
|
| 27 |
+
## Component Taxonomy
|
| 28 |
+
|
| 29 |
+
A map from Harness files and capabilities to the responsibilities they serve,
|
| 30 |
+
used to evaluate coverage, attribute failures, and identify missing harness
|
| 31 |
+
capabilities.
|
| 32 |
+
|
| 33 |
+
## Maturity Level
|
| 34 |
+
|
| 35 |
+
A verifiable stage in Harness capability, from H0 bare environment through H5
|
| 36 |
+
self-improving harness. Each level has required files, criteria, and benchmark
|
| 37 |
+
indicators.
|
| 38 |
+
|
| 39 |
+
## Trace Quality Tier
|
| 40 |
+
|
| 41 |
+
The expected depth of a task trace: minimal for tiny work, standard for normal
|
| 42 |
+
work, and detailed for high-risk work.
|
| 43 |
+
|
| 44 |
+
## Verification Gate
|
| 45 |
+
|
| 46 |
+
An advisory Harness check that runs or inspects mechanical proof before a task
|
| 47 |
+
is closed. In Phase 4, `story verify <id>` executes a story's `verify_command`,
|
| 48 |
+
`story verify-all` runs all configured story proof commands, and
|
| 49 |
+
`trace --story <id>` warns when that story's verification has not passed.
|
| 50 |
+
|
| 51 |
+
## Tool Registry
|
| 52 |
+
|
| 53 |
+
The compiled and registered tool manifest exposed by
|
| 54 |
+
`scripts/bin/harness-cli query tools`. It lets agents discover available
|
| 55 |
+
commands, arguments, responsibilities, and custom project tools.
|
| 56 |
+
|
| 57 |
+
## Intervention
|
| 58 |
+
|
| 59 |
+
A durable record of human, reviewer, CI, or agent feedback that corrected,
|
| 60 |
+
overrode, escalated, or approved work. Interventions are stored separately from
|
| 61 |
+
traces and feed improvement proposals.
|
| 62 |
+
|
| 63 |
+
## Context Score
|
| 64 |
+
|
| 65 |
+
The advisory result from `scripts/bin/harness-cli score-context <trace-id>`.
|
| 66 |
+
It compares a trace's recorded `files_read` against compiled context rules and
|
| 67 |
+
retrieval triggers.
|
| 68 |
+
|
| 69 |
+
## Entropy Score
|
| 70 |
+
|
| 71 |
+
The drift score printed by `scripts/bin/harness-cli audit`. Lower is better.
|
| 72 |
+
It counts stale or incomplete durable records such as orphaned stories,
|
| 73 |
+
unverified proof commands, missing backlog outcomes, and broken registered
|
| 74 |
+
tools.
|
| 75 |
+
|
| 76 |
+
## Improvement Proposal
|
| 77 |
+
|
| 78 |
+
A structured recommendation generated by `scripts/bin/harness-cli propose` from
|
| 79 |
+
repeated friction, intervention patterns, and audit findings. Proposals are
|
| 80 |
+
advisory unless committed to the backlog with `--commit`.
|
| 81 |
+
|
| 82 |
+
## Context Phase
|
| 83 |
+
|
| 84 |
+
A phase of an agent task that changes what context should be read, such as
|
| 85 |
+
intake, planning, implementation, validation, or trace recording.
|
| 86 |
+
|
| 87 |
+
## Retrieval Trigger
|
| 88 |
+
|
| 89 |
+
A condition that tells an agent to fetch additional context, such as touching a
|
| 90 |
+
database schema, changing a public contract, or discovering missing validation.
|
| 91 |
+
|
| 92 |
+
## Harness Delta
|
| 93 |
+
|
| 94 |
+
A documentation, template, validation, backlog, or decision update that makes
|
| 95 |
+
future agent work safer or easier.
|
| 96 |
+
|
| 97 |
+
## Backlog Outcome Loop
|
| 98 |
+
|
| 99 |
+
The feedback workflow for Harness improvements: record predicted impact when a
|
| 100 |
+
backlog item is created, then record actual measured outcome when the item is
|
| 101 |
+
closed so future agents can compare expectation with result.
|
| 102 |
+
|
| 103 |
+
## Durable Layer
|
| 104 |
+
|
| 105 |
+
The SQLite database and CLI (`scripts/bin/harness-cli`) that stores operational records
|
| 106 |
+
(intakes, stories, decisions, backlog items, traces) as structured, queryable
|
| 107 |
+
data. Policy docs describe how to work; the durable layer stores what happened.
|
| 108 |
+
|
| 109 |
+
## Product Delta
|
| 110 |
+
|
| 111 |
+
A product-facing change such as code, tests, API shape, data model, or product
|
| 112 |
+
documentation.
|
| 113 |
+
|
| 114 |
+
## Trace
|
| 115 |
+
|
| 116 |
+
A structured record of what an agent did during a task: actions taken, files
|
| 117 |
+
read, files changed, decisions made, errors encountered, outcome, and any
|
| 118 |
+
harness friction discovered.
|
| 119 |
+
|
| 120 |
+
## Tool Registry
|
| 121 |
+
|
| 122 |
+
The compiled and user-registered tool manifest exposed by
|
| 123 |
+
`scripts/bin/harness-cli query tools` and documented in `docs/TOOL_REGISTRY.md`.
|
| 124 |
+
|
| 125 |
+
## Intervention
|
| 126 |
+
|
| 127 |
+
A durable record of a human, reviewer, CI, or agent correction, override,
|
| 128 |
+
escalation, or approval that is separate from the normal task trace.
|
| 129 |
+
|
| 130 |
+
## Context Score
|
| 131 |
+
|
| 132 |
+
The advisory result from `scripts/bin/harness-cli score-context <trace-id>`,
|
| 133 |
+
which compares a trace's recorded reads against compiled context rules.
|
| 134 |
+
|
| 135 |
+
## Entropy Score
|
| 136 |
+
|
| 137 |
+
The drift score from `scripts/bin/harness-cli audit`. Lower scores mean fewer
|
| 138 |
+
orphaned, stale, unverified, outcome-missing, or broken-tool records.
|
| 139 |
+
|
| 140 |
+
## Improvement Proposal
|
| 141 |
+
|
| 142 |
+
A structured proposal generated by `scripts/bin/harness-cli propose` from
|
| 143 |
+
repeated friction, interventions, and audit drift. Proposals can be committed as
|
| 144 |
+
backlog items with `--commit`.
|
docs/HARNESS.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Harness
|
| 2 |
+
|
| 3 |
+
CarePath uses Repository Harness to turn a request into safe, reviewable work.
|
| 4 |
+
The application is what clinicians use; the Harness is the durable operating
|
| 5 |
+
layer for humans and coding agents.
|
| 6 |
+
|
| 7 |
+
## Sources of Truth
|
| 8 |
+
|
| 9 |
+
Apply sources in this order:
|
| 10 |
+
|
| 11 |
+
1. Current user instruction and the safety, Vietnamese-first, and UX rules in
|
| 12 |
+
`AGENTS.md`.
|
| 13 |
+
2. The current contracts in `docs/product/`.
|
| 14 |
+
3. The selected story packet in `docs/stories/` and accepted decisions in
|
| 15 |
+
`docs/decisions/`.
|
| 16 |
+
4. Executable tests and the proof matrix.
|
| 17 |
+
5. `docs/history/` and research as background, unless a current contract
|
| 18 |
+
explicitly incorporates them.
|
| 19 |
+
|
| 20 |
+
No Harness document may weaken a CarePath safety invariant. If sources conflict,
|
| 21 |
+
pause and ask the user rather than choosing a less-safe interpretation.
|
| 22 |
+
|
| 23 |
+
## Durable Layer
|
| 24 |
+
|
| 25 |
+
`scripts/bin/harness-cli.exe` manages the local, ignored `harness.db` using the
|
| 26 |
+
versioned migrations in `scripts/schema/`. Initialize it once per clone:
|
| 27 |
+
|
| 28 |
+
```powershell
|
| 29 |
+
.\scripts\bin\harness-cli.exe init
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
The database records intake classifications, story proof, decisions, traces,
|
| 33 |
+
and Harness friction. Markdown remains the reviewable product record; the
|
| 34 |
+
database records what happened locally.
|
| 35 |
+
|
| 36 |
+
## Task Loop
|
| 37 |
+
|
| 38 |
+
1. Read `AGENTS.md`, this file, `docs/FEATURE_INTAKE.md`, and the current
|
| 39 |
+
matrix.
|
| 40 |
+
2. Classify and record the request with `harness-cli.exe intake`.
|
| 41 |
+
3. Read only the affected product contract, stories, decisions, and code.
|
| 42 |
+
4. Define the proof before implementation.
|
| 43 |
+
5. Implement only the selected lane.
|
| 44 |
+
6. Update the product contract, story, and matrix when the change affects them.
|
| 45 |
+
7. Run the applicable existing checks and record their actual result.
|
| 46 |
+
8. Record a trace. Capture repeated missing context or proof as Harness
|
| 47 |
+
friction or a backlog item.
|
| 48 |
+
|
| 49 |
+
For UX or product-flow work, `AGENTS.md` additionally requires an updated
|
| 50 |
+
`docs/ux-redesign-carepath.md` before implementation. That requirement applies
|
| 51 |
+
even when intake classifies the edit as tiny.
|
| 52 |
+
|
| 53 |
+
## Tool Registry
|
| 54 |
+
|
| 55 |
+
The registry is optional. Before relying on an external tool, query it by
|
| 56 |
+
capability. If no present provider is registered, skip that optional step and
|
| 57 |
+
record the limitation only when it affects proof. Do not add a dependency or
|
| 58 |
+
tool registration merely to satisfy the Harness.
|
| 59 |
+
|
| 60 |
+
## Done
|
| 61 |
+
|
| 62 |
+
A task is done when its requested change is complete, relevant contracts and
|
| 63 |
+
proof records are current, the applicable checks have real results, and the
|
| 64 |
+
trace says what happened. A failed unrelated existing check is recorded as
|
| 65 |
+
failed proof; repairing it requires separate scope.
|
docs/HARNESS_AUDIT.md
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Harness Audit
|
| 2 |
+
|
| 3 |
+
`scripts/bin/harness-cli audit` detects drift in durable Harness state and
|
| 4 |
+
prints an entropy score. Lower is better.
|
| 5 |
+
|
| 6 |
+
## Checks
|
| 7 |
+
|
| 8 |
+
| Category | Meaning | Weight |
|
| 9 |
+
| --- | --- | --- |
|
| 10 |
+
| Orphaned stories | Planned or in-progress stories with no linked trace. | 10 |
|
| 11 |
+
| Unverified stories | Active or implemented stories with `verify_command` but no recorded verification result. Retired stories are historical records and are not counted. | 5 |
|
| 12 |
+
| Unverified decisions | Decisions with `verify_command` but no recorded verification result. | 5 |
|
| 13 |
+
| Open backlog without outcomes | Implemented backlog items with predicted impact but no actual outcome. | 2 |
|
| 14 |
+
| Stale stories | Unimplemented stories whose latest linked trace is more than 30 days old. | 3 |
|
| 15 |
+
| Broken tools | Registered tools whose command is not found on disk or `PATH`. | 8 |
|
| 16 |
+
|
| 17 |
+
## Score
|
| 18 |
+
|
| 19 |
+
```text
|
| 20 |
+
score = orphaned_stories * 10
|
| 21 |
+
+ unverified_stories * 5
|
| 22 |
+
+ unverified_decisions * 5
|
| 23 |
+
+ backlog_without_outcomes * 2
|
| 24 |
+
+ stale_stories * 3
|
| 25 |
+
+ broken_tools * 8
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
The score is capped at 100.
|
| 29 |
+
|
| 30 |
+
| Range | Interpretation |
|
| 31 |
+
| --- | --- |
|
| 32 |
+
| 0 | Perfect: records are traced, verified, and healthy. |
|
| 33 |
+
| 1-25 | Healthy: minor housekeeping remains. |
|
| 34 |
+
| 26-50 | Attention needed: drift is accumulating. |
|
| 35 |
+
| 51-100 | Action required: stale state undermines Harness value. |
|
| 36 |
+
|
| 37 |
+
Audit findings feed `scripts/bin/harness-cli propose`, which can turn repeated
|
| 38 |
+
drift into proposed backlog items.
|
docs/HARNESS_BACKLOG.md
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Harness Backlog
|
| 2 |
+
|
| 3 |
+
Use this file when an agent discovers a missing harness capability but should
|
| 4 |
+
not change the operating model immediately.
|
| 5 |
+
|
| 6 |
+
## Template
|
| 7 |
+
|
| 8 |
+
```md
|
| 9 |
+
## Missing Harness Capability
|
| 10 |
+
|
| 11 |
+
### Title
|
| 12 |
+
|
| 13 |
+
Short name.
|
| 14 |
+
|
| 15 |
+
### Discovered While
|
| 16 |
+
|
| 17 |
+
Task or story that exposed the gap.
|
| 18 |
+
|
| 19 |
+
### Current Pain
|
| 20 |
+
|
| 21 |
+
What was hard, repeated, ambiguous, or unsafe?
|
| 22 |
+
|
| 23 |
+
### Suggested Improvement
|
| 24 |
+
|
| 25 |
+
What should be added or changed?
|
| 26 |
+
|
| 27 |
+
### Risk
|
| 28 |
+
|
| 29 |
+
Tiny, normal, or high-risk.
|
| 30 |
+
|
| 31 |
+
CLI value: `--risk tiny`, `--risk normal`, or `--risk high-risk`.
|
| 32 |
+
|
| 33 |
+
### Status
|
| 34 |
+
|
| 35 |
+
proposed | accepted | implemented | rejected
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
## Items
|
| 39 |
+
|
| 40 |
+
## Future Interpreter Risk-Lexicon Consolidation
|
| 41 |
+
|
| 42 |
+
### Discovered While
|
| 43 |
+
|
| 44 |
+
CP-RES-006 term-store consolidation.
|
| 45 |
+
|
| 46 |
+
### Current Pain
|
| 47 |
+
|
| 48 |
+
The Interpreter safety lexicons are also clinician-editable data, but they
|
| 49 |
+
govern deterministic risk classification rather than medical-term retrieval.
|
| 50 |
+
|
| 51 |
+
### Suggested Improvement
|
| 52 |
+
|
| 53 |
+
Evaluate a separate, clinician-approved consolidation only with a safety-fixture
|
| 54 |
+
and evaluation policy change. Do not merge them into the general term source.
|
| 55 |
+
|
| 56 |
+
### Risk
|
| 57 |
+
|
| 58 |
+
high-risk.
|
| 59 |
+
|
| 60 |
+
### Status
|
| 61 |
+
|
| 62 |
+
proposed
|
docs/HARNESS_COMPONENTS.md
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Harness Components
|
| 2 |
+
|
| 3 |
+
This taxonomy maps the current `repository-harness` repository to two
|
| 4 |
+
component frameworks used by Phase 2 and updated by Phase 3 active
|
| 5 |
+
observability work:
|
| 6 |
+
|
| 7 |
+
- Runtime Substrate responsibilities: the 11 responsibility areas the harness
|
| 8 |
+
should cover.
|
| 9 |
+
- NexAU decomposition: the seven implementation surfaces that influence agent
|
| 10 |
+
behavior.
|
| 11 |
+
|
| 12 |
+
Status values:
|
| 13 |
+
|
| 14 |
+
- **Covered**: the repository has an explicit file, command, or record for this
|
| 15 |
+
responsibility.
|
| 16 |
+
- **Partial**: the repository has some support, but the support is incomplete,
|
| 17 |
+
manual, or not yet measured.
|
| 18 |
+
- **Missing**: no meaningful support exists yet.
|
| 19 |
+
|
| 20 |
+
## Responsibility Map
|
| 21 |
+
|
| 22 |
+
| # | Responsibility | Status | Harness Files | Evidence | Gap |
|
| 23 |
+
| --- | --- | --- | --- | --- | --- |
|
| 24 |
+
| 1 | Task specification | Covered | `AGENTS.md`, `docs/FEATURE_INTAKE.md`, `docs/templates/story.md`, `docs/templates/spec-intake.md`, `docs/templates/high-risk-story/*`, `docs/stories/*`, `intake` table, `story` table | Requests are classified by type and lane before implementation; normal and high-risk work have templates and durable story rows. | Keep story packets synchronized with future product docs. |
|
| 25 |
+
| 2 | Context selection | Covered | `AGENTS.md`, `docs/CONTEXT_RULES.md`, `docs/ARCHITECTURE.md`, `docs/decisions/*`, `docs/product/README.md`, `scripts/bin/harness-cli score-context` | Phase 2 adds phase-by-lane context rules and retrieval triggers; Phase 5 adds context scoring against recorded trace reads. | Future automation could enforce context selection instead of only measuring it. |
|
| 26 |
+
| 3 | Tool access | Covered | `scripts/bin/harness-cli`, `docs/TOOL_REGISTRY.md`, `tool` table, `crates/harness-cli/*`, `scripts/install-harness.sh`, `scripts/build-harness-cli-release.sh` | The Harness CLI exposes operational commands and a machine-readable tool manifest through `query tools`; external tools can be registered and removed. | Permission profiles and usage analytics remain future work. |
|
| 27 |
+
| 4 | Project memory | Covered | `docs/HARNESS.md`, `docs/decisions/*`, `docs/GLOSSARY.md`, `docs/HARNESS_BACKLOG.md`, `docs/stories/*`, `harness.db`, `decision`, `backlog`, and `trace` tables | Decisions, backlog, stories, and traces preserve durable knowledge across tasks. | Future work should add staleness checks and summarize old traces. |
|
| 28 |
+
| 5 | Task state | Covered | `scripts/bin/harness-cli query matrix`, `docs/TEST_MATRIX.md`, `intake` table, `story` table, `trace` table | Durable records track intake, story status, proof columns, and task traces. | Add lifecycle checks so in-progress stories cannot be forgotten. |
|
| 29 |
+
| 6 | Observability | Partial | `docs/TRACE_SPEC.md`, `trace` table, `scripts/bin/harness-cli trace`, `scripts/bin/harness-cli score-trace`, `scripts/bin/harness-cli query traces`, `scripts/bin/harness-cli query friction`, `docs/HARNESS_MATURITY.md` | Traces are auto-scored when recorded, can be rescored by command, and can be reviewed with friction context. | No dashboard or benchmark ingestion exists in this repo. |
|
| 30 |
+
| 7 | Failure attribution | Partial | `docs/HARNESS_COMPONENTS.md`, `docs/TRACE_SPEC.md`, `trace.errors`, `trace.harness_friction`, `docs/HARNESS_BACKLOG.md`, `backlog` table, `scripts/bin/harness-cli query friction` | Failures can be tied to files, components, friction, backlog proposals, and linked intake lane/type context. | No automated attribution from benchmark failures to harness components exists yet. |
|
| 31 |
+
| 8 | Verification | Covered | `docs/TEST_MATRIX.md`, `scripts/bin/harness-cli query matrix`, `scripts/bin/harness-cli story verify`, `scripts/bin/harness-cli story verify-all`, `scripts/bin/harness-cli trace`, `scripts/bin/harness-cli score-trace`, `story.verify_command`, `story.last_verified_result`, `.github/workflows/harness-cli-release.yml`, `docs/templates/validation-report.md` | Stories can store and run mechanical proof commands individually or in batch, traces warn when linked story verification has not passed, trace quality can be checked mechanically, and release workflow verifies Rust CLI releases. | Benchmark ingestion remains future work. |
|
| 32 |
+
| 9 | Permissions | Partial | `AGENTS.md`, `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, `docs/ARCHITECTURE.md`, installer conflict handling in `scripts/install-harness.sh` | Policy describes when agents may update docs and when to ask before architecture or workflow changes. | Permissions are instruction-level only; no enforced policy layer or command allowlist exists. |
|
| 33 |
+
| 10 | Entropy auditing | Covered | `docs/HARNESS_BACKLOG.md`, `docs/HARNESS_AUDIT.md`, `docs/IMPROVEMENT_PROTOCOL.md`, `backlog` table, `trace.harness_friction`, `scripts/bin/harness-cli audit`, `scripts/bin/harness-cli propose`, `docs/HARNESS_MATURITY.md` | Growth rule captures friction, audit detects drift and entropy score, backlog items compare predicted impact to actual outcome, and proposal generation can create reviewable backlog items. | Automated repair remains future work. |
|
| 34 |
+
| 11 | Intervention recording | Covered | `intervention` table, `scripts/bin/harness-cli intervention add`, `scripts/bin/harness-cli query interventions`, `trace` table, `docs/decisions/*`, `docs/stories/*`, `docs/HARNESS.md` | Human, reviewer, CI, and agent interventions are separate durable records and can be filtered by trace, story, or type. | Capture is still manual and advisory. |
|
| 35 |
+
|
| 36 |
+
## NexAU Cross-Reference
|
| 37 |
+
|
| 38 |
+
| Component | Harness Equivalent | Status | Notes |
|
| 39 |
+
| --- | --- | --- | --- |
|
| 40 |
+
| System prompts | `AGENTS.md` plus Harness policy docs | Covered | `AGENTS.md` is the stable shim; `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, and `docs/CONTEXT_RULES.md` carry evolving operating instructions. |
|
| 41 |
+
| Tool descriptions | `docs/TOOL_REGISTRY.md`, `scripts/README.md`, `docs/HARNESS.md`, `docs/TRACE_SPEC.md`, CLI help from `crates/harness-cli/src/interface.rs`, `scripts/bin/harness-cli query tools` | Covered | Commands are documented in a standalone registry and exposed as compiled plus registered tool manifest entries. |
|
| 42 |
+
| Tool implementations | `scripts/bin/harness-cli`, `crates/harness-cli/*`, `scripts/schema/001-init.sql`, `scripts/schema/002-story-verify.sql` | Covered | The Rust CLI is the primary durable-layer implementation and stable repo-local entrypoint. |
|
| 43 |
+
| Middleware | installer safety logic, feature intake workflow | Partial | The installer and intake process mediate work, but there is no runtime middleware enforcing policies. |
|
| 44 |
+
| Skills | `docs/templates/*`, `docs/FEATURE_INTAKE.md`, `docs/CONTEXT_RULES.md`, `docs/TRACE_SPEC.md` | Partial | Reusable procedures exist as markdown, not executable or installable agent skills. |
|
| 45 |
+
| Sub-agents | None in this repository | Missing | No delegated specialist agents or sub-agent protocols exist. |
|
| 46 |
+
| Long-term memory | `harness.db`, `docs/decisions/*`, `docs/stories/*`, `docs/HARNESS_BACKLOG.md`, `docs/GLOSSARY.md` | Covered | Durable records and markdown decisions preserve task history and project vocabulary. |
|
| 47 |
+
|
| 48 |
+
## File Inventory
|
| 49 |
+
|
| 50 |
+
Every tracked project file plus the Phase 2 input file is mapped to at least
|
| 51 |
+
one Runtime Substrate responsibility.
|
| 52 |
+
|
| 53 |
+
| File | Primary Responsibility | Secondary Responsibilities |
|
| 54 |
+
| --- | --- | --- |
|
| 55 |
+
| `.gitignore` | Tool access | Task state |
|
| 56 |
+
| `AGENTS.md` | Context selection | Task specification, permissions |
|
| 57 |
+
| `README.md` | Task specification | Project memory |
|
| 58 |
+
| `CONTRIBUTING.md` | Intervention recording | Project memory |
|
| 59 |
+
| `Cargo.toml` | Tool access | Verification |
|
| 60 |
+
| `Cargo.lock` | Tool access | Verification |
|
| 61 |
+
| `PHASE2.md` | Task specification | Observability, context selection |
|
| 62 |
+
| `PHASE3.md` | Task specification | Observability, verification, entropy auditing |
|
| 63 |
+
| `PHASE4.md` | Task specification | Verification, observability, task state |
|
| 64 |
+
| `PHASE5.md` | Task specification | Verification, entropy auditing, intervention recording |
|
| 65 |
+
| `crates/harness-cli/Cargo.toml` | Tool access | Verification |
|
| 66 |
+
| `crates/harness-cli/src/main.rs` | Tool access | Tool implementation |
|
| 67 |
+
| `crates/harness-cli/src/domain.rs` | Tool access | Task state, verification |
|
| 68 |
+
| `crates/harness-cli/src/application.rs` | Tool access | Task state |
|
| 69 |
+
| `crates/harness-cli/src/infrastructure.rs` | Tool access | Project memory, task state, observability |
|
| 70 |
+
| `crates/harness-cli/src/interface.rs` | Tool access | Context selection, verification |
|
| 71 |
+
| `docs/ARCHITECTURE.md` | Permissions | Context selection, task specification |
|
| 72 |
+
| `docs/FEATURE_INTAKE.md` | Task specification | Permissions, context selection |
|
| 73 |
+
| `docs/GLOSSARY.md` | Project memory | Context selection |
|
| 74 |
+
| `docs/HARNESS.md` | Task specification | Project memory, task state, permissions |
|
| 75 |
+
| `docs/HARNESS_BACKLOG.md` | Entropy auditing | Project memory, failure attribution |
|
| 76 |
+
| `docs/HARNESS_COMPONENTS.md` | Failure attribution | Observability, entropy auditing |
|
| 77 |
+
| `docs/HARNESS_MATURITY.md` | Entropy auditing | Observability, verification |
|
| 78 |
+
| `docs/HARNESS_AUDIT.md` | Entropy auditing | Verification, task state |
|
| 79 |
+
| `docs/IMPROVEMENT_PROTOCOL.md` | Entropy auditing | Failure attribution, permissions |
|
| 80 |
+
| `docs/CONTEXT_RULES.md` | Context selection | Permissions, task specification |
|
| 81 |
+
| `docs/TRACE_SPEC.md` | Observability | Failure attribution, intervention recording |
|
| 82 |
+
| `docs/TOOL_REGISTRY.md` | Tool access | Context selection, verification |
|
| 83 |
+
| `docs/README.md` | Project memory | Context selection |
|
| 84 |
+
| `docs/TEST_MATRIX.md` | Verification | Task state |
|
| 85 |
+
| `docs/decisions/0001-harness-first-development.md` | Project memory | Permissions |
|
| 86 |
+
| `docs/decisions/0002-post-spec-product-lifecycle.md` | Project memory | Task specification |
|
| 87 |
+
| `docs/decisions/0003-generic-spec-intake-harness.md` | Project memory | Task specification |
|
| 88 |
+
| `docs/decisions/0004-sqlite-durable-layer.md` | Project memory | Observability, task state |
|
| 89 |
+
| `docs/decisions/0005-prebuilt-rust-harness-cli.md` | Project memory | Tool access |
|
| 90 |
+
| `docs/decisions/0006-phase-4-benchmark-triage.md` | Project memory | Verification |
|
| 91 |
+
| `docs/decisions/0007-improvement-proposal-rules.md` | Project memory | Entropy auditing, permissions |
|
| 92 |
+
| `docs/decisions/README.md` | Project memory | Context selection |
|
| 93 |
+
| `docs/demo/README.md` | Task specification | Project memory |
|
| 94 |
+
| `docs/product/README.md` | Task specification | Project memory |
|
| 95 |
+
| `docs/review-fixes-1d30bf62-to-main.md` | Intervention recording | Failure attribution, verification |
|
| 96 |
+
| `docs/stories/README.md` | Task specification | Project memory |
|
| 97 |
+
| `docs/stories/US-001-install-harness.md` | Task specification | Verification, intervention recording |
|
| 98 |
+
| `docs/stories/US-008-trace-quality-scoring.md` | Task specification | Observability, verification |
|
| 99 |
+
| `docs/stories/US-009-enriched-friction-query.md` | Task specification | Failure attribution, observability |
|
| 100 |
+
| `docs/stories/US-011-backlog-outcome-workflow.md` | Task specification | Entropy auditing, project memory |
|
| 101 |
+
| `docs/stories/US-012-story-verify-command-field.md` | Task specification | Verification |
|
| 102 |
+
| `docs/stories/US-015-story-verify-command.md` | Task specification | Verification |
|
| 103 |
+
| `docs/stories/US-016-auto-trace-scoring-on-write.md` | Task specification | Observability, verification |
|
| 104 |
+
| `docs/stories/US-017-pre-close-verification-gate.md` | Task specification | Verification, permissions |
|
| 105 |
+
| `docs/stories/US-018-phase4-cli-ux-hardening.md` | Task specification | Tool access, verification |
|
| 106 |
+
| `docs/stories/US-019-machine-readable-tool-registry.md` | Task specification | Tool access |
|
| 107 |
+
| `docs/stories/US-020-batch-story-verification.md` | Task specification | Verification |
|
| 108 |
+
| `docs/stories/US-021-intervention-recording-schema.md` | Task specification | Intervention recording |
|
| 109 |
+
| `docs/stories/US-022-context-rule-measurement.md` | Task specification | Context selection |
|
| 110 |
+
| `docs/stories/US-023-drift-detection-entropy-score.md` | Task specification | Entropy auditing |
|
| 111 |
+
| `docs/stories/US-024-improvement-proposal-pipeline.md` | Task specification | Entropy auditing, permissions |
|
| 112 |
+
| `docs/stories/backlog.md` | Task specification | Project memory |
|
| 113 |
+
| `docs/stories/epics/README.md` | Task specification | Project memory |
|
| 114 |
+
| `docs/stories/epics/E01-durable-layer/US-002-rust-harness-cli/overview.md` | Task specification | Project memory |
|
| 115 |
+
| `docs/stories/epics/E01-durable-layer/US-002-rust-harness-cli/design.md` | Task specification | Tool access, permissions |
|
| 116 |
+
| `docs/stories/epics/E01-durable-layer/US-002-rust-harness-cli/execplan.md` | Task specification | Verification, task state |
|
| 117 |
+
| `docs/stories/epics/E01-durable-layer/US-002-rust-harness-cli/validation.md` | Verification | Intervention recording |
|
| 118 |
+
| `docs/stories/epics/E02-phase-2-observability-taxonomy/phase-2-progress.md` | Task state | Intervention recording |
|
| 119 |
+
| `docs/stories/epics/E03-phase-5-evolution-infrastructure/phase-5-progress.md` | Task state | Verification, entropy auditing |
|
| 120 |
+
| `docs/templates/decision.md` | Project memory | Task specification |
|
| 121 |
+
| `docs/templates/spec-intake.md` | Task specification | Context selection |
|
| 122 |
+
| `docs/templates/story.md` | Task specification | Verification |
|
| 123 |
+
| `docs/templates/validation-report.md` | Verification | Intervention recording |
|
| 124 |
+
| `docs/templates/high-risk-story/overview.md` | Task specification | Context selection |
|
| 125 |
+
| `docs/templates/high-risk-story/design.md` | Task specification | Permissions |
|
| 126 |
+
| `docs/templates/high-risk-story/execplan.md` | Task state | Verification |
|
| 127 |
+
| `docs/templates/high-risk-story/validation.md` | Verification | Failure attribution |
|
| 128 |
+
| `scripts/README.md` | Tool access | Context selection |
|
| 129 |
+
| `scripts/bin/harness-cli` | Tool access | Task state, observability |
|
| 130 |
+
| `scripts/bin/harness-cli` | Tool access | Task state, observability |
|
| 131 |
+
| `scripts/install-harness.sh` | Tool access | Permissions |
|
| 132 |
+
| `scripts/build-harness-cli-release.sh` | Verification | Tool access |
|
| 133 |
+
| `scripts/schema/001-init.sql` | Task state | Observability, project memory |
|
| 134 |
+
| `scripts/schema/002-story-verify.sql` | Verification | Task state, project memory |
|
| 135 |
+
| `scripts/schema/003-tool-registry.sql` | Tool access | Project memory |
|
| 136 |
+
| `scripts/schema/004-intervention.sql` | Intervention recording | Failure attribution |
|
| 137 |
+
| `.github/ISSUE_TEMPLATE/agent-failure-case.md` | Failure attribution | Entropy auditing |
|
| 138 |
+
| `.github/ISSUE_TEMPLATE/pattern-request.md` | Entropy auditing | Intervention recording |
|
| 139 |
+
| `.github/ISSUE_TEMPLATE/real-world-example.md` | Project memory | Intervention recording |
|
| 140 |
+
| `.github/workflows/harness-cli-release.yml` | Verification | Tool access |
|
| 141 |
+
|
| 142 |
+
## Coverage Summary
|
| 143 |
+
|
| 144 |
+
- Covered: 8/11 responsibilities.
|
| 145 |
+
- Partial: 3/11 responsibilities.
|
| 146 |
+
- Missing: 0/11 responsibilities.
|
| 147 |
+
|
| 148 |
+
Covered responsibilities:
|
| 149 |
+
|
| 150 |
+
- Task specification.
|
| 151 |
+
- Context selection.
|
| 152 |
+
- Tool access.
|
| 153 |
+
- Project memory.
|
| 154 |
+
- Task state.
|
| 155 |
+
- Verification.
|
| 156 |
+
- Entropy auditing.
|
| 157 |
+
- Intervention recording.
|
| 158 |
+
Partial responsibilities:
|
| 159 |
+
|
| 160 |
+
- Observability.
|
| 161 |
+
- Failure attribution.
|
| 162 |
+
- Permissions.
|
| 163 |
+
|
| 164 |
+
Phase 5 converts tool access, entropy auditing, and intervention recording into
|
| 165 |
+
covered responsibilities with a registry, drift audit, proposal loop, and
|
| 166 |
+
intervention schema. Later phases should focus on benchmark ingestion,
|
| 167 |
+
component-level attribution, permission enforcement, and tool usage analytics.
|
docs/HARNESS_MATURITY.md
ADDED
|
@@ -0,0 +1,316 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Harness Maturity Ladder
|
| 2 |
+
|
| 3 |
+
This ladder defines how `repository-harness` should progress from static
|
| 4 |
+
agent instructions to measurable harness improvement.
|
| 5 |
+
|
| 6 |
+
The levels are intentionally verifiable. A level is achieved only when its
|
| 7 |
+
criteria can be inspected in repository files, durable Harness records, or
|
| 8 |
+
benchmark output.
|
| 9 |
+
|
| 10 |
+
## Levels
|
| 11 |
+
|
| 12 |
+
### H0 - Bare Environment
|
| 13 |
+
|
| 14 |
+
The model operates with no repository harness. It receives a prompt and may
|
| 15 |
+
produce a patch, but the repo does not tell it how to classify, validate, or
|
| 16 |
+
record work.
|
| 17 |
+
|
| 18 |
+
Criteria:
|
| 19 |
+
|
| 20 |
+
- No `AGENTS.md` Harness block exists.
|
| 21 |
+
- No feature intake policy exists.
|
| 22 |
+
- No story, decision, validation, or trace artifact exists.
|
| 23 |
+
|
| 24 |
+
Required files:
|
| 25 |
+
|
| 26 |
+
- None.
|
| 27 |
+
|
| 28 |
+
Benchmark indicators:
|
| 29 |
+
|
| 30 |
+
- Functional score is the only meaningful metric.
|
| 31 |
+
- Harness compliance: 0%.
|
| 32 |
+
- Trace quality: 0/3.
|
| 33 |
+
|
| 34 |
+
Current status:
|
| 35 |
+
|
| 36 |
+
- Passed. This repository is beyond H0.
|
| 37 |
+
|
| 38 |
+
Activated responsibilities:
|
| 39 |
+
|
| 40 |
+
- None.
|
| 41 |
+
|
| 42 |
+
### H1 - Scaffolding And Policy
|
| 43 |
+
|
| 44 |
+
The repository contains static operating instructions, templates, risk lanes,
|
| 45 |
+
and source-of-truth rules. Agents can follow a documented workflow, but durable
|
| 46 |
+
state may still be manual or incomplete.
|
| 47 |
+
|
| 48 |
+
Criteria:
|
| 49 |
+
|
| 50 |
+
- `AGENTS.md` points agents to the Harness operating docs.
|
| 51 |
+
- `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, and `docs/ARCHITECTURE.md`
|
| 52 |
+
exist.
|
| 53 |
+
- Story, decision, and validation templates exist under `docs/templates/`.
|
| 54 |
+
- `docs/TEST_MATRIX.md` defines proof columns and status meanings.
|
| 55 |
+
|
| 56 |
+
Required files:
|
| 57 |
+
|
| 58 |
+
- `AGENTS.md`
|
| 59 |
+
- `docs/HARNESS.md`
|
| 60 |
+
- `docs/FEATURE_INTAKE.md`
|
| 61 |
+
- `docs/ARCHITECTURE.md`
|
| 62 |
+
- `docs/TEST_MATRIX.md`
|
| 63 |
+
- `docs/templates/story.md`
|
| 64 |
+
- `docs/templates/decision.md`
|
| 65 |
+
- `docs/templates/validation-report.md`
|
| 66 |
+
|
| 67 |
+
Benchmark indicators:
|
| 68 |
+
|
| 69 |
+
- Harness compliance: 20-40%.
|
| 70 |
+
- Lane accuracy improves when agents read the intake policy.
|
| 71 |
+
- Trace quality remains low unless traces are separately requested.
|
| 72 |
+
|
| 73 |
+
Current status:
|
| 74 |
+
|
| 75 |
+
- Achieved. H1 files exist and are used by current Harness instructions.
|
| 76 |
+
|
| 77 |
+
Activated responsibilities:
|
| 78 |
+
|
| 79 |
+
- Task specification.
|
| 80 |
+
- Permissions.
|
| 81 |
+
- Project memory.
|
| 82 |
+
- Verification.
|
| 83 |
+
|
| 84 |
+
### H2 - Durable State And Observability
|
| 85 |
+
|
| 86 |
+
The repository has structured operational records and explicit observation
|
| 87 |
+
rules. Agents can record what happened, connect work to stories, and write
|
| 88 |
+
traces with predictable depth.
|
| 89 |
+
|
| 90 |
+
Criteria:
|
| 91 |
+
|
| 92 |
+
- `scripts/bin/harness-cli` can record intake, story, decision, backlog, and trace
|
| 93 |
+
data in `harness.db`.
|
| 94 |
+
- `scripts/schema/001-init.sql` defines durable tables for intake, story,
|
| 95 |
+
decision, backlog, and trace records.
|
| 96 |
+
- `docs/HARNESS_COMPONENTS.md` maps files and responsibilities.
|
| 97 |
+
- `docs/HARNESS_MATURITY.md` defines H0-H5 with measurable criteria.
|
| 98 |
+
- `docs/TRACE_SPEC.md` defines trace fields, quality tiers, and friction
|
| 99 |
+
capture.
|
| 100 |
+
- `docs/CONTEXT_RULES.md` defines phase-by-lane context rules.
|
| 101 |
+
- `AGENTS.md` and `docs/HARNESS.md` reference the Phase 2 operating docs.
|
| 102 |
+
|
| 103 |
+
Required files:
|
| 104 |
+
|
| 105 |
+
- `scripts/bin/harness-cli`
|
| 106 |
+
- `scripts/schema/001-init.sql`
|
| 107 |
+
- `docs/HARNESS_COMPONENTS.md`
|
| 108 |
+
- `docs/HARNESS_MATURITY.md`
|
| 109 |
+
- `docs/TRACE_SPEC.md`
|
| 110 |
+
- `docs/CONTEXT_RULES.md`
|
| 111 |
+
|
| 112 |
+
Benchmark indicators:
|
| 113 |
+
|
| 114 |
+
- Harness compliance: 75-90%.
|
| 115 |
+
- Trace quality: at least 2.0/3 on normal-lane tasks.
|
| 116 |
+
- Lane accuracy: 6/6 on the current benchmark suite.
|
| 117 |
+
- Friction captured: at least 4/6 benchmark tasks when friction exists.
|
| 118 |
+
|
| 119 |
+
Current status:
|
| 120 |
+
|
| 121 |
+
- Achieved. Durable state exists, and the Phase 2 docs define the
|
| 122 |
+
observability and context specification. Phase 3 active scoring builds on
|
| 123 |
+
this layer.
|
| 124 |
+
|
| 125 |
+
Activated responsibilities:
|
| 126 |
+
|
| 127 |
+
- Task state.
|
| 128 |
+
- Observability.
|
| 129 |
+
- Failure attribution.
|
| 130 |
+
- Context selection.
|
| 131 |
+
- Entropy auditing.
|
| 132 |
+
|
| 133 |
+
### H3 - Active Observability And Evolution
|
| 134 |
+
|
| 135 |
+
The harness can evaluate its own operational data and turn repeated failures
|
| 136 |
+
into prioritized improvements.
|
| 137 |
+
|
| 138 |
+
Criteria:
|
| 139 |
+
|
| 140 |
+
- Trace quality can be scored by a repeatable command or benchmark step.
|
| 141 |
+
- Harness friction can be grouped by component from `docs/HARNESS_COMPONENTS.md`.
|
| 142 |
+
- Backlog items include predicted impact and actual outcome after completion.
|
| 143 |
+
- Benchmark comparison output identifies which harness responsibility moved or
|
| 144 |
+
regressed.
|
| 145 |
+
|
| 146 |
+
Required files:
|
| 147 |
+
|
| 148 |
+
- H2 files.
|
| 149 |
+
- A benchmark protocol or report that references maturity levels.
|
| 150 |
+
- A documented trace quality scoring method.
|
| 151 |
+
- A documented friction-to-backlog review loop.
|
| 152 |
+
|
| 153 |
+
Benchmark indicators:
|
| 154 |
+
|
| 155 |
+
- Harness compliance: 85-95%.
|
| 156 |
+
- Trace quality: 2.3-2.7/3.
|
| 157 |
+
- Friction captured and classified by component for most failed or awkward
|
| 158 |
+
tasks.
|
| 159 |
+
- Regressions include an attributed harness component.
|
| 160 |
+
|
| 161 |
+
Current status:
|
| 162 |
+
|
| 163 |
+
- Partially achieved by Phase 3. `scripts/bin/harness-cli score-trace` scores trace
|
| 164 |
+
quality against tier rules, `query friction` includes linked intake context,
|
| 165 |
+
the `trace` command now prints that score at write time, and the backlog
|
| 166 |
+
outcome loop documents predicted impact versus actual outcome. Full H3 still
|
| 167 |
+
requires benchmark comparison output that attributes moved or regressed
|
| 168 |
+
responsibilities.
|
| 169 |
+
|
| 170 |
+
Activated responsibilities:
|
| 171 |
+
|
| 172 |
+
- Observability.
|
| 173 |
+
- Failure attribution.
|
| 174 |
+
- Entropy auditing.
|
| 175 |
+
- Intervention recording.
|
| 176 |
+
|
| 177 |
+
### H4 - Automated Verification
|
| 178 |
+
|
| 179 |
+
The harness can run or orchestrate proof checks consistently and can reject or
|
| 180 |
+
flag incomplete work before the final response.
|
| 181 |
+
|
| 182 |
+
Criteria:
|
| 183 |
+
|
| 184 |
+
- A documented verification command or protocol runs the expected checks for a
|
| 185 |
+
selected story and lane.
|
| 186 |
+
- Stories can store and execute a `verify_command`.
|
| 187 |
+
- Trace recording warns when a linked story has a verification command that has
|
| 188 |
+
not passed.
|
| 189 |
+
- Missing validation evidence is surfaced before a task is marked implemented.
|
| 190 |
+
|
| 191 |
+
Required files:
|
| 192 |
+
|
| 193 |
+
- H3 files.
|
| 194 |
+
- A verification protocol or command reference.
|
| 195 |
+
- Validation report examples tied to story proof columns.
|
| 196 |
+
- Story verification command documentation.
|
| 197 |
+
|
| 198 |
+
Benchmark indicators:
|
| 199 |
+
|
| 200 |
+
- Functional score remains stable.
|
| 201 |
+
- Harness compliance: at least 90%.
|
| 202 |
+
- Fewer false "done" claims in benchmark review.
|
| 203 |
+
- Missing proof is detected before merge or final response.
|
| 204 |
+
|
| 205 |
+
Current status:
|
| 206 |
+
|
| 207 |
+
- Achieved by Phase 5. `scripts/bin/harness-cli story verify <id>` runs
|
| 208 |
+
story-level proof commands, records pass/fail state, `trace --story` warns
|
| 209 |
+
before close when verification has not passed, and
|
| 210 |
+
`scripts/bin/harness-cli story verify-all` runs all configured story proof
|
| 211 |
+
commands in one pass. Proof-column automation remains a future enhancement,
|
| 212 |
+
but H4's required automated verification gate is now present.
|
| 213 |
+
|
| 214 |
+
Activated responsibilities:
|
| 215 |
+
|
| 216 |
+
- Verification.
|
| 217 |
+
- Task state.
|
| 218 |
+
- Permissions.
|
| 219 |
+
- Intervention recording.
|
| 220 |
+
|
| 221 |
+
### H5 - Self-Improving Harness
|
| 222 |
+
|
| 223 |
+
The harness can use traces, benchmark results, and backlog outcomes to propose
|
| 224 |
+
or apply safe improvements to itself.
|
| 225 |
+
|
| 226 |
+
Criteria:
|
| 227 |
+
|
| 228 |
+
- Repeated friction patterns are summarized into proposed harness changes.
|
| 229 |
+
- Proposed changes include predicted impact, risk, validation plan, and rollback
|
| 230 |
+
criteria.
|
| 231 |
+
- Completed changes compare predicted impact with actual benchmark or trace
|
| 232 |
+
outcomes.
|
| 233 |
+
- High-risk harness changes pause for human confirmation before changing source
|
| 234 |
+
hierarchy, architecture direction, or validation requirements.
|
| 235 |
+
|
| 236 |
+
Required files:
|
| 237 |
+
|
| 238 |
+
- H4 files.
|
| 239 |
+
- Self-improvement protocol.
|
| 240 |
+
- Historical improvement reports.
|
| 241 |
+
- Backlog outcome reviews.
|
| 242 |
+
|
| 243 |
+
Benchmark indicators:
|
| 244 |
+
|
| 245 |
+
- Harness compliance remains at least 90% across repeated benchmark runs.
|
| 246 |
+
- Trace quality remains at least 2.5/3.
|
| 247 |
+
- Improvements show measurable positive deltas or are explicitly reverted.
|
| 248 |
+
- Scope creep and validation weakening are caught by policy.
|
| 249 |
+
|
| 250 |
+
Current status:
|
| 251 |
+
|
| 252 |
+
- Partially achieved by Phase 5. `scripts/bin/harness-cli audit` detects
|
| 253 |
+
durable-state drift, `scripts/bin/harness-cli propose` generates structured
|
| 254 |
+
improvement proposals from friction, interventions, and audit results, and
|
| 255 |
+
`docs/IMPROVEMENT_PROTOCOL.md` defines the review loop. H5 is not fully
|
| 256 |
+
achieved until repeated benchmark outcomes prove proposed improvements create
|
| 257 |
+
measurable positive deltas or are explicitly reverted.
|
| 258 |
+
|
| 259 |
+
Activated responsibilities:
|
| 260 |
+
|
| 261 |
+
- Entropy auditing.
|
| 262 |
+
- Failure attribution.
|
| 263 |
+
- Intervention recording.
|
| 264 |
+
- Permissions.
|
| 265 |
+
|
| 266 |
+
## Current Assessment
|
| 267 |
+
|
| 268 |
+
| Level | Status | Evidence |
|
| 269 |
+
| --- | --- | --- |
|
| 270 |
+
| H0 | Passed | Harness docs, templates, and durable records exist. |
|
| 271 |
+
| H1 | Achieved | `AGENTS.md`, `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, `docs/ARCHITECTURE.md`, `docs/templates/*`, and `docs/TEST_MATRIX.md` exist. |
|
| 272 |
+
| H2 | Achieved | `scripts/bin/harness-cli`, `scripts/schema/001-init.sql`, durable story records, `docs/HARNESS_COMPONENTS.md`, `docs/HARNESS_MATURITY.md`, `docs/TRACE_SPEC.md`, and `docs/CONTEXT_RULES.md` define the Phase 2 surface. |
|
| 273 |
+
| H3 | Partial | Phase 3 adds `scripts/bin/harness-cli score-trace`, enriched friction context, and the backlog outcome loop; Phase 4 auto-scores traces on write. Component-level benchmark attribution remains open. |
|
| 274 |
+
| H4 | Achieved | Phase 4 adds story-level `verify_command`, `story verify`, and trace-time verification warnings. Phase 5 adds `story verify-all` for batch story proof. |
|
| 275 |
+
| H5 | Partial | Phase 5 adds `audit`, `score-context`, `intervention add/query`, `propose`, `docs/HARNESS_AUDIT.md`, and `docs/IMPROVEMENT_PROTOCOL.md`; repeated benchmark outcome proof remains open. |
|
| 276 |
+
|
| 277 |
+
## Responsibility Activation
|
| 278 |
+
|
| 279 |
+
| Responsibility | H0 | H1 | H2 | H3 | H4 | H5 |
|
| 280 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 281 |
+
| Task specification | Missing | Covered | Covered | Covered | Covered | Covered |
|
| 282 |
+
| Context selection | Missing | Partial | Covered | Covered | Covered | Covered |
|
| 283 |
+
| Tool access | Missing | Partial | Partial | Partial | Covered | Covered |
|
| 284 |
+
| Project memory | Missing | Covered | Covered | Covered | Covered | Covered |
|
| 285 |
+
| Task state | Missing | Partial | Covered | Covered | Covered | Covered |
|
| 286 |
+
| Observability | Missing | Missing | Partial | Covered | Covered | Covered |
|
| 287 |
+
| Failure attribution | Missing | Missing | Partial | Covered | Covered | Covered |
|
| 288 |
+
| Verification | Missing | Partial | Partial | Partial | Covered | Covered |
|
| 289 |
+
| Permissions | Missing | Partial | Partial | Partial | Covered | Covered |
|
| 290 |
+
| Entropy auditing | Missing | Missing | Partial | Covered | Covered | Covered |
|
| 291 |
+
| Intervention recording | Missing | Partial | Partial | Covered | Covered | Covered |
|
| 292 |
+
|
| 293 |
+
## Phase 3 Interpretation
|
| 294 |
+
|
| 295 |
+
Phase 3 starts the H2 to H3 transition. It claims active trace scoring and a
|
| 296 |
+
documented improvement feedback loop, but it does not claim full H3 because
|
| 297 |
+
benchmark comparison and component-level regression attribution are explicitly
|
| 298 |
+
outside this repository's Phase 3 scope.
|
| 299 |
+
|
| 300 |
+
## Phase 4 Interpretation
|
| 301 |
+
|
| 302 |
+
Phase 4 starts the H3 to H4 transition. It gives stories the same mechanical
|
| 303 |
+
verification pattern that decisions already had, records story verification
|
| 304 |
+
results in the durable layer, auto-scores traces when they are written, and
|
| 305 |
+
warns before close when a linked story's verification has not passed. It does
|
| 306 |
+
not claim full H4 because benchmark execution, batch verification, and automatic
|
| 307 |
+
proof-column updates remain separate work.
|
| 308 |
+
|
| 309 |
+
## Phase 5 Interpretation
|
| 310 |
+
|
| 311 |
+
Phase 5 completes H4 by adding batch story verification and starts H5 by adding
|
| 312 |
+
tool discovery, intervention records, context scoring, drift audit, and
|
| 313 |
+
deterministic proposal generation. The repository may claim H5 partial only
|
| 314 |
+
when those commands and docs are present and validated; it must not claim full
|
| 315 |
+
H5 until benchmark runs or trace outcomes prove the proposal loop improves the
|
| 316 |
+
harness over time.
|
docs/IMPROVEMENT_PROTOCOL.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Improvement Protocol
|
| 2 |
+
|
| 3 |
+
Phase 5 starts the self-improvement loop:
|
| 4 |
+
|
| 5 |
+
```text
|
| 6 |
+
friction + interventions + audit findings
|
| 7 |
+
-> harness-cli propose
|
| 8 |
+
-> proposed backlog item
|
| 9 |
+
-> human review
|
| 10 |
+
-> implementation with predicted impact
|
| 11 |
+
-> close with actual outcome
|
| 12 |
+
```
|
| 13 |
+
|
| 14 |
+
## Generate Proposals
|
| 15 |
+
|
| 16 |
+
```bash
|
| 17 |
+
scripts/bin/harness-cli propose
|
| 18 |
+
```
|
| 19 |
+
|
| 20 |
+
The command is rule-based. It looks for:
|
| 21 |
+
|
| 22 |
+
- repeated trace friction,
|
| 23 |
+
- repeated intervention patterns,
|
| 24 |
+
- non-zero audit categories.
|
| 25 |
+
|
| 26 |
+
Each proposal includes title, component, evidence, predicted impact, risk,
|
| 27 |
+
suggested action, validation plan, and confidence.
|
| 28 |
+
|
| 29 |
+
## Commit Proposals
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
scripts/bin/harness-cli propose --commit
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
Committed proposals become `proposed` backlog items. Humans review them with:
|
| 36 |
+
|
| 37 |
+
```bash
|
| 38 |
+
scripts/bin/harness-cli query backlog --open
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
## Review Rules
|
| 42 |
+
|
| 43 |
+
- Tiny proposals may be implemented directly when they only clarify docs.
|
| 44 |
+
- Normal proposals need a story packet or clear backlog acceptance.
|
| 45 |
+
- High-risk proposals need a durable decision record before changing source
|
| 46 |
+
hierarchy, architecture direction, validation requirements, or risk policy.
|
| 47 |
+
- Completed proposal work must close the backlog item with actual outcome
|
| 48 |
+
evidence.
|
| 49 |
+
|
| 50 |
+
## Validation
|
| 51 |
+
|
| 52 |
+
After implementation, compare the predicted impact with:
|
| 53 |
+
|
| 54 |
+
- `scripts/bin/harness-cli audit`,
|
| 55 |
+
- `scripts/bin/harness-cli query friction`,
|
| 56 |
+
- `scripts/bin/harness-cli query interventions`,
|
| 57 |
+
- benchmark trace quality and harness compliance when benchmark proof applies.
|
docs/README.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Documentation
|
| 2 |
+
|
| 3 |
+
Start every task with `AGENTS.md`. This index directs agents to the smallest
|
| 4 |
+
relevant context rather than every document in the repository.
|
| 5 |
+
|
| 6 |
+
## Current Product and Work
|
| 7 |
+
|
| 8 |
+
- `product/` — accepted contracts for CarePath, Ghi chép bệnh án AI, and Phiên
|
| 9 |
+
dịch khám bệnh trực tiếp.
|
| 10 |
+
- `ux-redesign-carepath.md` — current Vietnamese-first UX implementation
|
| 11 |
+
backlog; required before UX or product-flow implementation.
|
| 12 |
+
- `onboarding-ux-fix-tasks.md` — current visual onboarding execution backlog.
|
| 13 |
+
- `../UI-FIX-PLAN.md` — current public-site issue plan.
|
| 14 |
+
- `deploy.md` — Hugging Face Space deployment instructions.
|
| 15 |
+
|
| 16 |
+
## Harness: Read for Daily Work
|
| 17 |
+
|
| 18 |
+
- `HARNESS.md` — source hierarchy, task loop, and done definition.
|
| 19 |
+
- `FEATURE_INTAKE.md` — lane selection and CarePath hard gates.
|
| 20 |
+
- `ARCHITECTURE.md` — module and delivery boundaries.
|
| 21 |
+
- `CONTEXT_RULES.md` — minimum context by lane.
|
| 22 |
+
- `TEST_MATRIX.md` — behavior-to-proof baseline.
|
| 23 |
+
- `TRACE_SPEC.md` — durable completion evidence.
|
| 24 |
+
- `stories/`, `decisions/`, and `templates/` — selected work, settled
|
| 25 |
+
tradeoffs, and new-work starters.
|
| 26 |
+
|
| 27 |
+
## Harness: Read Only When Triggered
|
| 28 |
+
|
| 29 |
+
- `TOOL_REGISTRY.md` — registering or using optional external tools.
|
| 30 |
+
- `HARNESS_AUDIT.md`, `HARNESS_COMPONENTS.md`, and `HARNESS_MATURITY.md` —
|
| 31 |
+
audit, observability, maturity, or benchmark work.
|
| 32 |
+
- `HARNESS_BACKLOG.md` and `IMPROVEMENT_PROTOCOL.md` — repeated Harness
|
| 33 |
+
friction or process improvements.
|
| 34 |
+
- `GLOSSARY.md` — extending shared Harness terminology.
|
| 35 |
+
|
| 36 |
+
## Evidence and History
|
| 37 |
+
|
| 38 |
+
- `qa-evidence/` — versioned visual QA proof and its retention index.
|
| 39 |
+
- `history/` — preserved MVP, demo-site, unification, and review documents.
|
| 40 |
+
These are context only; do not reopen completed tickets as current work.
|
docs/TEST_MATRIX.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CarePath Test Matrix
|
| 2 |
+
|
| 3 |
+
This matrix maps the current CarePath baseline to proof. Status changes only
|
| 4 |
+
after the named evidence is actually run and recorded.
|
| 5 |
+
|
| 6 |
+
| Story | Contract | Unit | Integration | E2E | Platform | Status | Evidence |
|
| 7 |
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
| 8 |
+
| CP-BASE-001 | Unified API and Ghi chép bệnh án AI remain available | yes | yes | no | no | implemented | 96 passed, 1 skipped; mock smoke passed (2026-07-12) |
|
| 9 |
+
| CP-BASE-002 | Interpreter remains translate-only and fail-closed | yes | yes | yes | no | implemented | Ruff; 109 passed, 1 skipped; mock eval 50/50; 4 browser tests passed (2026-07-12) |
|
| 10 |
+
| CP-BASE-003 | Public CarePath site remains Vietnamese-first and deployable | yes | no | yes | yes | implemented | 45 unit, 5 deploy-env, 7 browser tests; Lighthouse 100/100/100 (2026-07-12) |
|
| 11 |
+
| CP-BASE-004 | Interpreter console remains buildable and usable | yes | no | yes | no | implemented | lint; 38 unit; build; 4 browser tests passed (2026-07-12) |
|
| 12 |
+
| GEC-001 | Offline GEC training uses ViMedCSS and synthetic pairs without human-labeling tooling | no | yes | no | no | implemented | Ruff; 26 focused GEC tests; root 95 passed, 1 skipped (2026-07-12) |
|
| 13 |
+
| HARN-001 | Agent work follows durable CarePath intake and proof rules | no | yes | no | no | implemented | pinned merge install, CLI 0.1.11, init, matrix, audit, trace (2026-07-12) |
|
| 14 |
+
| CP-UX-10 | Public landing leads with Ghi chép bệnh án AI; Interpreter is visibly unavailable | yes | yes | yes | yes | implemented | lint; 46 unit; 5 deploy-env; build + NFC; 5 browser tests at 320–1440px; combined app 3 passed; Lighthouse 100/100/100 (2026-07-14) |
|
| 15 |
+
| CP-UX-11 | Vietnamese-only landing starts with the doctor's documentation burden and teaches the Scribe workflow before upload | yes | yes | yes | yes | implemented | lint; 45 unit; 5 deploy-env; build + NFC; 8 browser tests at 320–1440px with Axe/light/dark/reduced-motion; combined app 3 passed; Lighthouse 100/100/100 (2026-07-15) |
|
| 16 |
+
|
| 17 |
+
Proof labels mean the relevant layer has evidence for the baseline; they do not
|
| 18 |
+
claim every historical behavior was retested. Story packets define the exact
|
| 19 |
+
commands for future changes.
|
docs/TOOL_REGISTRY.md
ADDED
|
@@ -0,0 +1,195 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Tool Registry
|
| 2 |
+
|
| 3 |
+
The harness deals with two distinct kinds of "tool". Keep them separate.
|
| 4 |
+
|
| 5 |
+
| | Capability manifest (outbound) | Inbound tool registry |
|
| 6 |
+
| --- | --- | --- |
|
| 7 |
+
| Direction | harness offers it to the agent | a project equips it for the harness to use |
|
| 8 |
+
| Examples | the `harness-cli` subcommands below | gitnexus, c3, a linter, a deploy check |
|
| 9 |
+
| Presence | always compiled in | optional; may be absent on any machine |
|
| 10 |
+
| If missing | n/a (it is the harness) | clean skip; never blocks the main process |
|
| 11 |
+
|
| 12 |
+
This document describes both. The **inbound registry** is the extension base:
|
| 13 |
+
it is where the harness learns what extra capability is equipped, what purpose
|
| 14 |
+
it serves, and whether it is actually present right now, so a workflow step can
|
| 15 |
+
adapt to what is installed without the core ever depending on it.
|
| 16 |
+
|
| 17 |
+
## Inbound Registry: Register A Tool
|
| 18 |
+
|
| 19 |
+
```bash
|
| 20 |
+
scripts/bin/harness-cli tool register \
|
| 21 |
+
--name deploy-check \
|
| 22 |
+
--kind cli \
|
| 23 |
+
--capability deploy-verification \
|
| 24 |
+
--command ./scripts/deploy-check.sh \
|
| 25 |
+
--description "Verify deploy health before release" \
|
| 26 |
+
--responsibility Verification \
|
| 27 |
+
--args "env:enum:required:staging,production"
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
Fields specific to inbound tools:
|
| 31 |
+
|
| 32 |
+
- `--kind` — how the tool is reached and probed. One of `cli`, `binary`, `mcp`,
|
| 33 |
+
`skill`, `http`. Defaults to `cli`. The kind tells each agent runtime what it
|
| 34 |
+
can orchestrate (a non-Claude agent simply treats a `skill` it cannot run as
|
| 35 |
+
absent) and tells `tool check` which probe to use.
|
| 36 |
+
- `--capability` — the workflow purpose a step looks the tool up by. Free-text
|
| 37 |
+
but normalized to kebab-case, so `Impact Analysis`, `impact_analysis`, and
|
| 38 |
+
`impact-analysis` all register as `impact-analysis`. This is the only coupling
|
| 39 |
+
between a step and a tool; steps reference the capability, never the tool name.
|
| 40 |
+
- `--scan` — for `mcp`/`skill`/`http`, a declarative path or URL that
|
| 41 |
+
`tool check` resolves to decide presence (e.g. `.c3`, `~/.claude/skills/c3`,
|
| 42 |
+
`https://localhost:8080/health`). `cli`/`binary` are probed via their command.
|
| 43 |
+
|
| 44 |
+
`--force` is only needed for `cli`/`binary` whose command is intentionally
|
| 45 |
+
absent on the current machine. `mcp`/`skill`/`http` are not on `PATH` by nature,
|
| 46 |
+
so they register without `--force`; their presence is resolved later by
|
| 47 |
+
`tool check`.
|
| 48 |
+
|
| 49 |
+
Registering an MCP server or a Claude skill (examples):
|
| 50 |
+
|
| 51 |
+
```bash
|
| 52 |
+
scripts/bin/harness-cli tool register --name gitnexus --kind mcp \
|
| 53 |
+
--capability impact-analysis --scan ".gitnexus" --command "mcp:gitnexus" \
|
| 54 |
+
--description "Code-graph blast radius" --responsibility Verification
|
| 55 |
+
scripts/bin/harness-cli tool register --name c3 --kind skill \
|
| 56 |
+
--capability impact-analysis --scan ".c3" --command "skill:c3" \
|
| 57 |
+
--description "Component model and drift audit (Claude skill)" \
|
| 58 |
+
--responsibility Verification
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Remove a tool with:
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
scripts/bin/harness-cli tool remove --name deploy-check
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
## Inbound Registry: Check Presence
|
| 68 |
+
|
| 69 |
+
Registration records intent. `tool check` reconciles intent with reality by
|
| 70 |
+
scanning each registered tool and persisting the verdict (`status` and
|
| 71 |
+
`checked_at`). Run it at intake start so status reflects current reality.
|
| 72 |
+
|
| 73 |
+
```bash
|
| 74 |
+
scripts/bin/harness-cli tool check # scan all registered tools
|
| 75 |
+
scripts/bin/harness-cli tool check --name c3 # scan one
|
| 76 |
+
scripts/bin/harness-cli tool check --json # machine-readable for agents
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
Probe per kind:
|
| 80 |
+
|
| 81 |
+
| Kind | Probe | `present` means |
|
| 82 |
+
| --- | --- | --- |
|
| 83 |
+
| `cli`, `binary` | command resolves on `PATH` or as a path | installed and runnable |
|
| 84 |
+
| `mcp`, `skill` | `scan_target` path resolves (`~` expands) | equipped/configured on disk |
|
| 85 |
+
| `http` | `scan_target` reachable over TCP (2s), else path | endpoint answers |
|
| 86 |
+
|
| 87 |
+
`tool check` always exits `0`: a missing extension is a fact to report, not a
|
| 88 |
+
CLI failure. A `cli`/`binary` is `present` when runnable. An `mcp`/`skill`/`http`
|
| 89 |
+
`present` means **equipped** (config/file resolves), not **live this session** —
|
| 90 |
+
the agent still confirms live usability at call time, since only the agent
|
| 91 |
+
runtime can see whether its MCP server is actually connected. With no
|
| 92 |
+
`scan_target`, the status is `unknown` and the agent must confirm.
|
| 93 |
+
|
| 94 |
+
## Inbound Registry: Look Up By Capability
|
| 95 |
+
|
| 96 |
+
A workflow step asks "what is present for this purpose?" rather than naming a
|
| 97 |
+
tool:
|
| 98 |
+
|
| 99 |
+
```bash
|
| 100 |
+
scripts/bin/harness-cli query tools --capability impact-analysis
|
| 101 |
+
scripts/bin/harness-cli query tools --capability impact-analysis --status present
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
The result is the set of providers. Multiple tools may provide one capability
|
| 105 |
+
(gitnexus and c3 both serve `impact-analysis` and are complementary), so a step
|
| 106 |
+
reads the set and degrades on how much of it is present.
|
| 107 |
+
|
| 108 |
+
### Degrade Ladder
|
| 109 |
+
|
| 110 |
+
The CLI reports facts (`status`); the agent applies policy. The generic rule,
|
| 111 |
+
keyed on the present-provider count for a capability:
|
| 112 |
+
|
| 113 |
+
| Providers present | Posture | Agent behavior |
|
| 114 |
+
| --- | --- | --- |
|
| 115 |
+
| none registered | Inactive | clean skip; note `capability X: inactive` in the trace. Not drift. |
|
| 116 |
+
| registered but none/some present | Degraded | run with what resolves; set the `Weak proof` flag; note the gap. |
|
| 117 |
+
| all present | Full | normal operation. |
|
| 118 |
+
|
| 119 |
+
A registered tool that scans as `missing` is a failed validity gate, not a skip.
|
| 120 |
+
A capability with no registered providers is simply inactive and is skipped
|
| 121 |
+
without penalty — this is what keeps the core seamless on a fresh install.
|
| 122 |
+
|
| 123 |
+
### Recommended Capability Vocabulary
|
| 124 |
+
|
| 125 |
+
Capability is open (no code change to add one), but a step and its providers
|
| 126 |
+
must agree on the exact string. Reuse these where they fit before coining a new
|
| 127 |
+
one; coin new ones in kebab-case:
|
| 128 |
+
|
| 129 |
+
```
|
| 130 |
+
impact-analysis · deploy-verification · coverage · security-scan
|
| 131 |
+
performance-benchmark · documentation-lookup
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
## Inspecting The Registry
|
| 135 |
+
|
| 136 |
+
```bash
|
| 137 |
+
scripts/bin/harness-cli query tools --summary
|
| 138 |
+
scripts/bin/harness-cli query tools --json
|
| 139 |
+
scripts/bin/harness-cli query tools --responsibility Verification
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
JSON records carry `kind`, `capability`, `scan_target`, `status`, and
|
| 143 |
+
`checked_at` alongside the existing fields, so any agent can read the registry
|
| 144 |
+
without parsing the human table.
|
| 145 |
+
|
| 146 |
+
## Compiled Harness Commands (Outbound Manifest)
|
| 147 |
+
|
| 148 |
+
| Command | Responsibility | Purpose | Arguments |
|
| 149 |
+
| --- | --- | --- | --- |
|
| 150 |
+
| `init` | Task state | Create the harness database. | none |
|
| 151 |
+
| `migrate` | Task state | Apply pending schema migrations. | none |
|
| 152 |
+
| `import brownfield` | Project memory | Seed durable records from markdown state. | none |
|
| 153 |
+
| `intake` | Task specification | Record a feature intake classification. | `--type`, `--summary`, `--lane` |
|
| 154 |
+
| `story add` | Task state | Create a durable story record. | `--id`, `--title`, `--lane`, optional `--verify` |
|
| 155 |
+
| `story update` | Task state | Update story status, proof flags, evidence, or verification command. | `--id`, optional proof/status fields |
|
| 156 |
+
| `story verify` | Verification | Run one story `verify_command` and record pass/fail. | story id |
|
| 157 |
+
| `story verify-all` | Verification | Run all configured story verification commands and skip stories without one. | none |
|
| 158 |
+
| `decision add` | Project memory | Create a durable decision record. | `--id`, `--title`, optional `--doc`, `--verify` |
|
| 159 |
+
| `decision verify` | Verification | Run one decision verification command. | decision id |
|
| 160 |
+
| `backlog add` | Entropy auditing | Record a harness improvement proposal. | `--title`, optional pain/suggestion/risk/predicted fields |
|
| 161 |
+
| `backlog close` | Entropy auditing | Close a backlog item with outcome evidence. | `--id`, optional `--status`, `--outcome` |
|
| 162 |
+
| `tool register` | Tool access | Register an external project tool. | `--name`, `--command`, `--description`, `--responsibility`, optional `--kind`, `--capability`, `--scan`, `--args`, `--force` |
|
| 163 |
+
| `tool check` | Tool access | Scan registered tools and persist present/missing/unknown status. | optional `--name`, `--json` |
|
| 164 |
+
| `tool remove` | Tool access | Remove a registered external tool. | `--name` |
|
| 165 |
+
| `intervention add` | Intervention recording | Record a human, reviewer, CI, or agent intervention. | `--type`, `--description`, `--source`, optional `--trace`, `--story`, `--impact` |
|
| 166 |
+
| `trace` | Observability | Record an agent execution trace and print trace quality. | `--summary`, optional trace fields |
|
| 167 |
+
| `score-trace` | Observability | Score trace detail against lane requirements. | optional `--id` |
|
| 168 |
+
| `score-context` | Context selection | Score trace reads against compiled context rules. | trace id |
|
| 169 |
+
| `audit` | Entropy auditing | Run drift checks and compute entropy score. | none |
|
| 170 |
+
| `propose` | Entropy auditing | Generate improvement proposals from friction, interventions, and audit findings. | optional `--commit` |
|
| 171 |
+
| `query matrix` | Task state | Show durable story proof matrix. | optional `--numeric` |
|
| 172 |
+
| `query backlog` | Entropy auditing | Show harness improvement backlog. | optional `--open`, `--closed` |
|
| 173 |
+
| `query decisions` | Project memory | Show durable decision records. | none |
|
| 174 |
+
| `query intakes` | Task specification | Show recent intake records. | none |
|
| 175 |
+
| `query traces` | Observability | Show recent trace records. | none |
|
| 176 |
+
| `query friction` | Failure attribution | Show traces with harness friction. | none |
|
| 177 |
+
| `query tools` | Tool access | Show compiled and registered tool entries. | optional `--json`, `--summary`, `--responsibility`, `--capability`, `--status` |
|
| 178 |
+
| `query interventions` | Intervention recording | Show intervention records. | optional `--trace`, `--story`, `--type` |
|
| 179 |
+
| `query stats` | Task state | Show durable record counts. | none |
|
| 180 |
+
| `query sql` | Tool access | Run arbitrary SQL against `harness.db`. | SQL text |
|
| 181 |
+
| `db changeset apply` | Task state | Apply one semantic changeset idempotently. | changeset path |
|
| 182 |
+
| `db rebuild` | Task state | Rebuild a fresh `harness.db` from semantic changesets. | `--from` changeset directory |
|
| 183 |
+
|
| 184 |
+
## Validation Rules
|
| 185 |
+
|
| 186 |
+
- Tool names must be unique among registered tools.
|
| 187 |
+
- Descriptions must be 10-200 characters.
|
| 188 |
+
- Responsibilities must match the Runtime Substrate responsibility list.
|
| 189 |
+
- `--kind` must be one of `cli`, `binary`, `mcp`, `skill`, `http`.
|
| 190 |
+
- `--capability` must be kebab-case (lowercase letters, digits, single hyphens);
|
| 191 |
+
spaces and underscores are normalized to hyphens.
|
| 192 |
+
- `--args` entries must use `name:type:required` or
|
| 193 |
+
`name:type:required:help`, with `required` or `optional` as the third field.
|
| 194 |
+
- For `cli`/`binary`, the command must exist as a path or on `PATH`, unless
|
| 195 |
+
`--force` is supplied. `mcp`/`skill`/`http` skip this check.
|
docs/TRACE_SPEC.md
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Trace Specification
|
| 2 |
+
|
| 3 |
+
The `trace` table records what happened during a Harness task. This document
|
| 4 |
+
defines the expected depth and format for each field so traces are useful for
|
| 5 |
+
review, benchmark scoring, failure attribution, and future harness evolution.
|
| 6 |
+
|
| 7 |
+
The current schema lives in `scripts/schema/001-init.sql` under the `trace`
|
| 8 |
+
table. The schema is not changed by Phase 2.
|
| 9 |
+
|
| 10 |
+
## Field Reference
|
| 11 |
+
|
| 12 |
+
| Field | Type | Required | Format | Example |
|
| 13 |
+
| --- | --- | --- | --- | --- |
|
| 14 |
+
| `id` | INTEGER | Automatic | SQLite autoincrement primary key. Do not set manually. | `42` |
|
| 15 |
+
| `created_at` | TEXT | Automatic | SQLite `datetime('now')`. Do not set manually. | `2026-05-27 09:24:37` |
|
| 16 |
+
| `task_summary` | TEXT | Yes | One sentence, at least 10 characters, naming the outcome or attempted outcome. | `Completed Phase 2 docs-only observability and taxonomy specification` |
|
| 17 |
+
| `intake_id` | INTEGER | Standard+ when an intake was recorded | Integer id from the related `intake` row. | `36` |
|
| 18 |
+
| `story_id` | TEXT | Standard+ when work maps to one story | Story id from the `story` table. Use the main story when one trace covers several; list the rest in `notes`. | `US-004` |
|
| 19 |
+
| `agent` | TEXT | Optional for minimal; Standard+ expected | Short agent/tool name. | `codex` |
|
| 20 |
+
| `actions_taken` | TEXT | Standard+ | JSON array text. With the current CLI, pass a comma-separated list and the CLI stores JSON text. | `["read PHASE2.md","drafted TRACE_SPEC.md","updated HARNESS.md"]` |
|
| 21 |
+
| `files_read` | TEXT | Standard+ | JSON array text of paths or command names. With the current CLI, pass a comma-separated list. | `["PHASE2.md","docs/HARNESS.md","scripts/bin/harness-cli query matrix"]` |
|
| 22 |
+
| `files_changed` | TEXT | Standard+ | JSON array text of changed file paths. With the current CLI, pass a comma-separated list; omit only when no files changed. | `["docs/TRACE_SPEC.md","docs/HARNESS.md"]` |
|
| 23 |
+
| `decisions_made` | TEXT | Detailed | JSON array text of decision strings. Include scope decisions, validation choices, and explicit non-goals. | `["Kept Phase 2 docs-only; installer propagation remains out of scope"]` |
|
| 24 |
+
| `errors` | TEXT | Standard+ if errors occurred; Detailed always | JSON array text of error or blocker strings. Until the CLI supports empty arrays directly, use `none` when a detailed trace needs explicit no-error evidence. | `["git diff --check failed before whitespace fix"]` |
|
| 25 |
+
| `outcome` | TEXT | Yes before final response | One of `completed`, `blocked`, `partial`, or `failed`. | `completed` |
|
| 26 |
+
| `duration_seconds` | INTEGER | Detailed when available | Positive integer estimate or measured duration. Leave null if unknown. | `1800` |
|
| 27 |
+
| `token_estimate` | INTEGER | Detailed when available | Positive integer estimate. Leave null if unknown. | `24000` |
|
| 28 |
+
| `harness_friction` | TEXT | Standard+ when friction exists; Detailed always | Free text naming what was hard, missing, ambiguous, or repeated. Use `none` only when the agent actively checked and found no friction. | `New Phase 2 docs are not in installer copy list; recorded as out-of-scope follow-up.` |
|
| 29 |
+
| `notes` | TEXT | Optional | Free text for review context that does not fit other fields. | `Trace covers US-003, US-004, US-005, and US-006.` |
|
| 30 |
+
|
| 31 |
+
## Quality Tiers
|
| 32 |
+
|
| 33 |
+
### Minimal (score: 1)
|
| 34 |
+
|
| 35 |
+
Minimum fields:
|
| 36 |
+
|
| 37 |
+
- `task_summary` is filled and at least 10 characters.
|
| 38 |
+
- `outcome` is filled before the final response.
|
| 39 |
+
|
| 40 |
+
Acceptable for:
|
| 41 |
+
|
| 42 |
+
- Tiny-lane tasks with no file changes or only low-risk copy/doc edits.
|
| 43 |
+
|
| 44 |
+
Not acceptable for:
|
| 45 |
+
|
| 46 |
+
- Normal or high-risk work.
|
| 47 |
+
- Any work that discovered friction, errors, or a missing validation path.
|
| 48 |
+
|
| 49 |
+
### Standard (score: 2)
|
| 50 |
+
|
| 51 |
+
Minimum fields:
|
| 52 |
+
|
| 53 |
+
- All Minimal fields.
|
| 54 |
+
- `intake_id` when an intake was recorded.
|
| 55 |
+
- `story_id` when the work maps cleanly to one story.
|
| 56 |
+
- `agent`.
|
| 57 |
+
- `actions_taken` as JSON array text.
|
| 58 |
+
- `files_read` as JSON array text.
|
| 59 |
+
- `files_changed` as JSON array text.
|
| 60 |
+
- At least one of `errors` or `harness_friction`.
|
| 61 |
+
|
| 62 |
+
Required for:
|
| 63 |
+
|
| 64 |
+
- Normal-lane tasks.
|
| 65 |
+
- Tiny tasks that changed Harness instructions, validation expectations, or
|
| 66 |
+
durable records.
|
| 67 |
+
|
| 68 |
+
Standard traces may leave `duration_seconds`, `token_estimate`, and
|
| 69 |
+
`decisions_made` empty when those details are not useful.
|
| 70 |
+
|
| 71 |
+
### Detailed (score: 3)
|
| 72 |
+
|
| 73 |
+
Minimum fields:
|
| 74 |
+
|
| 75 |
+
- All Standard fields.
|
| 76 |
+
- `decisions_made` as JSON array text.
|
| 77 |
+
- `errors` as JSON array text, using `none` with the current CLI when no
|
| 78 |
+
errors occurred.
|
| 79 |
+
- `harness_friction`, using `none` only after checking for friction.
|
| 80 |
+
- `duration_seconds` or a note explaining why duration is unavailable.
|
| 81 |
+
- `token_estimate` or a note explaining why token estimate is unavailable.
|
| 82 |
+
- `notes` when one trace covers multiple stories, multiple risk flags, or
|
| 83 |
+
skipped validation.
|
| 84 |
+
|
| 85 |
+
Required for:
|
| 86 |
+
|
| 87 |
+
- High-risk tasks.
|
| 88 |
+
- Changes touching architecture direction, source-of-truth hierarchy,
|
| 89 |
+
validation requirements, auth, authorization, data loss, audit/security, or
|
| 90 |
+
external provider behavior.
|
| 91 |
+
- Benchmark or release work where later review needs precise proof.
|
| 92 |
+
|
| 93 |
+
For high-risk work, `decisions_made` in the trace summarizes what was decided.
|
| 94 |
+
It does not replace a durable decision record. If the work changes behavior,
|
| 95 |
+
architecture, authorization, data ownership, API shape, or validation
|
| 96 |
+
requirements, add a `docs/decisions/NNNN-*.md` file and record it with
|
| 97 |
+
`scripts/bin/harness-cli decision add`.
|
| 98 |
+
|
| 99 |
+
## Lane Mapping
|
| 100 |
+
|
| 101 |
+
| Lane | Expected Tier | Minimum Trace Behavior |
|
| 102 |
+
| --- | --- | --- |
|
| 103 |
+
| Tiny | Minimal | Record summary and outcome; use Standard if friction or Harness docs changed. |
|
| 104 |
+
| Normal | Standard | Record intake, actions, files read, files changed, outcome, and friction/errors. |
|
| 105 |
+
| High-risk | Detailed | Record all fields or explicitly explain unavailable duration/token estimates. |
|
| 106 |
+
|
| 107 |
+
## Friction Capture Protocol
|
| 108 |
+
|
| 109 |
+
Populate `harness_friction` when any of these occur:
|
| 110 |
+
|
| 111 |
+
- The agent had to infer a missing rule or source of truth.
|
| 112 |
+
- Required validation was unclear, unavailable, or too expensive to run.
|
| 113 |
+
- A document, durable record, or story packet was stale or contradictory.
|
| 114 |
+
- The task revealed a repeated manual step that should become a template,
|
| 115 |
+
command, or checklist.
|
| 116 |
+
- A requested change was out of scope but likely important later.
|
| 117 |
+
- A benchmark or review failure could not be attributed to a component.
|
| 118 |
+
|
| 119 |
+
How to write friction:
|
| 120 |
+
|
| 121 |
+
- Name the concrete pain, not a vague mood.
|
| 122 |
+
- Include the missing capability or contradiction.
|
| 123 |
+
- If the friction should become work, also add or update a backlog item with
|
| 124 |
+
`scripts/bin/harness-cli backlog add`.
|
| 125 |
+
- If there was no friction, use `none` only for Detailed traces.
|
| 126 |
+
|
| 127 |
+
Good friction:
|
| 128 |
+
|
| 129 |
+
```text
|
| 130 |
+
New Phase 2 docs are not copied by scripts/install-harness.sh, but installer
|
| 131 |
+
propagation is out of scope for docs-only Phase 2.
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
Weak friction:
|
| 135 |
+
|
| 136 |
+
```text
|
| 137 |
+
docs confusing
|
| 138 |
+
```
|
| 139 |
+
|
| 140 |
+
## Examples
|
| 141 |
+
|
| 142 |
+
### Good Trace (Detailed)
|
| 143 |
+
|
| 144 |
+
```bash
|
| 145 |
+
scripts/bin/harness-cli trace \
|
| 146 |
+
--summary "Completed high-risk auth role migration with audit proof" \
|
| 147 |
+
--intake 51 \
|
| 148 |
+
--story US-014 \
|
| 149 |
+
--agent codex \
|
| 150 |
+
--outcome completed \
|
| 151 |
+
--duration 4200 \
|
| 152 |
+
--tokens 52000 \
|
| 153 |
+
--actions "read access-control docs,created migration,updated audit tests,ran integration suite" \
|
| 154 |
+
--read "docs/product/permissions.md,docs/decisions/0008-auth-boundary.md,src/auth/roles.ts" \
|
| 155 |
+
--changed "src/auth/roles.ts,src/audit/events.ts,tests/auth-roles.test.ts" \
|
| 156 |
+
--decisions "kept manager role scoped to workspace,recorded audit event on every role change" \
|
| 157 |
+
--errors "none" \
|
| 158 |
+
--friction "Existing permission docs did not define delegated admin; added backlog item for role glossary." \
|
| 159 |
+
--notes "Detailed trace required because the task touched authorization and audit behavior."
|
| 160 |
+
```
|
| 161 |
+
|
| 162 |
+
### Adequate Trace (Standard)
|
| 163 |
+
|
| 164 |
+
```bash
|
| 165 |
+
scripts/bin/harness-cli trace \
|
| 166 |
+
--summary "Added Phase 2 trace specification and Harness reference" \
|
| 167 |
+
--intake 36 \
|
| 168 |
+
--story US-004 \
|
| 169 |
+
--agent codex \
|
| 170 |
+
--outcome completed \
|
| 171 |
+
--actions "read PHASE2.md,drafted TRACE_SPEC.md,updated HARNESS.md,ran rg checks" \
|
| 172 |
+
--read "PHASE2.md,docs/HARNESS.md,scripts/schema/001-init.sql" \
|
| 173 |
+
--changed "docs/TRACE_SPEC.md,docs/HARNESS.md" \
|
| 174 |
+
--friction "none"
|
| 175 |
+
```
|
| 176 |
+
|
| 177 |
+
### Insufficient Trace
|
| 178 |
+
|
| 179 |
+
```bash
|
| 180 |
+
scripts/bin/harness-cli trace \
|
| 181 |
+
--summary "did phase 2" \
|
| 182 |
+
--outcome completed
|
| 183 |
+
```
|
| 184 |
+
|
| 185 |
+
Why this is insufficient for normal-lane Phase 2 work:
|
| 186 |
+
|
| 187 |
+
- It does not identify actions.
|
| 188 |
+
- It does not list files read or changed.
|
| 189 |
+
- It does not connect to intake or stories.
|
| 190 |
+
- It gives no friction or error signal.
|
| 191 |
+
|
| 192 |
+
## Review Checklist
|
| 193 |
+
|
| 194 |
+
Before the final response, check:
|
| 195 |
+
|
| 196 |
+
- The trace tier matches the lane.
|
| 197 |
+
- Review the score printed automatically by `scripts/bin/harness-cli trace`.
|
| 198 |
+
Use `scripts/bin/harness-cli score-trace --id N` when re-checking a specific
|
| 199 |
+
historical trace.
|
| 200 |
+
- `files_changed` matches the actual changed-file set at a useful level.
|
| 201 |
+
- `errors` names real blockers or is `none` for Detailed traces when the
|
| 202 |
+
current CLI is used.
|
| 203 |
+
- `harness_friction` either names a concrete issue or is intentionally `none`.
|
| 204 |
+
- Any friction that should become future work is recorded in the backlog.
|
docs/decisions/0001-harness-first-development.md
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0001 Harness-First Development
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-05
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The repository currently contains a product README and a large product
|
| 12 |
+
specification. There is no application implementation yet.
|
| 13 |
+
|
| 14 |
+
The project will likely involve human direction plus agent implementation over
|
| 15 |
+
many evolving stories. A single massive specification is not enough for safe
|
| 16 |
+
agent work because it becomes hard to locate current truth, risk, proof, and
|
| 17 |
+
change history.
|
| 18 |
+
|
| 19 |
+
## Decision
|
| 20 |
+
|
| 21 |
+
Create Harness v0 before scaffolding product code.
|
| 22 |
+
|
| 23 |
+
Harness v0 defines:
|
| 24 |
+
|
| 25 |
+
- Agent entrypoint.
|
| 26 |
+
- Product doc split.
|
| 27 |
+
- Feature intake and risk lanes.
|
| 28 |
+
- Story packet templates.
|
| 29 |
+
- Decision records.
|
| 30 |
+
- Test matrix.
|
| 31 |
+
- Harness backlog.
|
| 32 |
+
|
| 33 |
+
No application code, fake scripts, CI, or tests are created in this decision.
|
| 34 |
+
|
| 35 |
+
## Consequences
|
| 36 |
+
|
| 37 |
+
Positive:
|
| 38 |
+
|
| 39 |
+
- Agents have a clear operating model before implementation starts.
|
| 40 |
+
- Product truth can split away from the massive spec.
|
| 41 |
+
- Risky work has a slower lane before code changes.
|
| 42 |
+
- Harness growth becomes part of the work.
|
| 43 |
+
|
| 44 |
+
Tradeoffs:
|
| 45 |
+
|
| 46 |
+
- Some docs are placeholders until real stories exercise them.
|
| 47 |
+
- Validation commands are only contracts until implementation begins.
|
| 48 |
+
- The harness must stay small enough to revise from real friction.
|
docs/decisions/0002-post-spec-product-lifecycle.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0002 Seed Specification Product Lifecycle
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-05
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Superseded by `0003-generic-spec-intake-harness.md`
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Harness v0 originally assumed the repository would include one seed
|
| 12 |
+
specification file for the first product. This decision explained how agents
|
| 13 |
+
should decompose that initial specification into product docs, story packets,
|
| 14 |
+
implementation, and validation proof, then continue working after the seed was
|
| 15 |
+
exhausted.
|
| 16 |
+
|
| 17 |
+
That approach fit a single project but made the harness less reusable.
|
| 18 |
+
|
| 19 |
+
## Decision
|
| 20 |
+
|
| 21 |
+
Treat the initial specification as a seed and historical snapshot, not the
|
| 22 |
+
permanent living product plan.
|
| 23 |
+
|
| 24 |
+
After the initial specification has been exhausted, new work should enter
|
| 25 |
+
through the same harness loop as one of these input types:
|
| 26 |
+
|
| 27 |
+
- Change request.
|
| 28 |
+
- New initiative.
|
| 29 |
+
- Maintenance request.
|
| 30 |
+
- Harness improvement.
|
| 31 |
+
|
| 32 |
+
Product docs under `docs/product/`, story packets under `docs/stories/`,
|
| 33 |
+
validation evidence in `docs/TEST_MATRIX.md`, and decision records under
|
| 34 |
+
`docs/decisions/` become the living operating surface.
|
| 35 |
+
|
| 36 |
+
Large future product areas should be captured as scoped initiative notes instead
|
| 37 |
+
of appended to the seed specification or rewritten as a second monolithic spec.
|
| 38 |
+
|
| 39 |
+
## Consequences
|
| 40 |
+
|
| 41 |
+
Positive:
|
| 42 |
+
|
| 43 |
+
- The original specification remains stable as historical context.
|
| 44 |
+
- Product truth moves into smaller, current, maintainable files.
|
| 45 |
+
- Future work keeps using the same intake, story, proof, and harness-growth
|
| 46 |
+
loop.
|
| 47 |
+
- Large ideas can still be planned without creating another oversized spec.
|
| 48 |
+
|
| 49 |
+
Tradeoffs:
|
| 50 |
+
|
| 51 |
+
- The repository will eventually need an initiative template if large new
|
| 52 |
+
product areas become common.
|
| 53 |
+
- Agents must be careful to update product docs and tests rather than relying on
|
| 54 |
+
the seed specification after initial buildout.
|
docs/decisions/0003-generic-spec-intake-harness.md
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0003 Generic Spec Intake Harness
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-05
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Harness v0 originally shipped with a project-specific `SPEC.md`, product docs,
|
| 12 |
+
candidate epics, architecture assumptions, and validation examples. That made
|
| 13 |
+
the harness useful for the first project but too specific to reuse as the outer
|
| 14 |
+
shell for a new project.
|
| 15 |
+
|
| 16 |
+
The desired direction is a default harness that can wait for any user-provided
|
| 17 |
+
spec, derive product docs from that spec, and then continue with the same
|
| 18 |
+
intake, story, proof, and decision loop.
|
| 19 |
+
|
| 20 |
+
## Decision
|
| 21 |
+
|
| 22 |
+
Remove the tracked project-specific spec and pre-sliced product domains from
|
| 23 |
+
Harness v0.
|
| 24 |
+
|
| 25 |
+
The harness now starts with:
|
| 26 |
+
|
| 27 |
+
- No baked-in `SPEC.md`.
|
| 28 |
+
- Empty product docs except for intake guidance.
|
| 29 |
+
- Generic story and epic examples.
|
| 30 |
+
- Stack-neutral architecture discovery rules.
|
| 31 |
+
- Stack-neutral validation columns.
|
| 32 |
+
- A source hierarchy that treats a future user-provided spec as input material,
|
| 33 |
+
not permanent living truth.
|
| 34 |
+
|
| 35 |
+
## Alternatives Considered
|
| 36 |
+
|
| 37 |
+
1. Keep the original `SPEC.md` as an example. Rejected because examples can be
|
| 38 |
+
mistaken for current product truth.
|
| 39 |
+
2. Move the original product docs into an examples folder. Rejected for now
|
| 40 |
+
because the user asked for a clean default harness.
|
| 41 |
+
|
| 42 |
+
## Consequences
|
| 43 |
+
|
| 44 |
+
Positive:
|
| 45 |
+
|
| 46 |
+
- The repository is easier to reuse for any new project.
|
| 47 |
+
- Future specs can define their own product domains and stack.
|
| 48 |
+
- Agents are less likely to confuse template truth with product truth.
|
| 49 |
+
|
| 50 |
+
Tradeoffs:
|
| 51 |
+
|
| 52 |
+
- The harness has fewer concrete examples until the next spec is supplied.
|
| 53 |
+
- The first spec intake must create product docs and candidate epics before
|
| 54 |
+
implementation planning can be precise.
|
| 55 |
+
|
| 56 |
+
## Follow-Up
|
| 57 |
+
|
| 58 |
+
- Add a spec-intake template if repeated projects reveal a stable format.
|
docs/decisions/0004-sqlite-durable-layer.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0004 SQLite Durable Layer
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-22
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Harness v0 stores all operational data in markdown files: `TEST_MATRIX.md` rows,
|
| 12 |
+
`HARNESS_BACKLOG.md` items, decision records, and story status. This works for
|
| 13 |
+
human reading but creates friction for agents:
|
| 14 |
+
|
| 15 |
+
- Editing markdown tables is error-prone and hard to validate.
|
| 16 |
+
- There is no structured way to query past intakes, traces, or friction reports.
|
| 17 |
+
- The harness has no observability foundation for future evolution.
|
| 18 |
+
|
| 19 |
+
Recent research on harness engineering (arXiv:2604.25850, arXiv:2605.13357,
|
| 20 |
+
arXiv:2603.28052) identifies observability and structured traces as the
|
| 21 |
+
foundation for harness improvement. All three approaches require queryable
|
| 22 |
+
operational data, not prose documents.
|
| 23 |
+
|
| 24 |
+
## Decision
|
| 25 |
+
|
| 26 |
+
Add a SQLite database (`harness.db`) and a thin CLI (`scripts/bin/harness-cli`) as the
|
| 27 |
+
durable layer for operational harness data.
|
| 28 |
+
|
| 29 |
+
The database stores:
|
| 30 |
+
|
| 31 |
+
- **Intake records**: classification of incoming work.
|
| 32 |
+
- **Stories**: work packets and their validation proof status (replaces manual
|
| 33 |
+
`TEST_MATRIX.md` rows).
|
| 34 |
+
- **Decisions**: durable records with optional verification commands.
|
| 35 |
+
- **Backlog items**: harness improvement proposals with predicted and actual
|
| 36 |
+
impact.
|
| 37 |
+
- **Traces**: agent execution records including actions, files, errors, outcome,
|
| 38 |
+
and harness friction.
|
| 39 |
+
|
| 40 |
+
The schema is version-controlled under `scripts/schema/`. The database file is
|
| 41 |
+
`.gitignore`d because each project instance generates its own operational data.
|
| 42 |
+
|
| 43 |
+
Policy docs (`HARNESS.md`, `FEATURE_INTAKE.md`, `ARCHITECTURE.md`) remain as
|
| 44 |
+
human-readable references. The database stores what agents produce, not what
|
| 45 |
+
they should do.
|
| 46 |
+
|
| 47 |
+
## Alternatives Considered
|
| 48 |
+
|
| 49 |
+
1. Keep everything in markdown. Rejected because it prevents structured queries,
|
| 50 |
+
makes observability impossible, and forces agents to edit fragile tables.
|
| 51 |
+
2. Use JSON files. Rejected because concurrent writes are unsafe and queries
|
| 52 |
+
require custom tooling.
|
| 53 |
+
3. Use a full database server. Rejected because it adds deployment complexity
|
| 54 |
+
that does not match Harness v0 scope.
|
| 55 |
+
|
| 56 |
+
## Consequences
|
| 57 |
+
|
| 58 |
+
Positive:
|
| 59 |
+
|
| 60 |
+
- Agents record structured data instead of editing markdown tables.
|
| 61 |
+
- Intake, story, decision, backlog, and trace data is queryable.
|
| 62 |
+
- The harness has an observability foundation for future evolution.
|
| 63 |
+
- Schema migrations enable the durable layer to grow with the harness.
|
| 64 |
+
|
| 65 |
+
Tradeoffs:
|
| 66 |
+
|
| 67 |
+
- Requires `sqlite3` to be available in the environment.
|
| 68 |
+
- The database is not version-controlled, so each instance starts empty.
|
| 69 |
+
- Markdown docs and the database may drift if agents use one but not the other.
|
| 70 |
+
|
| 71 |
+
## Follow-Up
|
| 72 |
+
|
| 73 |
+
- Seed existing decisions (0001-0003) into the database during init.
|
| 74 |
+
- Add context engineering rules keyed by task type and risk lane.
|
| 75 |
+
- Add harness maturity ladder (H0-H4) once the durable layer proves useful.
|
docs/decisions/0005-prebuilt-rust-harness-cli.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0005 Prebuilt Rust Harness CLI
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-23
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted, amended 2026-05-31, amended 2026-06-09
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The durable layer started as a thin shell wrapper around SQLite. That wrapper
|
| 12 |
+
is now large enough to carry meaningful architecture risk: it mixes command
|
| 13 |
+
parsing, SQL construction, migrations, import behavior, query rendering, and
|
| 14 |
+
help text in one script.
|
| 15 |
+
|
| 16 |
+
The previous installer copied a shell wrapper into target repositories. That
|
| 17 |
+
kept Harness easy to install, but it also meant a Rust rewrite was not only an
|
| 18 |
+
implementation change. It changed the distribution contract for every project
|
| 19 |
+
that receives Harness.
|
| 20 |
+
|
| 21 |
+
## Decision
|
| 22 |
+
|
| 23 |
+
The future Rust implementation of the Harness CLI should be shipped as a
|
| 24 |
+
prebuilt binary downloaded by the installer.
|
| 25 |
+
|
| 26 |
+
The command path for users and agents is the installed Rust binary:
|
| 27 |
+
|
| 28 |
+
```bash
|
| 29 |
+
scripts/bin/harness-cli <command>
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
On Windows, the repository-local binary is installed as:
|
| 33 |
+
|
| 34 |
+
```powershell
|
| 35 |
+
.\scripts\bin\harness-cli.exe <command>
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
The installer should download, verify, and install the platform-specific Rust
|
| 39 |
+
binary directly at that path. There should be no shell wrapper command contract.
|
| 40 |
+
|
| 41 |
+
The Rust CLI should follow the existing architecture rules:
|
| 42 |
+
|
| 43 |
+
- Domain: harness records, statuses, lanes, and value types.
|
| 44 |
+
- Application: use cases for intake, stories, decisions, backlog, traces, and
|
| 45 |
+
queries.
|
| 46 |
+
- Infrastructure: SQLite repositories and schema migrations.
|
| 47 |
+
- Interface: command-line parsing, terminal output, and installer integration.
|
| 48 |
+
|
| 49 |
+
Release automation now follows the same distribution contract. After a PR is
|
| 50 |
+
merged to `main`, the post-merge maintenance workflow updates `CHANGELOG.md`.
|
| 51 |
+
When the merged PR changed the Rust CLI source, schema, Cargo metadata, or CLI
|
| 52 |
+
release packaging, it also bumps the CLI patch version, updates the installer
|
| 53 |
+
release tag pin, creates a `harness-cli-v*` tag, and invokes the reusable
|
| 54 |
+
Harness CLI release workflow for that tag.
|
| 55 |
+
|
| 56 |
+
## Alternatives Considered
|
| 57 |
+
|
| 58 |
+
1. Keep the shell CLI permanently. Rejected because the script has crossed from
|
| 59 |
+
a thin wrapper into a growing application surface with weak testability.
|
| 60 |
+
2. Copy Rust source into every target project and build locally. Rejected
|
| 61 |
+
because it makes Harness installation depend on a local Rust toolchain and
|
| 62 |
+
increases setup friction for projects that only need the harness.
|
| 63 |
+
3. Require users to install a global `harness` binary separately. Rejected
|
| 64 |
+
because Harness should remain repository-local for agents.
|
| 65 |
+
4. Download a prebuilt binary through the installer. Accepted because it keeps
|
| 66 |
+
target repos simple while allowing the CLI internals to become typed,
|
| 67 |
+
testable, and platform-aware.
|
| 68 |
+
|
| 69 |
+
## Consequences
|
| 70 |
+
|
| 71 |
+
Positive:
|
| 72 |
+
|
| 73 |
+
- The durable-layer CLI can move to typed command parsing and tested use cases.
|
| 74 |
+
- Target projects do not need a Rust toolchain just to use Harness.
|
| 75 |
+
- The `scripts/bin/harness-cli` command is the stable entrypoint for agents on
|
| 76 |
+
macOS/Linux; Windows uses the same repo-local path with the `.exe` suffix.
|
| 77 |
+
- Prebuilt releases can include a known SQLite linkage strategy.
|
| 78 |
+
|
| 79 |
+
Tradeoffs:
|
| 80 |
+
|
| 81 |
+
- The installer must learn platform detection and binary download behavior.
|
| 82 |
+
- Release artifacts need checksums or another integrity check.
|
| 83 |
+
- Unsupported platforms need a clear error path.
|
| 84 |
+
- The project needs a repeatable release process for supported platforms.
|
| 85 |
+
|
| 86 |
+
## Follow-Up
|
| 87 |
+
|
| 88 |
+
- Implement the migration through `US-002 Rust Harness CLI`.
|
| 89 |
+
- Remove the old shell wrapper from installed project payloads.
|
| 90 |
+
- Add checksum verification for downloaded binaries.
|
| 91 |
+
- Treat the Rust CLI as the primary durable-layer implementation.
|
docs/decisions/0006-phase-4-benchmark-triage.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0006 Phase 4 Benchmark Triage
|
| 2 |
+
|
| 3 |
+
Date: 2026-05-31
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The first Phase 4 benchmark re-run found that T4 authentication failed
|
| 12 |
+
`decision_recorded` even though the trace included decisions text. The same run
|
| 13 |
+
also showed command churn from agents trying `story update` proof flags with
|
| 14 |
+
`yes` and `no`, and trying to use `story verify` as if it accepted proof flags.
|
| 15 |
+
|
| 16 |
+
## Decision
|
| 17 |
+
|
| 18 |
+
Harness instructions and CLI help must distinguish durable records from trace
|
| 19 |
+
evidence and must show the current Rust CLI command shape at the point agents
|
| 20 |
+
need it:
|
| 21 |
+
|
| 22 |
+
- High-risk behavior changes require a markdown decision under
|
| 23 |
+
`docs/decisions/` and a durable `decision` row.
|
| 24 |
+
- Trace `--decisions` is evidence for trace quality, not the decision log.
|
| 25 |
+
- `story update` proof flags use `1` and `0`.
|
| 26 |
+
- `story verify <id>` only runs the configured `verify_command`; proof flags
|
| 27 |
+
stay on `story update`.
|
| 28 |
+
|
| 29 |
+
## Alternatives Considered
|
| 30 |
+
|
| 31 |
+
1. Rely on trace auto-scoring to catch the missing T4 decision. Rejected because
|
| 32 |
+
trace scoring can confirm detailed trace content but cannot prove a durable
|
| 33 |
+
decision record exists.
|
| 34 |
+
2. Change the CLI to accept `yes` and `no`. Deferred because v0.1.5 already has
|
| 35 |
+
a numeric command contract and the immediate benchmark issue is stale
|
| 36 |
+
guidance, not missing parser capability.
|
| 37 |
+
|
| 38 |
+
## Consequences
|
| 39 |
+
|
| 40 |
+
Positive:
|
| 41 |
+
|
| 42 |
+
- High-risk agents get explicit decision-log instructions before closing work.
|
| 43 |
+
- Command examples align with the Rust CLI v0.1.5 parser.
|
| 44 |
+
- `story verify` and `story update` have separate mental models in docs.
|
| 45 |
+
|
| 46 |
+
Tradeoffs:
|
| 47 |
+
|
| 48 |
+
- Docs now duplicate a few command examples so the common path is visible
|
| 49 |
+
without repeated help discovery.
|
| 50 |
+
|
| 51 |
+
## Follow-Up
|
| 52 |
+
|
| 53 |
+
- Re-run the Phase 4 benchmark and check whether T4 records a durable decision.
|
| 54 |
+
- Watch for remaining command churn around `story update` and `story verify`.
|
docs/decisions/0007-improvement-proposal-rules.md
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0007 Improvement Proposal Rules
|
| 2 |
+
|
| 3 |
+
Date: 2026-06-04
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Phase 5 adds `harness-cli propose`, which changes the harness evolution model.
|
| 12 |
+
The command must be useful without becoming an unchecked source of scope creep
|
| 13 |
+
or circular recommendations.
|
| 14 |
+
|
| 15 |
+
## Decision
|
| 16 |
+
|
| 17 |
+
Improvement proposals are advisory, rule-based, and evidence-backed. The command
|
| 18 |
+
may summarize repeated friction, repeated interventions, and audit drift. It may
|
| 19 |
+
create `proposed` backlog items only when `--commit` is supplied.
|
| 20 |
+
|
| 21 |
+
Every proposal must include:
|
| 22 |
+
|
| 23 |
+
- affected Harness component,
|
| 24 |
+
- concrete evidence,
|
| 25 |
+
- predicted impact,
|
| 26 |
+
- risk lane,
|
| 27 |
+
- suggested action,
|
| 28 |
+
- validation plan,
|
| 29 |
+
- confidence level.
|
| 30 |
+
|
| 31 |
+
High-risk proposal implementation still requires human review and a durable
|
| 32 |
+
decision record when it changes source hierarchy, architecture direction,
|
| 33 |
+
validation requirements, or risk policy.
|
| 34 |
+
|
| 35 |
+
## Alternatives Considered
|
| 36 |
+
|
| 37 |
+
1. Generate free-form LLM recommendations. Rejected because Phase 5 needs a
|
| 38 |
+
deterministic and auditable evolution role.
|
| 39 |
+
2. Automatically apply proposed changes. Rejected because the harness must not
|
| 40 |
+
rewrite its own policy without review.
|
| 41 |
+
3. Only report audit findings. Rejected because H5 requires proposed
|
| 42 |
+
improvements, not just drift detection.
|
| 43 |
+
|
| 44 |
+
## Consequences
|
| 45 |
+
|
| 46 |
+
Positive:
|
| 47 |
+
|
| 48 |
+
- Repeated operational patterns can become backlog items.
|
| 49 |
+
- Proposal output is explainable and testable.
|
| 50 |
+
- Human review remains the gate for risky harness changes.
|
| 51 |
+
|
| 52 |
+
Tradeoffs:
|
| 53 |
+
|
| 54 |
+
- Rule-based grouping can miss semantically similar phrasing.
|
| 55 |
+
- Audit-based proposals may be housekeeping rather than strategic evolution.
|
| 56 |
+
|
| 57 |
+
## Follow-Up
|
| 58 |
+
|
| 59 |
+
- Use benchmark runs and closed backlog outcomes to improve proposal quality in
|
| 60 |
+
later phases.
|
docs/decisions/0008-carepath-harness-adoption.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0008 CarePath Harness Adoption
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-12
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
CarePath has two clinical workflows, strict safety invariants, multiple test
|
| 12 |
+
surfaces, and historical plans. Future coding agents need a durable way to
|
| 13 |
+
classify work, preserve those boundaries, and record actual proof.
|
| 14 |
+
|
| 15 |
+
## Decision
|
| 16 |
+
|
| 17 |
+
Adopt Repository Harness from upstream commit `14e6f102a4a645562d046f7c693c61401261cac6`
|
| 18 |
+
with its pinned, checksum-verified Windows CLI v0.1.11. Keep `AGENTS.md` as the
|
| 19 |
+
highest repository-level safety and UX instruction, tailor the Harness to the
|
| 20 |
+
current CarePath contracts, and load the same instructions from `CLAUDE.md`.
|
| 21 |
+
|
| 22 |
+
## Alternatives Considered
|
| 23 |
+
|
| 24 |
+
1. Keep only historical plans and ad-hoc validation notes.
|
| 25 |
+
2. Adopt a documentation-only workflow without durable local records.
|
| 26 |
+
3. Replace the existing CarePath instructions with the generic upstream shim.
|
| 27 |
+
|
| 28 |
+
## Consequences
|
| 29 |
+
|
| 30 |
+
Positive:
|
| 31 |
+
|
| 32 |
+
- New work has explicit risk lanes, proof expectations, and durable traces.
|
| 33 |
+
- Codex and Claude Code share the same project contract.
|
| 34 |
+
- The Harness introduces no product dependencies or CI changes.
|
| 35 |
+
|
| 36 |
+
Tradeoffs:
|
| 37 |
+
|
| 38 |
+
- Agents must perform a small intake and trace step for future work.
|
| 39 |
+
- `harness.db` is local-only, so durable operational records are per clone.
|
| 40 |
+
|
| 41 |
+
## Follow-Up
|
| 42 |
+
|
| 43 |
+
- Upgrade the Harness only as a separately reviewed maintenance task.
|
| 44 |
+
- Register optional tools only when a recurring validation need proves one.
|
docs/decisions/0009-restructure-target-layout.md
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0009 CarePath Restructure Target Layout
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Superseded in part by 0016
|
| 8 |
+
|
| 9 |
+
> Location update (2026-07-13): the approved Scribe training boundary now
|
| 10 |
+
> resides at `scribe/training/`. The remaining Interpreter and shared-layout
|
| 11 |
+
> decisions in this record remain accepted.
|
| 12 |
+
|
| 13 |
+
## Context
|
| 14 |
+
|
| 15 |
+
CarePath serves two independent clinical workflows from one process, while the
|
| 16 |
+
repository currently places their runtime, frontends, tests, evaluation, and
|
| 17 |
+
training assets in mixed top-level locations. The approved restructure requires
|
| 18 |
+
a durable target layout without changing import names, routes, or safety
|
| 19 |
+
behavior.
|
| 20 |
+
|
| 21 |
+
## Decision
|
| 22 |
+
|
| 23 |
+
Move runtime ownership into `scribe/` and `interpreter/`, and reserve `shared/`
|
| 24 |
+
for the later shared package. Decision 0016 supersedes this record's original
|
| 25 |
+
top-level GEC-training location. Keep the import names `carepath` and `app`,
|
| 26 |
+
keep all public routes unchanged, and retain the root `pyproject.toml` as the
|
| 27 |
+
Scribe distribution. Move the interpreter evaluation harness with
|
| 28 |
+
`interpreter/`; it is not GEC training.
|
| 29 |
+
|
| 30 |
+
Each relocation phase must update path math, packaging, CI, Docker, deployment
|
| 31 |
+
documentation, and the affected product contracts, then pass its relevant
|
| 32 |
+
existing proof before the next phase begins.
|
| 33 |
+
|
| 34 |
+
## Alternatives Considered
|
| 35 |
+
|
| 36 |
+
1. Leave the current mixed top-level layout.
|
| 37 |
+
2. Rename both runtime imports to match their new directory names.
|
| 38 |
+
3. Create separate Python distributions before the relocations.
|
| 39 |
+
|
| 40 |
+
## Consequences
|
| 41 |
+
|
| 42 |
+
Positive:
|
| 43 |
+
|
| 44 |
+
- Runtime and training ownership are visible from the repository layout.
|
| 45 |
+
- Existing imports and public API routes remain stable.
|
| 46 |
+
- The risk-evaluation suite stays with the Interpreter that it validates.
|
| 47 |
+
|
| 48 |
+
Tradeoffs:
|
| 49 |
+
|
| 50 |
+
- Relative path calculations and deployment files must be audited after every move.
|
| 51 |
+
- The Vercel project root setting requires owner action when `scribe/frontend/` moves.
|
| 52 |
+
|
| 53 |
+
## Follow-Up
|
| 54 |
+
|
| 55 |
+
- Record a separate high-risk Harness intake and trace for every phase.
|
| 56 |
+
- Do not begin owner-led clinical-data collection without legal consent and data-handling approval.
|
docs/decisions/0010-shared-normalization-contract.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0010 Shared Normalization Contract
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Interpreter, Scribe scoring, training scoring, and both lexical retrievers
|
| 12 |
+
carried local normalization implementations. Their names overlapped, but their
|
| 13 |
+
contracts differed: the Interpreter normalizes units, Vietnamese number words,
|
| 14 |
+
and relative dates; scoring compares case-insensitively without semantic
|
| 15 |
+
conversion; lexical retrieval additionally folds diacritics and punctuation.
|
| 16 |
+
|
| 17 |
+
## Decision
|
| 18 |
+
|
| 19 |
+
Create the installable `shared/carepath_shared` package. Move the Interpreter
|
| 20 |
+
normalizer there unchanged as `normalize_text`, and make every former local
|
| 21 |
+
normalizer a direct import from this package. Keep the existing scoring and
|
| 22 |
+
retrieval semantics as the separately named `normalize_for_metrics` and
|
| 23 |
+
`normalize_for_match` functions, rather than silently changing metric scores or
|
| 24 |
+
term matching.
|
| 25 |
+
|
| 26 |
+
## Consequences
|
| 27 |
+
|
| 28 |
+
- Normalization algorithms have one owner and one characterization suite.
|
| 29 |
+
- Existing module import paths remain compatible.
|
| 30 |
+
- A future semantic consolidation must explicitly revise the characterization
|
| 31 |
+
suite and evaluation baseline; it is not part of this restructure.
|
docs/decisions/0011-canonical-medical-term-source.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0011 Canonical Medical-Term Source
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Scribe retrieval reads `data/medical_lexicon.json`; Interpreter safety glossary
|
| 12 |
+
seeding reads `interpreter/app/glossary/data/seed_glossary.csv`. Both are
|
| 13 |
+
clinician-editable medical terminology, but their serving schemas differ.
|
| 14 |
+
|
| 15 |
+
## Decision
|
| 16 |
+
|
| 17 |
+
`shared/carepath_shared/terms/medical_terms.json` is the only authored source.
|
| 18 |
+
Each row defines Vietnamese and English terms, kind, aliases, risk flags, and
|
| 19 |
+
target-specific rendering metadata. `scripts/build_term_artifacts.py` emits
|
| 20 |
+
both existing serving artifacts without changing either runtime reader.
|
| 21 |
+
|
| 22 |
+
Interpreter risk lexicons remain independent safety data and are explicitly
|
| 23 |
+
out of scope for this source.
|
| 24 |
+
|
| 25 |
+
## Consequences
|
| 26 |
+
|
| 27 |
+
- Serving paths and schemas stay stable.
|
| 28 |
+
- CI regenerates artifacts and rejects source/artifact drift.
|
| 29 |
+
- Any taxonomy or risk-flag meaning change now requires this decision's
|
| 30 |
+
characterization and safety proof to be revisited.
|
docs/decisions/0012-interpreter-runtime-hardening.md
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0012 Interpreter Runtime Hardening
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The Interpreter compared admin tokens directly, purged retained sessions only
|
| 12 |
+
at startup, and hard-coded development CORS origins. The combined FastAPI
|
| 13 |
+
service duplicated the standalone Interpreter startup path.
|
| 14 |
+
|
| 15 |
+
## Decision
|
| 16 |
+
|
| 17 |
+
Use `hmac.compare_digest` for the admin token. Give the Interpreter one shared
|
| 18 |
+
lifespan that seeds, purges on startup, runs a daily retention purge task, and
|
| 19 |
+
cancels it on shutdown; the combined service enters the same lifespan. Parse
|
| 20 |
+
CSV `CORS_ORIGINS` through Interpreter settings, with the existing two Vite
|
| 21 |
+
origins as the default.
|
| 22 |
+
|
| 23 |
+
## Consequences
|
| 24 |
+
|
| 25 |
+
- Admin authentication avoids an avoidable timing leak.
|
| 26 |
+
- Retention continues while a process stays up.
|
| 27 |
+
- Standalone and combined Interpreter startup cannot drift.
|
| 28 |
+
- Production operators configure allowed cross-origin development clients via
|
| 29 |
+
`.env`; same-origin deployment remains the default topology.
|
docs/decisions/0013-gec-training-data-governance.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0013 GEC Training Data Governance
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
GEC training previously accepted a dataset name and profile without a durable
|
| 12 |
+
record of its source, consent status, immutable version, or deterministic run
|
| 13 |
+
configuration. Clinical audio is sensitive personal data and cannot be sourced
|
| 14 |
+
or approved by an agent.
|
| 15 |
+
|
| 16 |
+
## Decision
|
| 17 |
+
|
| 18 |
+
Use versioned JSON run configs with fixed seeds and a dataset-manifest reference.
|
| 19 |
+
`run_pipeline.py` refuses every training stage until the referenced manifest has
|
| 20 |
+
owner-approved consent and a non-placeholder SHA-256. Add a text-only frozen
|
| 21 |
+
stratified evaluation fixture with a separate immutable hash; it contains no
|
| 22 |
+
patient audio or identifiers.
|
| 23 |
+
|
| 24 |
+
## Consequences
|
| 25 |
+
|
| 26 |
+
- The agent can validate pipeline governance without collecting clinical data.
|
| 27 |
+
- The owner must complete lawful sourcing, consent, de-identification, and hash
|
| 28 |
+
verification before a real training run.
|
| 29 |
+
- Evaluation categories make drug, dosage, laterality, negation, number, and
|
| 30 |
+
diacritic regressions visible independently.
|
docs/decisions/0014-gec-safety-weighted-regression-gate.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0014 GEC Safety-Weighted Regression Gate
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
Aggregate WER can improve while a correction damages a medication name or a
|
| 12 |
+
dosage. Those errors have a higher clinical cost than ordinary transcription
|
| 13 |
+
differences. The training pipeline also lacked a versioned committed baseline
|
| 14 |
+
that CI could validate without model or GPU dependencies.
|
| 15 |
+
|
| 16 |
+
## Decision
|
| 17 |
+
|
| 18 |
+
Commit a deterministic report for the hashed, text-only frozen GEC fixture.
|
| 19 |
+
Require its report to stay reproducible in CI. A trained adapter must pass the
|
| 20 |
+
existing aggregate gate and, before export, a frozen-fixture gate that rejects
|
| 21 |
+
any regression in `drug_name.term_recall` or
|
| 22 |
+
`dosage.number_unit_preservation` versus raw ASR.
|
| 23 |
+
|
| 24 |
+
## Consequences
|
| 25 |
+
|
| 26 |
+
- Overall WER alone can never approve an adapter that harms drug or dosage
|
| 27 |
+
preservation.
|
| 28 |
+
- CI remains CPU-only: it validates the 12-row fixture, committed report, and
|
| 29 |
+
a fake-adapter export/injected-generation smoke test.
|
| 30 |
+
- A real adapter run still needs the owner-approved training manifest and an
|
| 31 |
+
available GPU; this policy does not authorize data collection or training.
|
docs/decisions/0015-soap-note-measurement-gate.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0015 SOAP Note Measurement Gate
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Superseded by [0017 In-House Testing Scope](0017-in-house-testing-scope.md)
|
| 8 |
+
on 2026-07-13.
|
| 9 |
+
|
| 10 |
+
## Context
|
| 11 |
+
|
| 12 |
+
The Scribe SOAP output uses a hosted clinical LLM with an offline fallback.
|
| 13 |
+
There is no legitimate basis to train or tune it from invented examples, and no
|
| 14 |
+
clinical rating protocol existed to distinguish missing content, hallucination,
|
| 15 |
+
and terminology errors.
|
| 16 |
+
|
| 17 |
+
## Decision
|
| 18 |
+
|
| 19 |
+
Before any SOAP fine-tuning decision, the owner must obtain at least 50
|
| 20 |
+
de-identified pilot notes rated by clinicians under the documented rubric.
|
| 21 |
+
The repository holds only a blank rating schema and a validator that accepts
|
| 22 |
+
anonymous rating metadata; it must never hold source audio, transcripts,
|
| 23 |
+
patient identifiers, or note text.
|
| 24 |
+
|
| 25 |
+
## Consequences
|
| 26 |
+
|
| 27 |
+
- The agent can make the measurement process reproducible without simulating
|
| 28 |
+
clinical evidence.
|
| 29 |
+
- A rating summary makes serious hallucinations and unsafe dispositions visible
|
| 30 |
+
to the owner before choosing training, retrieval, prompting, or no change.
|
| 31 |
+
- The clinical-data study, ratings, and any subsequent model decision remain
|
| 32 |
+
owner-led and require an approved environment.
|
docs/decisions/0016-scribe-training-ownership.md
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0016 Scribe Training Ownership
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The offline DARAG/GEC training code, fixtures, reports, notebooks, and scripts
|
| 12 |
+
already support the Scribe's Vietnamese clinical-note workflow. The top-level
|
| 13 |
+
`training/` directory obscures that ownership and risks being confused with the
|
| 14 |
+
Interpreter's safety evaluation harness at `interpreter/eval/`.
|
| 15 |
+
|
| 16 |
+
## Decision
|
| 17 |
+
|
| 18 |
+
Move the complete offline training boundary to `scribe/training/`. It contains
|
| 19 |
+
the GEC package, configs, fixtures, manifests, reports, notebooks, scripts,
|
| 20 |
+
tests, and SOAP measurement tooling. `interpreter/eval/` remains the
|
| 21 |
+
Interpreter's independent safety regression harness.
|
| 22 |
+
|
| 23 |
+
The Scribe serving package at `scribe/carepath/` must not import training code;
|
| 24 |
+
the production image copies only the Scribe runtime package, not
|
| 25 |
+
`scribe/training/`. Training keeps its standalone `gec.*` command-line import
|
| 26 |
+
boundary without adding a distribution or dependency.
|
| 27 |
+
|
| 28 |
+
This supersedes only the top-level training-location portion of decision 0009.
|
| 29 |
+
|
| 30 |
+
## Consequences
|
| 31 |
+
|
| 32 |
+
- Training commands and CI use `scribe/training/...` paths.
|
| 33 |
+
- Runtime packaging stays independent of offline model-development artifacts.
|
| 34 |
+
- The repository has a visibly separate Scribe training boundary and
|
| 35 |
+
Interpreter safety-evaluation boundary.
|
docs/decisions/0017-in-house-testing-scope.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0017 In-House Testing Scope
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The restructure and public deployment work are complete. The remaining
|
| 12 |
+
50-note clinician-rating target in decision 0015 would require an external
|
| 13 |
+
clinical-evidence program that is outside the current in-house testing scope.
|
| 14 |
+
|
| 15 |
+
## Decision
|
| 16 |
+
|
| 17 |
+
Remove the 50-note clinician-rating target from the restructure completion
|
| 18 |
+
criteria and from the SOAP review utility. In-house review is optional and
|
| 19 |
+
informational: it may inspect synthetic or already-approved de-identified
|
| 20 |
+
material with the existing score-only rubric, but it cannot authorize a SOAP
|
| 21 |
+
fine-tuning, provider, or safety-policy change.
|
| 22 |
+
|
| 23 |
+
No audio, transcript, patient identifier, or note text may enter this
|
| 24 |
+
repository. A future clinical-evidence program or model decision requires a
|
| 25 |
+
separate owner decision.
|
| 26 |
+
|
| 27 |
+
## Consequences
|
| 28 |
+
|
| 29 |
+
- The restructure can complete without external clinical-rating evidence.
|
| 30 |
+
- The rubric remains available for bounded in-house quality checks, without a
|
| 31 |
+
minimum sample-size goal or readiness result.
|
| 32 |
+
- CarePath makes no clinical-quality or model-readiness claim from in-house
|
| 33 |
+
testing.
|
| 34 |
+
|
| 35 |
+
## Alternatives Considered
|
| 36 |
+
|
| 37 |
+
1. Keep the 50-note gate: rejected because it conflicts with the current
|
| 38 |
+
owner-directed in-house scope.
|
| 39 |
+
2. Delete the rubric and validator: rejected because they remain useful for
|
| 40 |
+
privacy-bounded internal QA.
|
docs/decisions/0018-research-only-scribe-model-development.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0018 Research-Only Scribe Model Development
|
| 2 |
+
|
| 3 |
+
Date: 2026-07-13
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The owner requested a research-led Vietnamese medical Scribe pipeline covering
|
| 12 |
+
papers, direct ASR adaptation, runtime-compatible GEC, grounded SOAP training,
|
| 13 |
+
and private Colab staging. Existing decisions authorize GEC governance and
|
| 14 |
+
safety gates, while decision 0017 deliberately prevented optional in-house
|
| 15 |
+
review from implicitly authorizing SOAP fine-tuning.
|
| 16 |
+
|
| 17 |
+
On 2026-07-13 the owner stated, "I approve research-only use of the listed
|
| 18 |
+
public datasets." This approves private research with ViMedCSS, VietMed,
|
| 19 |
+
MTS-Dialog, ACI-BENCH, and MedEV. Public availability and this approval do not
|
| 20 |
+
prove commercial data rights, and synthetic or translated SOAP supervision does
|
| 21 |
+
not establish clinical validity.
|
| 22 |
+
|
| 23 |
+
## Decision
|
| 24 |
+
|
| 25 |
+
Authorize public/synthetic, research-only Scribe model development in private
|
| 26 |
+
Google Colab runtimes under story CP-RES-013.
|
| 27 |
+
|
| 28 |
+
- Compare the current Gipformer baseline, single-best GEC/RAC, and direct
|
| 29 |
+
Vietnamese medical ASR adapters on frozen overall and safety-weighted metrics.
|
| 30 |
+
- Permit one Qwen3-4B QLoRA SOAP adapter used in two grounded passes: extract
|
| 31 |
+
source-supported facts, then write the existing SOAP schema from those facts.
|
| 32 |
+
- Require immutable data manifests, source split provenance, artifact hashes,
|
| 33 |
+
and the labels `usage_scope=research_only` and
|
| 34 |
+
`promotion_status=blocked_research_only`.
|
| 35 |
+
- Keep public benchmark audio in ephemeral Colab caches only. No CarePath
|
| 36 |
+
consultation audio, transcript, note, or identifier enters training.
|
| 37 |
+
- Keep production providers, deployments, public routes, and response schemas
|
| 38 |
+
unchanged. Staging uses an in-process FastAPI client in Colab, not a public
|
| 39 |
+
endpoint.
|
| 40 |
+
- Pin each public source to an immutable commit and verified hashes. Use
|
| 41 |
+
`tranth3truong/carepath-scribe-research` only as the private accepted-bundle
|
| 42 |
+
destination after an explicit upload command; this decision does not perform
|
| 43 |
+
or authorize public publication.
|
| 44 |
+
|
| 45 |
+
MedEV's paper describes research and educational availability, but its dataset
|
| 46 |
+
repository does not declare a standard license. VietMed's Hugging Face card and
|
| 47 |
+
paper also present different license labels. Both therefore remain research-only
|
| 48 |
+
and blocked from commercial or production promotion pending a separate rights
|
| 49 |
+
decision.
|
| 50 |
+
|
| 51 |
+
This decision supersedes decision 0017 only where it said in-house work could
|
| 52 |
+
not authorize SOAP fine-tuning. Decision 0017's privacy boundary and prohibition
|
| 53 |
+
on clinical-readiness claims remain in force.
|
| 54 |
+
|
| 55 |
+
## Consequences
|
| 56 |
+
|
| 57 |
+
- CPU/mock smoke tests can prove governance and safety logic locally; real model
|
| 58 |
+
metrics require owner-run Colab GPU evidence.
|
| 59 |
+
- A successful research bundle still cannot be promoted without a separate data
|
| 60 |
+
rights decision, qualified Vietnamese clinician review, and production rollout
|
| 61 |
+
decision.
|
| 62 |
+
- ROUGE and LLM judges remain diagnostic only. Unsupported critical facts,
|
| 63 |
+
medication/dose corruption, number/unit corruption, or negation corruption
|
| 64 |
+
block export.
|
| 65 |
+
|
| 66 |
+
## Alternatives Considered
|
| 67 |
+
|
| 68 |
+
1. Keep prompt-only SOAP indefinitely: rejected because it cannot test the
|
| 69 |
+
approved research hypothesis.
|
| 70 |
+
2. Train a single end-to-end speech-to-note model: rejected because it weakens
|
| 71 |
+
attribution, comparison, and fail-closed safety evidence.
|
| 72 |
+
3. Reproduce every paper ablation first: rejected because pilot evidence should
|
| 73 |
+
determine whether expensive experiments are warranted.
|
docs/decisions/0019-diacritic-sensitive-risk-matching.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 0019 Diacritic-Sensitive Risk Matching
|
| 2 |
+
|
| 3 |
+
Date: 2026-08-11
|
| 4 |
+
|
| 5 |
+
## Status
|
| 6 |
+
|
| 7 |
+
Accepted
|
| 8 |
+
|
| 9 |
+
## Context
|
| 10 |
+
|
| 11 |
+
The risk engine matched every lexicon term against diacritic-folded text, so
|
| 12 |
+
that undiacritized ASR output ("toi bi di ung") would still reach the allergy
|
| 13 |
+
gate. Folding is lossy in Vietnamese, and unrelated words collapse onto the
|
| 14 |
+
same string:
|
| 15 |
+
|
| 16 |
+
- `nhỏ tai` (ear drops, a route cue) and the `nhớ tái` in `nhớ tái khám`
|
| 17 |
+
(remember to come back) both fold to `nho tai`.
|
| 18 |
+
- `nhẹ` (mild, a critical symptom-severity cue) and `nhé` (a friendly
|
| 19 |
+
sentence-final particle) both fold to `nhe`.
|
| 20 |
+
|
| 21 |
+
Ordinary clinical courtesy therefore produced high and even critical risk
|
| 22 |
+
tiers. `Nhớ tái khám nhé` classified as **critical**, gating a benign sentence
|
| 23 |
+
behind clinician confirmation. Over-gating is the safe direction for a single
|
| 24 |
+
turn, but at this rate the confirmation gate carries no signal: it fires on
|
| 25 |
+
almost everything, so a clinician learns to dismiss it, which is exactly how a
|
| 26 |
+
real dose error gets waved through.
|
| 27 |
+
|
| 28 |
+
The existing 91-case fixture suite could not detect this. It asserts
|
| 29 |
+
`expected_kinds <= produced_kinds`, a subset check, so it constrains recall
|
| 30 |
+
only and is structurally blind to false positives.
|
| 31 |
+
|
| 32 |
+
## Decision
|
| 33 |
+
|
| 34 |
+
Match diacritic-sensitively first. Fall back to folded matching only where the
|
| 35 |
+
matched region of the original text carries no diacritics of its own.
|
| 36 |
+
|
| 37 |
+
The check is applied per matched region rather than per turn, so text that lost
|
| 38 |
+
tone marks on only some words still matches on those words. `Tôi bị di ung
|
| 39 |
+
penicillin` continues to reach the allergy gate.
|
| 40 |
+
|
| 41 |
+
Word-anchor multi-word terms as well as single-word terms; previously only
|
| 42 |
+
single-word terms were anchored and multi-word terms used an unanchored
|
| 43 |
+
substring search.
|
| 44 |
+
|
| 45 |
+
Add `interpreter/eval/fixtures/risk_precision_cases.jsonl` and
|
| 46 |
+
`test_risk_precision_cases`, a second fixture set carrying `max_tier` and
|
| 47 |
+
`forbidden_kinds` so false positives are assertable. It is kept separate from
|
| 48 |
+
`risk_cases.jsonl` so the 30-failure-mode coverage contract there is untouched.
|
| 49 |
+
|
| 50 |
+
## Consequences
|
| 51 |
+
|
| 52 |
+
- Benign Vietnamese no longer gates. `Nhớ tái khám nhé` is `low`, not
|
| 53 |
+
`critical`; `Cảm ơn bác sĩ` is `medium`, not gated.
|
| 54 |
+
- Recall on undiacritized and partially diacritized ASR text is preserved, and
|
| 55 |
+
is now asserted rather than assumed.
|
| 56 |
+
- All 91 risk fixtures, the 6 risk engine tests, and the 50-row mock safety
|
| 57 |
+
eval pass unchanged. Zero misses on critical fixtures still holds.
|
| 58 |
+
- False positives are now a testable property of the engine. Any future
|
| 59 |
+
matching change that reintroduces a folding collision fails
|
| 60 |
+
`test_risk_precision_cases`; the guard was verified by mutation, confirming
|
| 61 |
+
it fails when the fix is reverted.
|
| 62 |
+
- Not addressed, and still open: `uống` (to take orally) is a route cue that
|
| 63 |
+
fires on any sentence about drinking, so `Tôi uống nước` is `high`; and
|
| 64 |
+
Vietnamese number words normalize to digits while English number words do
|
| 65 |
+
not, so `sau ba ngày` against `after three days` raises a false
|
| 66 |
+
`number_mismatch`. Both are lexicon and normalization concerns rather than
|
| 67 |
+
matching concerns, and are left for a separate decision.
|