docs/spec/ — machine-readable spec extract
docs/je_validation_task_spec_3.html is the source of truth. Every file here is either
generated (never hand-edit; regenerate with the command below) or hand-transcribed
from the spec's prose sections (environment.json, scoring.json — these carry a
source_sha256, and the script fails if the HTML changes until they are re-verified).
node scripts/extract_spec_json.mjs
| File | What it holds |
|---|---|
index.json |
All 20 tasks in one line each: id, title, tier, family, sources, tools, score weights. Start here. |
tasks/JE-*.json |
One file per task: scenario, brief, tool surface, seeded ground truth, verifier checks, judge focus, difficulty dials, shortcut guard, fixture manifest (real vs synthesised data). Read the one you are implementing. |
sources.json |
DS-A … DS-J data sources with access commands, volumes, ingest gotchas and licence notes. |
tools.json |
The 12 environment tools with signatures and design intent. |
postingiser.json |
The 7 steps that turn DS-A single-sided expenditure lines into double-entry journal entries. |
run_config.json |
Difficulty knobs and model list only. The console's full run-config contract shape (incl. judge: global_rubric_v1, 3 judges on L4) is transcribed in environment.json under console_run_config_contract. |
sample_run.json |
A full worked JE-06 episode with step-by-step scoring, including the auto-fail. The reference for how scoring must behave. |
environment.json |
Hand-transcribed: episode loop, trajectory record, derived process metrics (three definitions are marked as transcriber inferences — the HTML only names them), fabrication rule, console contract. |
scoring.json |
Hand-transcribed: component definitions and algebra, judge rubric + protocol, pass bar, anti-gaming rules. |
Two spec sections are deliberately not mirrored here: the "Design rules" bullets and
the "Open items" list live in docs/ROADMAP.md (build order and open-decisions table),
where they are actionable rather than archival.