# Report-readiness eval

Scores the deterministic `is_report_ready` signal (`src/agents/report/readiness.py`)
that the Help skill consumes to decide whether to nudge the user toward generating a
report. No LLM, no DB — each golden case declares an analysis state + a set of
persisted records/reports, and the runner feeds them through `is_report_ready` via
injectable fake stores.

## Run

```bash
uv run python -m eval.readiness.run_eval
uv run python -m eval.readiness.run_eval --limit 5     # smoke test
uv run python -m eval.readiness.run_eval --no-table    # summary only
```

Each run writes a timestamped `results/readiness_result_<ts>.json` (never
overwritten, diffable across runs).

## What it measures

- **Floor correctness** — exact `ready` + `missing` for the deterministic floor
  (validated goal · ≥1 substantive record · delta-since-report). Should sit at ~100%;
  this is the regression guard as criteria evolve.
- **Alignment gap** — `alignment` cases have substantive records (floor says
  `ready=true`) but `aligned=false`: the analyses don't address the problem statement.
  The floor can't see this. The gap count is the evidence for/against adding the
  deferred LLM-judge — "ship the floor, earn the judge."

## Dataset

`readiness_dataset.json` — groups: `floor`, `delta`, `edge` (doc-only product
question), `alignment`. See the `_about` / `_alignment` doc keys in the file. The
`aligned` label is a semantic judgment; owner: Rifqi (report semantics) + Sofhia.