[NOTICKET] feat(eval): report-readiness eval (dataset + runner + baseline)

Deterministic eval for is_report_ready (no LLM, no DB — fake stores). 16 golden
cases across floor / delta / edge / alignment; scores both the ready boolean and the
missing gaps, and counts the alignment gap (floor says ready but analyses unaligned to
the problem statement). Baseline run: floor 16/16 (100%), alignment gap = 2
(align_01, align_02) — the evidence for/against the deferred LLM-judge.

Run: uv run python -m eval.readiness.run_eval

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

eval/readiness/README.md

@@ -0,0 +1,34 @@
+# Report-readiness eval
+
+Scores the deterministic `is_report_ready` signal (`src/agents/report/readiness.py`)
+that the Help skill consumes to decide whether to nudge the user toward generating a
+report. No LLM, no DB — each golden case declares an analysis state + a set of
+persisted records/reports, and the runner feeds them through `is_report_ready` via
+injectable fake stores.
+
+## Run
+
+```bash
+uv run python -m eval.readiness.run_eval
+uv run python -m eval.readiness.run_eval --limit 5     # smoke test
+uv run python -m eval.readiness.run_eval --no-table    # summary only
+```
+
+Each run writes a timestamped `results/readiness_result_<ts>.json` (never
+overwritten, diffable across runs).
+
+## What it measures
+
+- **Floor correctness** — exact `ready` + `missing` for the deterministic floor
+  (validated goal · ≥1 substantive record · delta-since-report). Should sit at ~100%;
+  this is the regression guard as criteria evolve.
+- **Alignment gap** — `alignment` cases have substantive records (floor says
+  `ready=true`) but `aligned=false`: the analyses don't address the problem statement.
+  The floor can't see this. The gap count is the evidence for/against adding the
+  deferred LLM-judge — "ship the floor, earn the judge."
+
+## Dataset
+
+`readiness_dataset.json` — groups: `floor`, `delta`, `edge` (doc-only product
+question), `alignment`. See the `_about` / `_alignment` doc keys in the file. The
+`aligned` label is a semantic judgment; owner: Rifqi (report semantics) + Sofhia.

eval/readiness/readiness_dataset.json

@@ -0,0 +1,39 @@
+{
+  "_about": "Golden dataset for the report-readiness signal (`src/agents/report/readiness.is_report_ready`). Unlike the intent eval, readiness is DETERMINISTIC (no LLM): each case declares an analysis state + a set of persisted AnalysisRecords/reports, and the runner feeds them through is_report_ready via injectable fake stores, scoring the boolean `ready` AND the `missing` gaps. Floor cases should score ~100% (that's the regression value). The point of interest is the `alignment` group — see _alignment.",
+  "_floor": "is_report_ready's deterministic floor: (1) problem_validated, (2) >=1 SUBSTANTIVE record (findings non-empty; a failed/empty run still persists a record and must NOT count), (3) delta-since-report (if report_id set, ready only when a substantive record is newer than the latest report).",
+  "_alignment": "ALIGNMENT cases probe the deferred LLM-judge: records are substantive so the floor says ready=true, but `aligned=false` means the analyses don't actually address the problem statement — a human would say NOT ready. These are scored as floor-correct (expected_ready follows the floor) but counted separately as the 'alignment gap'. The gap count is the evidence that earns (or doesn't) the LLM-judge. Owner of the alignment label: Rifqi (report semantics) + Sofhia.",
+  "_fields": "records[].age_min / reports[].age_min = minutes ago (smaller = more recent); records[].findings = count of findings (0 = failed/empty run). missing codes map to is_report_ready's constants in the runner: problem|analysis|delta.",
+  "schema": {
+    "id": "stable per-case handle, <group>_<NN>",
+    "group": "floor | delta | edge | alignment",
+    "problem_validated": "bool",
+    "report_id": "null = never generated; a string = a report exists",
+    "records": "[{ findings: int, age_min: int }]",
+    "reports": "[{ age_min: int }] (only meaningful when report_id set)",
+    "aligned": "bool — do the analyses address the problem statement? (floor ignores this)",
+    "expected_ready": "what the deterministic floor SHOULD return",
+    "expected_missing": "subset of [problem, analysis, delta]",
+    "note": "human-readable description"
+  },
+  "cases": [
+    { "id": "floor_01", "group": "floor", "problem_validated": false, "report_id": null, "records": [], "reports": [], "aligned": false, "expected_ready": false, "expected_missing": ["problem", "analysis"], "note": "new analysis: no validated goal and no records yet" },
+    { "id": "floor_02", "group": "floor", "problem_validated": false, "report_id": null, "records": [{ "findings": 2, "age_min": 30 }], "reports": [], "aligned": true, "expected_ready": false, "expected_missing": ["problem"], "note": "has a substantive record but goal not validated (isolates the problem gap)" },
+    { "id": "floor_03", "group": "floor", "problem_validated": true, "report_id": null, "records": [], "reports": [], "aligned": false, "expected_ready": false, "expected_missing": ["analysis"], "note": "validated goal but no analysis run yet" },
+    { "id": "floor_04", "group": "floor", "problem_validated": true, "report_id": null, "records": [{ "findings": 0, "age_min": 20 }], "reports": [], "aligned": false, "expected_ready": false, "expected_missing": ["analysis"], "note": "only a failed/empty run (no findings) — must not count as substantive" },
+    { "id": "floor_05", "group": "floor", "problem_validated": true, "report_id": null, "records": [{ "findings": 0, "age_min": 40 }, { "findings": 0, "age_min": 10 }], "reports": [], "aligned": false, "expected_ready": false, "expected_missing": ["analysis"], "note": "multiple runs, all empty — still not ready" },
+    { "id": "floor_06", "group": "floor", "problem_validated": true, "report_id": null, "records": [{ "findings": 2, "age_min": 15 }], "reports": [], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "validated + one substantive record, no prior report → ready" },
+    { "id": "floor_07", "group": "floor", "problem_validated": true, "report_id": null, "records": [{ "findings": 3, "age_min": 40 }, { "findings": 1, "age_min": 10 }], "reports": [], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "multiple substantive records → ready" },
+    { "id": "floor_08", "group": "floor", "problem_validated": true, "report_id": null, "records": [{ "findings": 0, "age_min": 30 }, { "findings": 2, "age_min": 10 }], "reports": [], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "mix of failed + substantive → the substantive one is enough" },
+
+    { "id": "delta_01", "group": "delta", "problem_validated": true, "report_id": "rep-1", "records": [{ "findings": 2, "age_min": 120 }], "reports": [{ "age_min": 5 }], "aligned": true, "expected_ready": false, "expected_missing": ["delta"], "note": "report exists, all analysis is older than it → nothing new to report" },
+    { "id": "delta_02", "group": "delta", "problem_validated": true, "report_id": "rep-1", "records": [{ "findings": 2, "age_min": 5 }], "reports": [{ "age_min": 120 }], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "report exists but a newer substantive analysis arrived → ready to regenerate" },
+    { "id": "delta_03", "group": "delta", "problem_validated": true, "report_id": "rep-1", "records": [{ "findings": 1, "age_min": 90 }, { "findings": 2, "age_min": 10 }], "reports": [{ "age_min": 60 }], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "one old + one newer-than-report analysis → ready" },
+    { "id": "delta_04", "group": "delta", "problem_validated": true, "report_id": "rep-2", "records": [{ "findings": 2, "age_min": 90 }], "reports": [{ "age_min": 200 }, { "age_min": 30 }], "aligned": true, "expected_ready": false, "expected_missing": ["delta"], "note": "multiple reports — newest wins; analysis older than the newest report → not ready" },
+
+    { "id": "edge_01", "group": "edge", "problem_validated": true, "report_id": null, "records": [], "reports": [], "aligned": false, "expected_ready": false, "expected_missing": ["analysis"], "note": "doc-only analysis (RAG, no structured run) produces no AnalysisRecord → never report-able under the floor. PRODUCT QUESTION: should doc-only be report-able? if yes, readiness needs a 2nd source." },
+
+    { "id": "align_01", "group": "alignment", "problem_validated": true, "report_id": null, "records": [{ "findings": 2, "age_min": 15 }], "reports": [], "aligned": false, "expected_ready": true, "expected_missing": [], "note": "GAP: substantive records but they don't address the problem statement. Floor says ready; a human would say not-ready. The judge would catch this." },
+    { "id": "align_02", "group": "alignment", "problem_validated": true, "report_id": null, "records": [{ "findings": 3, "age_min": 25 }, { "findings": 1, "age_min": 5 }], "reports": [], "aligned": false, "expected_ready": true, "expected_missing": [], "note": "GAP: lots of analysis, none aligned to the goal — floor over-reports ready" },
+    { "id": "align_03", "group": "alignment", "problem_validated": true, "report_id": null, "records": [{ "findings": 2, "age_min": 15 }], "reports": [], "aligned": true, "expected_ready": true, "expected_missing": [], "note": "control: substantive AND aligned → genuinely ready, no gap" }
+  ]
+}

eval/readiness/results/readiness_result_2026-06-22_101645.json

@@ -0,0 +1,268 @@
+{
+  "run": {
+    "timestamp": "2026-06-22T10:16:45",
+    "dataset": "readiness_dataset.json",
+    "target": "src/agents/report/readiness.is_report_ready",
+    "total": 16,
+    "passed": 16,
+    "accuracy": 1.0,
+    "runtime_avg_ms": 0.0
+  },
+  "alignment_gap": {
+    "count": 2,
+    "ids": [
+      "align_01",
+      "align_02"
+    ]
+  },
+  "by_group": {
+    "floor": {
+      "n": 8,
+      "passed": 8,
+      "accuracy": 1.0
+    },
+    "delta": {
+      "n": 4,
+      "passed": 4,
+      "accuracy": 1.0
+    },
+    "edge": {
+      "n": 1,
+      "passed": 1,
+      "accuracy": 1.0
+    },
+    "alignment": {
+      "n": 3,
+      "passed": 3,
+      "accuracy": 1.0
+    }
+  },
+  "cases": [
+    {
+      "id": "floor_01",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a validated problem statement",
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "a validated problem statement",
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_02",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a validated problem statement"
+      ],
+      "got_missing": [
+        "a validated problem statement"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_03",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_04",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_05",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_06",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_07",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_08",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_01",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_02",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_03",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_04",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "edge_01",
+      "group": "edge",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_01",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_02",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_03",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    }
+  ]
+}

eval/readiness/run_eval.py

@@ -0,0 +1,282 @@
+"""Report-readiness eval runner.
+
+Feeds each golden case in `readiness_dataset.json` to the deterministic
+`is_report_ready` signal (`src/agents/report/readiness.py`) via injectable FAKE
+stores — no LLM, no DB — then scores both the boolean `ready` and the `missing`
+gaps. Prints a per-case detail table + aggregate summary and writes a timestamped
+JSON report under `results/` (never overwritten — one file per run, diffable).
+
+Two metrics matter:
+  - FLOOR correctness (ready + missing exact) — should be ~100%; this is the
+    regression guard as the criteria evolve.
+  - ALIGNMENT GAP — cases the floor calls ready=true but whose analyses are NOT
+    aligned to the problem statement (`aligned=false`). The floor can't see this;
+    the gap count is the evidence for/against adding the deferred LLM-judge.
+
+Invoke as a module so `src` imports resolve:
+
+    uv run python -m eval.readiness.run_eval
+    uv run python -m eval.readiness.run_eval --limit 5
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import statistics
+import time
+from dataclasses import asdict, dataclass, field
+from datetime import UTC, datetime, timedelta
+from pathlib import Path
+from typing import Any
+
+from src.agents.gate import stub_analysis_state
+from src.agents.report.readiness import (
+    _MISSING_ANALYSIS,
+    _MISSING_DELTA,
+    _MISSING_PROBLEM,
+    is_report_ready,
+)
+
+_HERE = Path(__file__).resolve().parent
+DATASET = _HERE / "readiness_dataset.json"
+RESULTS_DIR = _HERE / "results"
+GROUPS = ["floor", "delta", "edge", "alignment"]
+
+# Dataset short codes -> the exact `missing` strings is_report_ready emits. Imported
+# from the module so the dataset stays readable and survives wording changes.
+_CODE_TO_MISSING = {
+    "problem": _MISSING_PROBLEM,
+    "analysis": _MISSING_ANALYSIS,
+    "delta": _MISSING_DELTA,
+}
+
+
+@dataclass
+class _FakeRecord:
+    findings: list[Any]
+    created_at: datetime
+
+
+@dataclass
+class _FakeReport:
+    generated_at: datetime
+
+
+class _FakeStore:
+    """Stand-in for the Postgres record/report store — returns canned rows."""
+
+    def __init__(self, rows: list[Any]) -> None:
+        self._rows = rows
+
+    async def list_for_analysis(self, _analysis_id: str) -> list[Any]:
+        return self._rows
+
+
+@dataclass
+class CaseResult:
+    id: str
+    group: str
+    expected_ready: bool
+    got_ready: bool
+    expected_missing: list[str]
+    got_missing: list[str]
+    correct: bool
+    aligned: bool
+    gap: bool  # floor said ready but analyses not aligned to the problem statement
+    latency_ms: float
+
+
+def load_cases(path: Path) -> list[dict[str, Any]]:
+    data = json.loads(path.read_text(encoding="utf-8"))
+    return list(data["cases"])
+
+
+def _build_records(specs: list[dict[str, Any]], now: datetime) -> list[_FakeRecord]:
+    return [
+        _FakeRecord(
+            findings=["f"] * int(spec["findings"]),
+            created_at=now - timedelta(minutes=int(spec["age_min"])),
+        )
+        for spec in specs
+    ]
+
+
+def _build_reports(specs: list[dict[str, Any]], now: datetime) -> list[_FakeReport]:
+    return [
+        _FakeReport(generated_at=now - timedelta(minutes=int(spec["age_min"])))
+        for spec in specs
+    ]
+
+
+async def run_case(case: dict[str, Any]) -> CaseResult:
+    now = datetime.now(UTC)
+    state = stub_analysis_state(problem_validated=bool(case["problem_validated"]))
+    if case.get("report_id"):
+        state = state.model_copy(update={"report_id": case["report_id"]})
+
+    record_store = _FakeStore(_build_records(case.get("records", []), now))
+    report_store = _FakeStore(_build_reports(case.get("reports", []), now))
+    expected_missing = sorted(_CODE_TO_MISSING[c] for c in case["expected_missing"])
+
+    start = time.perf_counter()
+    rr = await is_report_ready(
+        case["id"], state, record_store=record_store, report_store=report_store
+    )
+    latency_ms = round((time.perf_counter() - start) * 1000, 1)
+
+    got_missing = sorted(rr.missing)
+    ready_ok = rr.ready == bool(case["expected_ready"])
+    missing_ok = got_missing == expected_missing
+    return CaseResult(
+        id=case["id"],
+        group=case["group"],
+        expected_ready=bool(case["expected_ready"]),
+        got_ready=rr.ready,
+        expected_missing=expected_missing,
+        got_missing=got_missing,
+        correct=ready_ok and missing_ok,
+        aligned=bool(case["aligned"]),
+        gap=rr.ready and not bool(case["aligned"]),
+        latency_ms=latency_ms,
+    )
+
+
+def _group_accuracy(results: list[CaseResult]) -> dict[str, dict[str, Any]]:
+    out: dict[str, dict[str, Any]] = {}
+    for g in GROUPS:
+        sub = [r for r in results if r.group == g]
+        if not sub:
+            continue
+        passed = sum(r.correct for r in sub)
+        out[g] = {"n": len(sub), "passed": passed, "accuracy": round(passed / len(sub), 3)}
+    return out
+
+
+def summarize(results: list[CaseResult]) -> dict[str, Any]:
+    n = len(results)
+    passed = sum(r.correct for r in results)
+    gaps = [r for r in results if r.gap]
+    latencies = [r.latency_ms for r in results]
+    return {
+        "total": n,
+        "passed": passed,
+        "accuracy": round(passed / n, 3) if n else 0.0,
+        "runtime_avg_ms": round(statistics.mean(latencies), 2) if latencies else 0,
+        "alignment_gap": {"count": len(gaps), "ids": [r.id for r in gaps]},
+        "by_group": _group_accuracy(results),
+    }
+
+
+def _fmt_bool(value: bool) -> str:
+    return "T" if value else "F"
+
+
+def _truncate(text: str, width: int) -> str:
+    return text if len(text) <= width else text[: width - 3] + "..."
+
+
+def format_table(results: list[CaseResult]) -> str:
+    header = (
+        f"{'ID':<12} {'GROUP':<10} {'RDY e/g':<8} "
+        f"{'MISSING (got)':<40} {'OK':<3} {'GAP':<4}"
+    )
+    rule = "-" * len(header)
+    lines = [rule, header, rule]
+    for r in results:
+        rdy = f"{_fmt_bool(r.expected_ready)}/{_fmt_bool(r.got_ready)}"
+        missing = ", ".join(r.got_missing) or "-"
+        ok = "ok" if r.correct else "X"
+        gap = "GAP" if r.gap else ""
+        lines.append(
+            f"{r.id:<12} {r.group:<10} {rdy:<8} "
+            f"{_truncate(missing, 40):<40} {ok:<3} {gap:<4}"
+        )
+    lines.append(rule)
+    return "\n".join(lines)
+
+
+def format_summary(summary: dict[str, Any], results: list[CaseResult]) -> str:
+    lines = ["SUMMARY"]
+    lines.append(
+        f"  Floor      {summary['passed']}/{summary['total']} correct"
+        f"   ({summary['accuracy'] * 100:.1f}%)   avg {summary['runtime_avg_ms']} ms"
+    )
+    gap = summary["alignment_gap"]
+    lines.append(
+        f"  Align gap  {gap['count']} case(s) ready-but-misaligned"
+        + (f"  -> {', '.join(gap['ids'])}" if gap["ids"] else "")
+    )
+    lines.append("    (floor can't catch these; this count is the LLM-judge justification)")
+    lines.append("")
+    lines.append("  By group")
+    for g, m in summary["by_group"].items():
+        lines.append(f"    {g:<12} {m['passed']}/{m['n']}  {m['accuracy'] * 100:.0f}%")
+    failures = [r for r in results if not r.correct]
+    lines.append("")
+    lines.append(f"  FAILURES ({len(failures)})")
+    for r in failures:
+        lines.append(
+            f"    {r.id:<12} ready {_fmt_bool(r.expected_ready)}->{_fmt_bool(r.got_ready)}"
+            f"  missing {r.expected_missing} -> {r.got_missing}"
+        )
+    return "\n".join(lines)
+
+
+def build_report(
+    results: list[CaseResult], summary: dict[str, Any], meta: dict[str, Any]
+) -> dict[str, Any]:
+    run = {**meta, **{k: summary[k] for k in ("total", "passed", "accuracy", "runtime_avg_ms")}}
+    return {
+        "run": run,
+        "alignment_gap": summary["alignment_gap"],
+        "by_group": summary["by_group"],
+        "cases": [asdict(r) for r in results],
+    }
+
+
+@dataclass
+class _Args:
+    dataset: Path = DATASET
+    limit: int = 0
+    no_table: bool = False
+    extra: dict[str, Any] = field(default_factory=dict)
+
+
+async def main() -> None:
+    parser = argparse.ArgumentParser(description="Report-readiness eval")
+    parser.add_argument("--dataset", type=Path, default=DATASET)
+    parser.add_argument("--limit", type=int, default=0, help="run first N cases only")
+    parser.add_argument("--no-table", action="store_true", help="skip the detail table")
+    args = parser.parse_args()
+
+    cases = load_cases(args.dataset)
+    if args.limit:
+        cases = cases[: args.limit]
+
+    started = datetime.now()
+    print(f"Report-Readiness Eval -- {started:%Y-%m-%d %H:%M:%S}")
+    print(f"dataset: {args.dataset.name} ({len(cases)} cases)  target: is_report_ready")
+
+    results = [await run_case(case) for case in cases]
+
+    summary = summarize(results)
+    if not args.no_table:
+        print(format_table(results))
+    print(format_summary(summary, results))
+
+    meta = {
+        "timestamp": started.isoformat(timespec="seconds"),
+        "dataset": args.dataset.name,
+        "target": "src/agents/report/readiness.is_report_ready",
+    }
+    report = build_report(results, summary, meta)
+    RESULTS_DIR.mkdir(parents=True, exist_ok=True)
+    out_path = RESULTS_DIR / f"readiness_result_{started:%Y-%m-%d_%H%M%S}.json"
+    out_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
+    print(f"\n-> saved: {out_path.relative_to(_HERE.parent.parent)}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())