Sync from GitHub via hub-sync

docs/DP.md

@@ -1,6 +1,6 @@
 # Design Proposal (DP) — forecaster-agent
 
-**Version:** 0.6 (Integrity & Learning Loop)  
+**Version:** 0.8 (Live Track Record Credibility)  
 **Last updated:** 2026-06-24
 
 Architectural design implementing [PRD.md](./PRD.md) under Harness Engineering standards.
@@ -224,7 +224,7 @@ CI: `.github/workflows/ci.yml` — Python 3.11 + 3.12 matrix.
 
 | Item | Priority | Status |
 |------|----------|--------|
-| `dashboard.py` monolith (~1150 LOC) | P1 | open |
+| `dashboard.py` monolith (~1150 LOC) | P1 | ✅ 104-LOC orchestrator + `ui/tabs/*` |
 | Discord/Telegram crowd bots | P2 | ✅ `bots/` |
 | No `MockLLMClient` for offline `run.py once` | P2 | open |
 | `src/` package layout + Poetry lock | P3 | open |
@@ -309,6 +309,56 @@ This closes the perception gap: users know their survey answers matter.
 
 ---
 
+## 12. v0.8 Design additions (Live Track Record Credibility)
+
+### 12.1 Origin classification (HR-11)
+
+Location: `services/track_record.py`
+
+| Function | Purpose |
+|----------|---------|
+| `seed_prediction_ids(seed_path)` | Load fingerprint set from `predictions_seed.json` |
+| `prediction_origin(p, seed_ids)` | Returns `"seed"` if `p.id` ∈ seed_ids else `"live"` |
+| `partition_by_origin(preds, seed_ids)` | Split into `(seed_preds, live_preds)` |
+| `scoreboard_subset(preds)` | Same shape as `Registry.scoreboard()` for a filtered list |
+| `upcoming_resolutions(preds, *, limit=10)` | Open/due preds sorted by `resolution_date` asc |
+
+**Rule:** seed IDs are computed once per dashboard render from the committed seed file;
+live predictions are everything else in the registry (including cron-generated rows).
+
+### 12.2 Live-only export
+
+`run.py export` filters with `partition_by_origin` before writing JSONL.
+Seed data never enters `predictions_live.jsonl` — it remains in `predictions_seed.json`.
+
+### 12.3 Export verification (CI guard)
+
+`run.py verify-export`:
+
+1. Load live predictions from DB (`partition_by_origin`)
+2. Load `predictions_live.jsonl`
+3. For each live id, assert matching `status`, `outcome`, `brier` in JSONL
+4. Exit code 1 on any mismatch (prevents silent resolve/export regression)
+
+Called in `.github/workflows/daily-pages.yml` immediately after `export`.
+
+### 12.4 Track Record UI layout
+
+```
+┌─ Curated benchmark (seed) ─────────────────┐
+│  resolved N | mean Brier X | table         │
+└────────────────────────────────────────────┘
+┌─ Live LLM predictions ─────────────────────┐
+│  open N | resolved N | mean Brier Y        │
+│  ▶ Upcoming resolutions (live only)        │
+│  resolved table (live only)                │
+└────────────────────────────────────────────┘
+```
+
+CSV download includes `origin` column on every row.
+
+---
+
 ## 11. Security & compliance defaults
 
 - `require_review: true` — never change default in repo

docs/PRD.md

@@ -1,6 +1,6 @@
 # Product Requirement Document (PRD) — forecaster-agent
 
-**Version:** 0.6 (Integrity & Learning Loop)  
+**Version:** 0.8 (Live Track Record Credibility)  
 **Last updated:** 2026-06-24
 
 An autonomous AI × economy forecasting system that generates, calibrates, and publishes **falsifiable** predictions — with explicit limits on historical extrapolation.
@@ -122,6 +122,7 @@ Parallel surface: `dashboard.py` (Streamlit) + `services/read_model.py` (read AP
 | **HR-8** | BUSL-1.1 license | Commercial SaaS/API requires separate license |
 | **HR-9** | Citation integrity | LLM-generated `sources` URLs pass schema check; arXiv IDs must not reference a future YYMM; `example.com` / placeholder domains rejected at parse time. Enforced in `forecast._sanitize_sources()`. |
 | **HR-10** | Resolved-state durability | `run.py export` serialises `status`, `outcome`, `brier`, `resolved_at`; `ensure_demo_registry` reloads them via `model_validate_json` preserving resolution. Track record must never regress to all-`open` after a cache eviction. |
+| **HR-11** | Origin transparency | Every prediction shown in the Track Record tab carries an explicit `origin` badge (`seed` = curated benchmark, `live` = daily LLM cron). `run.py export` writes **live-only** rows to `predictions_live.jsonl`; `run.py verify-export` fails CI if DB live state diverges from the committed file after a resolve. |
 
 ---
 
@@ -196,6 +197,22 @@ LLM accumulation run (12 predictions via Groq):
 - [x] **ROOT_CAUSE contrastive pairs** in `predictions_seed.json` + `track_record_summary` WRONG entries show 280-char rationale window
 - [x] **PR-aligned commit-back**: `predictions_live.jsonl` now correctly detects new/changed file via `git add` before `git diff --cached`
 
+### Phase 7 — Live Track Record Credibility (v0.8, 2026-06-24)
+
+Problem: seed demo data (12 resolved) and daily LLM predictions (12 open) were merged
+in the UI with no origin label — users could not tell curated benchmark from real agent
+performance.
+
+- [x] **HR-11 Origin split**: Track Record tab shows separate panels for *Curated benchmark*
+  (seed) and *Live LLM* with independent resolved counts and mean Brier
+- [x] **Upcoming resolutions**: timeline of open live predictions sorted by
+  `resolution_date`, showing criteria so users see the loop is active
+- [x] **CSV `origin` column**: public download includes `seed|live` for replication
+- [x] **Live-only export**: `run.py export` writes only non-seed predictions to
+  `predictions_live.jsonl` (seed stays in `predictions_seed.json`)
+- [x] **CI guard**: `run.py verify-export` after daily cron; fails if live DB state
+  ≠ committed JSONL (catches silent resolve/export regressions)
+
 ---
 
 ## 6. Success metrics
@@ -207,13 +224,15 @@ LLM accumulation run (12 predictions via Groq):
 | OOD fires → confidence downshift | Qualitative audit |
 | P0 bugs open | 0 |
 | MCP read tools documented | Phase 3 exit |
-|| LLM-generated sources with hallucinated arXiv IDs | 0 (HR-9) |
-|| Resolved predictions surviving cache eviction | 100% (HR-10) |
-|| `predictions_live.jsonl` committed per real-LLM cron run | ✅ (daily) |
+| LLM-generated sources with hallucinated arXiv IDs | 0 (HR-9) |
+| Resolved predictions surviving cache eviction | 100% (HR-10) |
+| `predictions_live.jsonl` committed per real-LLM cron run | ✅ (daily) |
+| Track Record UI shows seed vs live origin | HR-11 (Phase 7) |
+| Live resolved count visible independently of seed | Phase 7 exit |
 
 ---
 
-## 7. Out of scope (v0.6)
+## 7. Out of scope (v0.8)
 
 - Auto-publishing with `require_review: false` as default
 - Financial advice positioning

run.py

@@ -10,8 +10,9 @@
   python run.py once --config config.ci.yaml   # CI / auto-publish profile
   python run.py calibrate-jobs       # BLS → KB displacement risk overlay
   python run.py calibrate-jobs --dry-run
-  python run.py export               # dump registry → data/predictions_live.jsonl
+  python run.py export               # dump live registry rows → predictions_live.jsonl
   python run.py export --out path.jsonl
+  python run.py verify-export        # assert DB live state == committed JSONL (HR-11)
   python run.py warmup               # import predictions_live.jsonl → DB (cache-miss recovery)
   python run.py warmup --src path.jsonl
 """
@@ -104,17 +105,43 @@ def cmd_warmup(cfg, src_path: str = "data/predictions_live.jsonl"):
     print(f"warmup: imported {len(added)} new prediction(s) from {src_path}")
 
 
-def cmd_export(cfg, out_path: str = "data/predictions_live.jsonl"):
-    """Dump the registry to a committable JSONL so accumulated real predictions
-    survive cache eviction and become visible to the HF Space demo."""
+def cmd_export(cfg, out_path: str = "data/predictions_live.jsonl", seed_path: str | None = None):
+    """Dump live-only predictions to JSONL (HR-11: seed stays in predictions_seed.json)."""
+    from services.track_record import partition_by_origin, seed_prediction_ids
+
     reg = Registry(cfg["database_path"])
-    preds = sorted(reg.load(), key=lambda p: (p.created_at, p.id))
+    seed_ids = seed_prediction_ids(seed_path)
+    _, live = partition_by_origin(reg.load(), seed_ids)
+    live.sort(key=lambda p: (p.created_at, p.id))
     path = Path(out_path)
     path.parent.mkdir(parents=True, exist_ok=True)
     with path.open("w", encoding="utf-8") as f:
-        for p in preds:
+        for p in live:
             f.write(p.model_dump_json() + "\n")
-    print(f"exported {len(preds)} prediction(s) to {out_path}")
+    print(f"exported {len(live)} live prediction(s) to {out_path}")
+
+
+def cmd_verify_export(
+    cfg,
+    out_path: str = "data/predictions_live.jsonl",
+    seed_path: str | None = None,
+):
+    """Fail if committed JSONL does not mirror live predictions in the DB (HR-11)."""
+    from services.dashboard_seed import _load_live_rows
+    from services.track_record import partition_by_origin, seed_prediction_ids, verify_live_export_sync
+
+    reg = Registry(cfg["database_path"])
+    seed_ids = seed_prediction_ids(seed_path)
+    _, db_live = partition_by_origin(reg.load(), seed_ids)
+    jsonl_live = _load_live_rows(Path(out_path))
+
+    errors = verify_live_export_sync(db_live, jsonl_live)
+    if errors:
+        print("verify-export FAILED:", file=sys.stderr)
+        for err in errors:
+            print(f"  - {err}", file=sys.stderr)
+        sys.exit(1)
+    print(f"verify-export OK ({len(db_live)} live prediction(s) in sync)")
 
 
 def cmd_approve(cfg):
@@ -189,6 +216,8 @@ def main():
         cmd_calibrate_jobs(cfg, dry_run=dry_run_flag)
     elif cmd == "export":
         cmd_export(cfg, out_path or "data/predictions_live.jsonl")
+    elif cmd == "verify-export":
+        cmd_verify_export(cfg, out_path or "data/predictions_live.jsonl")
     elif cmd == "warmup":
         src = _extract_opt(args, "--src") or out_path or "data/predictions_live.jsonl"
         cmd_warmup(cfg, src)

services/dashboard_data.py

@@ -10,6 +10,12 @@ from registry import Registry
 from schemas import Prediction
 from services.config_loader import load_config
 from services.read_model import get_ood_assessment, get_scoreboard, search_jobs
+from services.track_record import (
+    partition_by_origin,
+    scoreboard_subset,
+    seed_prediction_ids,
+    upcoming_resolutions,
+)
 
 
 def get_scoreboard_data() -> dict[str, Any]:
@@ -20,6 +26,21 @@ def get_predictions() -> list[Prediction]:
     return Registry().load()
 
 
+def get_track_record_views() -> dict[str, Any]:
+    """Predictions partitioned by origin with per-origin scoreboards."""
+    preds = get_predictions()
+    seed_ids = seed_prediction_ids()
+    seed_preds, live_preds = partition_by_origin(preds, seed_ids)
+    return {
+        "seed_ids": seed_ids,
+        "seed_preds": seed_preds,
+        "live_preds": live_preds,
+        "seed_scoreboard": scoreboard_subset(seed_preds),
+        "live_scoreboard": scoreboard_subset(live_preds),
+        "upcoming_live": upcoming_resolutions(live_preds),
+    }
+
+
 def build_evolution_prior(scenario: dict[str, Any], *, n_bootstrap: int | None = None) -> ev.EvolutionPrior:
     cfg = load_config()
     boot = n_bootstrap if n_bootstrap is not None else int(cfg.get("evolution", {}).get("n_bootstrap", 50))

services/track_record.py

@@ -0,0 +1,149 @@
+"""Track-record helpers: seed vs live origin, scoreboards, export verification (HR-11)."""
+from __future__ import annotations
+
+from datetime import date, datetime, timezone
+from pathlib import Path
+from typing import Literal
+
+from paths import PROJECT_ROOT
+from schemas import Prediction, Status
+
+Origin = Literal["seed", "live"]
+
+_RESOLVED = (Status.resolved_true, Status.resolved_false)
+_OPEN = (Status.open, Status.due)
+
+
+def seed_prediction_ids(seed_path: str | Path | None = None) -> set[str]:
+    """Fingerprint IDs of curated benchmark rows in predictions_seed.json."""
+    from services.dashboard_seed import _load_seed_rows
+
+    path = Path(seed_path) if seed_path else PROJECT_ROOT / "data" / "predictions_seed.json"
+    return {p.id for p in _load_seed_rows(path)}
+
+
+def prediction_origin(p: Prediction, seed_ids: set[str]) -> Origin:
+    return "seed" if p.id in seed_ids else "live"
+
+
+def partition_by_origin(
+    preds: list[Prediction],
+    seed_ids: set[str] | None = None,
+) -> tuple[list[Prediction], list[Prediction]]:
+    """Split predictions into (seed, live) lists."""
+    if seed_ids is None:
+        seed_ids = seed_prediction_ids()
+    seed: list[Prediction] = []
+    live: list[Prediction] = []
+    for p in preds:
+        if prediction_origin(p, seed_ids) == "seed":
+            seed.append(p)
+        else:
+            live.append(p)
+    return seed, live
+
+
+def scoreboard_subset(preds: list[Prediction]) -> dict:
+    """Same shape as ``Registry.scoreboard()`` for an arbitrary prediction list."""
+    scored = [p for p in preds if p.brier is not None]
+    n = len(scored)
+    mean_brier = sum(p.brier for p in scored) / n if n else None
+
+    buckets: dict[int, list[Prediction]] = {}
+    for p in scored:
+        b = min(9, int(p.confidence * 10))
+        buckets.setdefault(b, []).append(p)
+
+    calibration = []
+    for b in sorted(buckets):
+        grp = buckets[b]
+        avg_conf = sum(x.confidence for x in grp) / len(grp)
+        hit_rate = sum(1 for x in grp if x.outcome) / len(grp)
+        calibration.append({
+            "bucket": f"{b*10}-{b*10+10}%",
+            "n": len(grp),
+            "avg_confidence": round(avg_conf, 3),
+            "actual_hit_rate": round(hit_rate, 3),
+        })
+
+    return {
+        "total": len(preds),
+        "open": sum(1 for p in preds if p.status in _OPEN),
+        "resolved": n,
+        "ambiguous": sum(1 for p in preds if p.status == Status.ambiguous),
+        "mean_brier": round(mean_brier, 4) if mean_brier is not None else None,
+        "calibration": calibration,
+    }
+
+
+def upcoming_resolutions(
+    preds: list[Prediction],
+    *,
+    today: date | None = None,
+    limit: int = 10,
+) -> list[Prediction]:
+    """Open/due predictions sorted by resolution_date (soonest first)."""
+    today = today or date.today()
+    open_preds = [p for p in preds if p.status in _OPEN]
+    open_preds.sort(key=lambda p: (p.resolution_date, p.created_at))
+    return open_preds[:limit]
+
+
+def days_until_resolution(p: Prediction, today: date | None = None) -> int:
+    today = today or date.today()
+    return (p.resolution_date - today).days
+
+
+def verify_live_export_sync(
+    db_live: list[Prediction],
+    jsonl_live: list[Prediction],
+) -> list[str]:
+    """Return human-readable error strings; empty list means OK."""
+    by_id = {p.id: p for p in jsonl_live}
+    db_ids = {p.id for p in db_live}
+    errors: list[str] = []
+
+    for p in db_live:
+        j = by_id.get(p.id)
+        if j is None:
+            errors.append(f"missing in JSONL: [{p.id[:8]}] {p.statement[:50]}")
+            continue
+        if p.status != j.status:
+            errors.append(
+                f"status mismatch [{p.id[:8]}]: db={p.status.value} jsonl={j.status.value}"
+            )
+        if p.outcome != j.outcome:
+            errors.append(
+                f"outcome mismatch [{p.id[:8]}]: db={p.outcome} jsonl={j.outcome}"
+            )
+        if p.brier is not None and j.brier is not None:
+            if abs(p.brier - j.brier) > 1e-6:
+                errors.append(
+                    f"brier mismatch [{p.id[:8]}]: db={p.brier} jsonl={j.brier}"
+                )
+
+    for jid, j in by_id.items():
+        if jid not in db_ids:
+            errors.append(f"orphan in JSONL (not in DB live set): [{jid[:8]}]")
+
+    return errors
+
+
+def prediction_to_csv_row(p: Prediction, origin: Origin) -> dict:
+    return {
+        "origin": origin,
+        "id": p.fingerprint(),
+        "statement": p.statement,
+        "category": p.category,
+        "confidence": p.confidence,
+        "horizon": p.horizon,
+        "status": p.status.value,
+        "resolution_date": p.resolution_date.isoformat() if p.resolution_date else "",
+        "outcome": p.outcome,
+        "brier": p.brier,
+        "judged_rationale": p.judged_rationale,
+        "sources": " | ".join(p.sources or []),
+        "resolved_at": (
+            p.resolved_at.isoformat() if p.resolved_at else ""
+        ),
+    }

tests/test_track_record.py

@@ -0,0 +1,170 @@
+"""Tests for services/track_record.py (HR-11)."""
+from __future__ import annotations
+
+import json
+import sys
+from datetime import date
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from sqlmodel import Session, delete
+
+from registry import Registry
+from schemas import Prediction, Status, engine
+from services.track_record import (
+    partition_by_origin,
+    prediction_origin,
+    scoreboard_subset,
+    seed_prediction_ids,
+    upcoming_resolutions,
+    verify_live_export_sync,
+)
+
+
+def _clean():
+    with Session(engine) as session:
+        session.exec(delete(Prediction))
+        session.commit()
+
+
+def test_seed_prediction_ids_loads_from_file():
+    ids = seed_prediction_ids()
+    assert len(ids) >= 8
+
+
+def test_partition_by_origin(tmp_path):
+    seed = tmp_path / "seed.json"
+    seed.write_text(json.dumps([{
+        "statement": "Seed only claim",
+        "rationale": "r", "category": "macro", "confidence": 0.6,
+        "horizon": "2026-Q4", "resolution_date": "2026-12-31",
+        "resolution_criteria": "c", "status": "resolved_true",
+        "outcome": True, "judged_rationale": "ok",
+    }]), encoding="utf-8")
+    seed_ids = seed_prediction_ids(seed)
+    assert len(seed_ids) == 1
+
+    seed_p = Prediction.model_validate(json.loads(seed.read_text())[0]).assign_id()
+    live_p = Prediction(
+        statement="Live cron claim",
+        rationale="r", confidence=0.5, horizon="2027-Q1",
+        resolution_date=date(2027, 3, 31), resolution_criteria="c",
+    ).assign_id()
+
+    s, l = partition_by_origin([seed_p, live_p], seed_ids)
+    assert len(s) == 1 and len(l) == 1
+    assert prediction_origin(seed_p, seed_ids) == "seed"
+    assert prediction_origin(live_p, seed_ids) == "live"
+
+
+def test_scoreboard_subset_mean_brier():
+    p_hit = Prediction(
+        statement="Hit", rationale="r", confidence=0.8, horizon="2026-Q1",
+        resolution_date=date(2026, 3, 31), resolution_criteria="c",
+    )
+    p_hit.resolve(True, "yes")
+    p_miss = Prediction(
+        statement="Miss", rationale="r", confidence=0.8, horizon="2026-Q1",
+        resolution_date=date(2026, 3, 31), resolution_criteria="c",
+    )
+    p_miss.resolve(False, "no")
+    sb = scoreboard_subset([p_hit, p_miss])
+    assert sb["resolved"] == 2
+    assert sb["mean_brier"] == pytest.approx(0.34)
+
+
+def test_upcoming_resolutions_sorted():
+    p1 = Prediction(
+        statement="Soon", rationale="r", confidence=0.5, horizon="2026-Q2",
+        resolution_date=date(2026, 6, 30), resolution_criteria="c",
+    ).assign_id()
+    p2 = Prediction(
+        statement="Later", rationale="r", confidence=0.5, horizon="2027-Q1",
+        resolution_date=date(2027, 3, 31), resolution_criteria="c",
+    ).assign_id()
+    out = upcoming_resolutions([p2, p1], today=date(2026, 1, 1))
+    assert out[0].statement == "Soon"
+
+
+def test_verify_live_export_sync_ok():
+    p = Prediction(
+        statement="Sync test", rationale="r", confidence=0.5, horizon="2026-Q4",
+        resolution_date=date(2026, 12, 31), resolution_criteria="c",
+    ).assign_id()
+    assert verify_live_export_sync([p], [p]) == []
+
+
+def test_verify_live_export_detects_status_mismatch():
+    p_db = Prediction(
+        statement="Resolved live", rationale="r", confidence=0.7, horizon="2026-Q1",
+        resolution_date=date(2026, 3, 31), resolution_criteria="c",
+    ).assign_id()
+    p_db.resolve(True, "facts")
+    p_jsonl = Prediction(
+        statement="Resolved live", rationale="r", confidence=0.7, horizon="2026-Q1",
+        resolution_date=date(2026, 3, 31), resolution_criteria="c",
+        status=Status.open,
+    ).assign_id()
+    p_jsonl.id = p_db.id
+    errs = verify_live_export_sync([p_db], [p_jsonl])
+    assert any("status mismatch" in e for e in errs)
+
+
+def test_export_live_only_excludes_seed(tmp_path):
+    import run
+
+    _clean()
+    seed = tmp_path / "seed.json"
+    seed.write_text(json.dumps([{
+        "statement": "Curated seed row",
+        "rationale": "r", "category": "macro", "confidence": 0.6,
+        "horizon": "2026-Q4", "resolution_date": "2026-12-31",
+        "resolution_criteria": "c", "status": "open",
+    }]), encoding="utf-8")
+    seed_p = Prediction.model_validate(json.loads(seed.read_text())[0]).assign_id()
+    live_p = Prediction(
+        statement="Daily LLM row",
+        rationale="r", confidence=0.5, horizon="2027-Q2",
+        resolution_date=date(2027, 6, 30), resolution_criteria="c",
+    ).assign_id()
+    Registry().add_many([seed_p, live_p])
+
+    out = tmp_path / "live.jsonl"
+    run.cmd_export(
+        {"database_path": "data/forecaster.db"},
+        str(out),
+        seed_path=str(seed),
+    )
+    lines = [ln for ln in out.read_text(encoding="utf-8").splitlines() if ln.strip()]
+    assert len(lines) == 1
+    assert "Daily LLM row" in lines[0]
+    assert "Curated seed row" not in lines[0]
+    _clean()
+
+
+def test_verify_export_cli(tmp_path):
+    import run
+
+    _clean()
+    seed = tmp_path / "seed.json"
+    seed.write_text("[]", encoding="utf-8")
+    p = Prediction(
+        statement="CLI verify", rationale="r", confidence=0.5, horizon="2026-Q4",
+        resolution_date=date(2026, 12, 31), resolution_criteria="c",
+    ).assign_id()
+    Registry().add_many([p])
+    out = tmp_path / "live.jsonl"
+    run.cmd_export(
+        {"database_path": "data/forecaster.db"},
+        str(out),
+        seed_path=str(seed),
+    )
+    run.cmd_verify_export(
+        {"database_path": "data/forecaster.db"},
+        str(out),
+        seed_path=str(seed),
+    )
+    _clean()

ui/i18n.py

@@ -220,9 +220,33 @@ Compare your AI scenario with **15+ historical tech transitions**.
     # ── accuracy tab ─────────────────────────────────────────────────────────
     "acc_title": {"en": "Forecast accuracy tracker", "zh": "预测准确度追踪"},
     "acc_intro": {
-        "en": "Tracks AI forecasts vs real outcomes. Lower **Brier score** = better calibration.",
-        "zh": "追踪 AI 预测与现实结果。**Brier 分数**越低表示校准越好。",
-    },
+        "en": "Tracks AI forecasts vs real outcomes. **Curated benchmark** rows are verifiable historical demos; **Live LLM** rows come from the daily forecasting cron.",
+        "zh": "追踪 AI 预测与现实结果。**精选基准** 为可核查的历史演示；**Live LLM** 来自每日预测 cron。",
+    },
+    "acc_origin_note": {
+        "en": "Two track records are shown separately so you can judge real agent performance without confusing it with curated demos.",
+        "zh": "两类战绩分开展示，避免将真实 LLM 表现与精选演示数据混淆。",
+    },
+    "acc_seed_panel": {"en": "📚 Curated benchmark (seed)", "zh": "📚 精选基准（seed）"},
+    "acc_live_panel": {"en": "🤖 Live LLM predictions", "zh": "🤖 Live LLM 预测"},
+    "acc_seed_explain": {
+        "en": "Hand-picked, source-backed predictions with known outcomes — used to demonstrate calibration honesty (includes deliberate misses).",
+        "zh": "人工精选、有来源背书、结果已知的预测——用于展示诚实的校准能力（含故意 MISS）。",
+    },
+    "acc_live_explain": {
+        "en": "Generated daily by Groq LLM; outcomes resolve automatically when `resolution_date` passes. This is the agent's real public scoreboard.",
+        "zh": "由 Groq LLM 每日生成；到达 `resolution_date` 后自动判定。这是代理的真实公开战绩。",
+    },
+    "acc_upcoming_title": {"en": "Upcoming live resolutions", "zh": "即将判定的 Live 预测"},
+    "acc_upcoming_intro": {
+        "en": "These live predictions will be judged automatically on their resolution date — watch this list shrink as the loop closes.",
+        "zh": "以下 Live 预测将在解析日自动判定——随着闭环运转，此列表会逐渐缩短。",
+    },
+    "acc_no_upcoming": {"en": "No open live predictions scheduled.", "zh": "暂无待判定的 Live 预测。"},
+    "col_origin": {"en": "Origin", "zh": "来源"},
+    "col_days_left": {"en": "Days left", "zh": "剩余天数"},
+    "origin_seed": {"en": "Benchmark", "zh": "基准"},
+    "origin_live": {"en": "Live LLM", "zh": "Live LLM"},
     "acc_total": {"en": "Total predictions", "zh": "预测总数"},
     "acc_open": {"en": "Active / open", "zh": "进行中"},
     "acc_resolved": {"en": "Resolved", "zh": "已解析"},

ui/tabs/accuracy.py

@@ -1,76 +1,158 @@
-"""Tab: accuracy."""
+"""Tab: accuracy — track record with seed vs live origin split (HR-11)."""
 from __future__ import annotations
 
-import pandas as pd
-import plotly.graph_objects as go
+import io
 from datetime import datetime, timezone
 
+import pandas as pd
+import plotly.graph_objects as go
 import streamlit as st
-from services.dashboard_data import get_predictions, get_scoreboard_data
+
 import market
-from schemas import Status
+from schemas import Prediction, Status
+from services.dashboard_data import get_predictions, get_track_record_views
+from services.track_record import days_until_resolution, prediction_origin, prediction_to_csv_row
 from ui.i18n import filter_all_label, prediction_category_label, t
 
 
+def _brier_str(val) -> str:
+    return f"{val:.4f}" if val is not None else "N/A"
+
+
+def _scoreboard_cards(sb: dict, *, prefix: str) -> None:
+    c1, c2, c3, c4 = st.columns(4)
+    with c1:
+        st.metric(t("acc_total"), sb["total"], key=f"{prefix}_total")
+    with c2:
+        st.metric(t("acc_open"), sb["open"], key=f"{prefix}_open")
+    with c3:
+        st.metric(t("acc_resolved"), sb["resolved"], key=f"{prefix}_resolved")
+    with c4:
+        st.metric(t("acc_brier"), _brier_str(sb["mean_brier"]), key=f"{prefix}_brier")
+
+
+def _resolved_table(preds: list[Prediction], origin_label: str) -> None:
+    if not preds:
+        st.write(t("acc_no_resolved"))
+        return
+    col_src_key = t("col_sources")
+    rows = []
+    for p in preds:
+        first_src = (p.sources or [])[0] if p.sources else ""
+        rows.append({
+            t("col_origin"): origin_label,
+            t("col_id"): p.fingerprint(),
+            t("col_statement"): p.statement,
+            t("col_outcome"): t("outcome_true") if p.outcome else t("outcome_false"),
+            t("col_confidence"): f"{p.confidence*100:.0f}%",
+            t("col_brier"): f"{p.brier:.4f}" if p.brier is not None else "",
+            t("col_rationale"): p.judged_rationale,
+            col_src_key: first_src,
+        })
+    st.dataframe(
+        pd.DataFrame(rows),
+        column_config={
+            col_src_key: st.column_config.LinkColumn(
+                t("col_sources"),
+                display_text=t("col_sources_label"),
+            ),
+        },
+        use_container_width=True,
+        hide_index=True,
+    )
+
+
+def _active_table(preds: list[Prediction], origin_label: str) -> None:
+    if not preds:
+        st.write(t("acc_no_active"))
+        return
+    st.dataframe(
+        pd.DataFrame([{
+            t("col_origin"): origin_label,
+            t("col_id"): p.fingerprint(),
+            t("col_statement"): p.statement,
+            t("col_category"): prediction_category_label(p.category),
+            t("col_confidence"): f"{p.confidence*100:.0f}%",
+            t("col_horizon"): p.horizon,
+            t("col_resolution"): p.resolution_date.strftime("%Y-%m-%d"),
+        } for p in preds]),
+        use_container_width=True,
+        hide_index=True,
+    )
+
+
 def render(scenario_input: dict, prior, job_radar_cfg: dict):
     st.subheader(t("acc_title"))
     st.markdown(t("acc_intro"))
+    st.caption(t("acc_origin_note"))
+
     try:
-        sb = get_scoreboard_data()
-        preds = get_predictions()
+        views = get_track_record_views()
+        seed_preds = views["seed_preds"]
+        live_preds = views["live_preds"]
+        seed_sb = views["seed_scoreboard"]
+        live_sb = views["live_scoreboard"]
+        upcoming = views["upcoming_live"]
+        seed_ids = views["seed_ids"]
+        all_preds = seed_preds + live_preds
+        sb_combined = {
+            "total": seed_sb["total"] + live_sb["total"],
+            "open": seed_sb["open"] + live_sb["open"],
+            "resolved": seed_sb["resolved"] + live_sb["resolved"],
+            "mean_brier": None,
+        }
+        all_briers = [
+            p.brier for p in all_preds
+            if p.brier is not None
+        ]
+        if all_briers:
+            sb_combined["mean_brier"] = round(sum(all_briers) / len(all_briers), 4)
     except Exception as e:
         st.error(t("acc_registry_err", e=e))
-        sb = {"total": 0, "open": 0, "resolved": 0, "mean_brier": None, "calibration": []}
-        preds = []
-
-    col1, col2, col3, col4 = st.columns(4)
-    with col1:
-        st.markdown(
-            f'<div class="metric-card"><div class="metric-value">{sb["total"]}</div>'
-            f'<div class="metric-label">{t("acc_total")}</div></div>',
-            unsafe_allow_html=True,
-        )
-    with col2:
-        st.markdown(
-            f'<div class="metric-card"><div class="metric-value">{sb["open"]}</div>'
-            f'<div class="metric-label">{t("acc_open")}</div></div>',
-            unsafe_allow_html=True,
-        )
-    with col3:
-        st.markdown(
-            f'<div class="metric-card"><div class="metric-value">{sb["resolved"]}</div>'
-            f'<div class="metric-label">{t("acc_resolved")}</div></div>',
-            unsafe_allow_html=True,
-        )
-    with col4:
-        brier_str = f"{sb['mean_brier']:.4f}" if sb['mean_brier'] is not None else "N/A"
-        st.markdown(
-            f'<div class="metric-card"><div class="metric-value">{brier_str}</div>'
-            f'<div class="metric-label">{t("acc_brier")}</div></div>',
-            unsafe_allow_html=True,
-        )
+        seed_preds, live_preds, upcoming = [], [], []
+        seed_sb = live_sb = sb_combined = {
+            "total": 0, "open": 0, "resolved": 0, "mean_brier": None, "calibration": [],
+        }
+        seed_ids = set()
+        all_preds = []
+
+    st.markdown(f"### {t('acc_seed_panel')}")
+    st.caption(t("acc_seed_explain"))
+    _scoreboard_cards(seed_sb, prefix="seed")
+
+    st.markdown(f"### {t('acc_live_panel')}")
+    st.caption(t("acc_live_explain"))
+    _scoreboard_cards(live_sb, prefix="live")
 
     st.caption(t("acc_brier_note"))
     with st.expander(t("acc_brier_explain_title"), expanded=False):
         st.markdown(t("acc_brier_explain_body"))
 
-    calibration = sb.get("calibration", [])
+    # Calibration curve uses all resolved predictions
+    resolved_all = [
+        p for p in all_preds
+        if p.status in (Status.resolved_true, Status.resolved_false)
+    ]
+    if resolved_all:
+        from services.track_record import scoreboard_subset
+        calibration = scoreboard_subset(resolved_all).get("calibration", [])
+    else:
+        calibration = []
+
     if calibration:
         df_cal = pd.DataFrame(calibration)
         fig = go.Figure()
         fig.add_trace(go.Scatter(
             x=[0.5, 1.0], y=[0.5, 1.0],
-            mode="lines",
-            name="Perfect",
-            line=dict(color="#8b949e", dash="dash"),
-            hoverinfo="none",
+            mode="lines", name="Perfect",
+            line=dict(color="#8b949e", dash="dash"), hoverinfo="none",
         ))
         fig.add_trace(go.Scatter(
             x=df_cal["avg_confidence"], y=df_cal["actual_hit_rate"],
-            mode="markers+lines",
-            name="Agent",
+            mode="markers+lines", name="Agent",
             line=dict(color="#58a6ff", width=3),
-            marker=dict(size=10, color="#bc8cff", symbol="circle", line=dict(color="#58a6ff", width=2)),
+            marker=dict(size=10, color="#bc8cff", symbol="circle",
+                        line=dict(color="#58a6ff", width=2)),
             text=[f"N={row['n']}" for _, row in df_cal.iterrows()],
             hoverinfo="text+x+y",
         ))
@@ -90,80 +172,71 @@ def render(scenario_input: dict, prior, job_radar_cfg: dict):
     else:
         st.info(t("acc_no_cal"))
 
+    st.markdown("---")
+    st.markdown(f"### {t('acc_upcoming_title')}")
+    st.caption(t("acc_upcoming_intro"))
+    if upcoming:
+        st.dataframe(
+            pd.DataFrame([{
+                t("col_statement"): p.statement,
+                t("col_confidence"): f"{p.confidence*100:.0f}%",
+                t("col_resolution"): p.resolution_date.strftime("%Y-%m-%d"),
+                t("col_days_left"): days_until_resolution(p),
+                t("col_rationale"): p.resolution_criteria[:120],
+            } for p in upcoming]),
+            use_container_width=True,
+            hide_index=True,
+        )
+    else:
+        st.info(t("acc_no_upcoming"))
+
     st.markdown("---")
     st.subheader(t("acc_explore"))
 
     all_label = filter_all_label()
-    if preds:
-        cat_options = [all_label] + sorted(list(set(p.category for p in preds)))
+    if all_preds:
+        cat_options = [all_label] + sorted(list(set(p.category for p in all_preds)))
         selected_cat = st.selectbox(
             t("acc_filter_cat"),
             cat_options,
             format_func=lambda c: all_label if c == all_label else prediction_category_label(c),
         )
 
-        filtered_preds = preds
-        if selected_cat != all_label:
-            filtered_preds = [p for p in preds if p.category == selected_cat]
+        def _filter_group(group: list[Prediction]) -> list[Prediction]:
+            if selected_cat == all_label:
+                return group
+            return [p for p in group if p.category == selected_cat]
 
-        open_preds = [p for p in filtered_preds if p.status in (Status.open, Status.due)]
-        open_preds.sort(key=lambda p: p.confidence, reverse=True)
+        seed_f = _filter_group(seed_preds)
+        live_f = _filter_group(live_preds)
 
-        resolved_preds = [
-            p for p in filtered_preds
-            if p.status in (Status.resolved_true, Status.resolved_false)
-        ]
-        resolved_preds.sort(
+        seed_resolved = sorted(
+            [p for p in seed_f if p.status in (Status.resolved_true, Status.resolved_false)],
             key=lambda p: p.resolved_at or datetime.min.replace(tzinfo=timezone.utc),
             reverse=True,
         )
+        live_resolved = sorted(
+            [p for p in live_f if p.status in (Status.resolved_true, Status.resolved_false)],
+            key=lambda p: p.resolved_at or datetime.min.replace(tzinfo=timezone.utc),
+            reverse=True,
+        )
+        live_open = sorted(
+            [p for p in live_f if p.status in (Status.open, Status.due)],
+            key=lambda p: p.resolution_date,
+        )
 
-        col_active, col_resolved = st.columns(2)
-        with col_active:
-            st.markdown(f"### {t('acc_active', n=len(open_preds))}")
-            if open_preds:
-                active_df = pd.DataFrame([{
-                    t("col_id"): p.fingerprint(),
-                    t("col_statement"): p.statement,
-                    t("col_category"): prediction_category_label(p.category),
-                    t("col_confidence"): f"{p.confidence*100:.0f}%",
-                    t("col_horizon"): p.horizon,
-                    t("col_resolution"): p.resolution_date.strftime("%Y-%m-%d"),
-                } for p in open_preds])
-                st.dataframe(active_df, width="stretch", hide_index=True)
-            else:
-                st.write(t("acc_no_active"))
-
-        with col_resolved:
-            st.markdown(f"### {t('acc_resolved_list', n=len(resolved_preds))}")
-            if resolved_preds:
-                col_src_key = t("col_sources")
-                resolved_rows = []
-                for p in resolved_preds:
-                    first_src = (p.sources or [])[0] if p.sources else ""
-                    resolved_rows.append({
-                        t("col_id"): p.fingerprint(),
-                        t("col_statement"): p.statement,
-                        t("col_outcome"): t("outcome_true") if p.outcome else t("outcome_false"),
-                        t("col_confidence"): f"{p.confidence*100:.0f}%",
-                        t("col_brier"): f"{p.brier:.4f}" if p.brier is not None else "",
-                        t("col_rationale"): p.judged_rationale,
-                        col_src_key: first_src,
-                    })
-                resolved_df = pd.DataFrame(resolved_rows)
-                st.dataframe(
-                    resolved_df,
-                    column_config={
-                        col_src_key: st.column_config.LinkColumn(
-                            t("col_sources"),
-                            display_text=t("col_sources_label"),
-                        ),
-                    },
-                    use_container_width=True,
-                    hide_index=True,
-                )
-            else:
-                st.write(t("acc_no_resolved"))
+        st.markdown(f"#### {t('acc_seed_panel')}")
+        st.markdown(f"**{t('acc_resolved_list', n=len(seed_resolved))}**")
+        _resolved_table(seed_resolved, t("origin_seed"))
+
+        st.markdown(f"#### {t('acc_live_panel')}")
+        col_live_open, col_live_res = st.columns(2)
+        with col_live_open:
+            st.markdown(f"**{t('acc_active', n=len(live_open))}**")
+            _active_table(live_open, t("origin_live"))
+        with col_live_res:
+            st.markdown(f"**{t('acc_resolved_list', n=len(live_resolved))}**")
+            _resolved_table(live_resolved, t("origin_live"))
     else:
         st.write(t("acc_no_preds"))
 
@@ -171,35 +244,27 @@ def render(scenario_input: dict, prior, job_radar_cfg: dict):
     st.markdown("---")
     st.subheader(t("acc_download_title"))
     st.markdown(t("acc_download_intro"))
-    if resolved_preds if preds else []:
-        try:
-            csv_rows = []
-            for p in (resolved_preds if preds else []):
-                csv_rows.append({
-                    "id": p.fingerprint(),
-                    "statement": p.statement,
-                    "category": p.category,
-                    "confidence": p.confidence,
-                    "horizon": p.horizon,
-                    "resolution_date": p.resolution_date.isoformat() if p.resolution_date else "",
-                    "outcome": p.outcome,
-                    "brier": p.brier,
-                    "judged_rationale": p.judged_rationale,
-                    "sources": " | ".join(p.sources or []),
-                })
-            if csv_rows:
-                import io
-                csv_df = pd.DataFrame(csv_rows)
-                buf = io.StringIO()
-                csv_df.to_csv(buf, index=False)
-                st.download_button(
-                    label=t("acc_download_btn"),
-                    data=buf.getvalue().encode("utf-8"),
-                    file_name="forecaster_track_record.csv",
-                    mime="text/csv",
-                )
-        except Exception:
-            pass
+    resolved_all_sorted = sorted(
+        [p for p in all_preds if p.status in (Status.resolved_true, Status.resolved_false)],
+        key=lambda p: p.resolved_at or datetime.min.replace(tzinfo=timezone.utc),
+        reverse=True,
+    )
+    if resolved_all_sorted:
+        csv_rows = [
+            prediction_to_csv_row(
+                p,
+                "seed" if prediction_origin(p, seed_ids) == "seed" else "live",
+            )
+            for p in resolved_all_sorted
+        ]
+        buf = io.StringIO()
+        pd.DataFrame(csv_rows).to_csv(buf, index=False)
+        st.download_button(
+            label=t("acc_download_btn"),
+            data=buf.getvalue().encode("utf-8"),
+            file_name="forecaster_track_record.csv",
+            mime="text/csv",
+        )
 
     st.markdown("---")
     st.subheader(t("acc_market_title"))