traceforge: pipeline behind a Gradio Space, keyed by content hash

BUILD.json

@@ -0,0 +1,7 @@
+{
+  "built_at": "2026-10-26T22:04:33",
+  "commit": "6bc868a",
+  "files": 48,
+  "with_store": false,
+  "with_card": true
+}

README.md

@@ -1,13 +1,122 @@
 ---
-title: Traceforge
-emoji: 😻
-colorFrom: gray
-colorTo: purple
+title: traceforge
+emoji: 🧭
+colorFrom: indigo
+colorTo: green
 sdk: gradio
 sdk_version: 6.30.0
-python_version: '3.13'
 app_file: app.py
 pinned: false
+license: other
+short_description: Turn real agent traces into ideal-world references
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# traceforge
+
+Turn a real agent trace into its **ideal-world reference**: the same work,
+with the dead ends removed, gated for structure and grounding.
+
+This Space is the portable face of the traceforge pipeline. The pipeline
+itself is unchanged — triage labels every message, a windowed rewriter
+produces the ideal path, and a judge admits only traces that pass the six
+pillars. The UI hides that machinery behind three actions.
+
+## Submit a trace
+
+Upload a JSON/JSONL trace (traceforge native, OpenAI chat, or a bare message
+list) or paste it. The app computes a **content hash** of the seed and
+queues it.
+
+**One seed, one output.** The hash covers the conversation content — message
+text, tool calls and the set of tools the agent had — and excludes session
+ids, timestamps, cwd and tool-call ids. Submitting the same trace twice
+returns the existing output instead of creating a second one. That is what
+makes the output dataset a set, not a log.
+
+## Sponsor a dataset
+
+Upload a JSONL of traces. Every row is hashed; new seeds are queued and
+attributed to the sponsor name you give, known seeds are skipped. Processed
+rows join the public output dataset. The dashboard shows the sponsor
+leaderboard.
+
+## Dashboard
+
+Public statistics over this Space's corpus and, when bundled, the reference
+corpus: seeds, outputs, queue, failure rate, task mix, languages, runtime
+distribution, and **time sensitivity**.
+
+## Time sensitivity
+
+A trace whose correct answer is a fact that changes over time is a bad
+training target when the agent had no way to look that fact up. Train on it
+and the model learns to *assert* the fact from its weights — "the current
+president is X" — which is wrong the moment X changes.
+
+The same question is a good target when the agent had a retrieval tool. The
+ideal path calls the tool and answers from its result, so what the model
+imitates is the procedure (search, read, answer), not the fact. The
+procedure does not decay.
+
+| class | meaning |
+|---|---|
+| `time_sensitive` | temporal fact, no retrieval tool — the answer decays |
+| `time_anchored` | temporal fact, but a retrieval tool can fetch it |
+| `not_time_sensitive` | no time-dependent fact in the task |
+
+`time_anchored` carries a sub-flag: whether the tool was actually **called**.
+A temporal task with a search tool available but never used is anchored in
+principle — the ideal rewrite should make the agent call it — but the raw
+trace answered from weights, so it needs a rewrite rather than a pass.
+
+The detector is deliberately conservative: a false `time_sensitive` throws
+away a good trace, so a pattern must name the **fact** (an officeholder, a
+price, a released version, the weather), not just a temporal word. "The
+current branch" and "the latest fix round" are local artifacts of the
+session, not decaying facts.
+
+## Configuration
+
+Set these as Space secrets / variables:
+
+| variable | meaning |
+|---|---|
+| `TRACEFORGE_BASE_URL` | OpenAI-compatible endpoint (`…/v1`) |
+| `TRACEFORGE_LLM_KEY` | bearer token |
+| `TRACEFORGE_MODEL` | writer/judge model id |
+| `TRACEFORGE_TRIAGE_MODEL` | cheap labeler (default: `TRACEFORGE_MODEL`) |
+| `TRACEFORGE_JUDGE_MODEL` | judge override (default: `TRACEFORGE_MODEL`) |
+| `TRACEFORGE_DATA_DIR` | store root (default `./data/space`) |
+| `TRACEFORGE_SPACE_WORKERS` | parallel pipeline workers (default 1) |
+
+Without an endpoint the app still starts and the dashboard still works; the
+Submit tab reports that the pipeline is not configured instead of hanging.
+
+## Storage
+
+The store is content-addressed on disk:
+
+```
+data/space/index.jsonl          append-only journal (last row per hash wins)
+data/space/seeds/<hash>.json    the seed exactly as submitted
+data/space/outputs/<hash>.json  the improved trace
+```
+
+A Hugging Face Space's disk is ephemeral unless persistent storage is
+attached. For a durable public corpus, mount persistent storage at
+`/data` and set `TRACEFORGE_DATA_DIR=/data/space`, or sync the store to a
+dataset repo.
+
+## Local use
+
+```bash
+pip install -e ".[webui]"
+python app.py                       # http://127.0.0.1:7860
+traceforge space serve --port 7860
+traceforge space hash trace.json    # print the seed hash
+traceforge space stats              # store census
+```
+
+## License
+
+Private research corpus. All rights reserved.

app.py

@@ -0,0 +1,24 @@
+"""Hugging Face Spaces entry point.
+
+The Space runs this file. Everything it needs is in `traceforge/space/`;
+configuration comes from Space secrets / variables:
+
+    TRACEFORGE_BASE_URL      OpenAI-compatible endpoint (…/v1)
+    TRACEFORGE_LLM_KEY       bearer token
+    TRACEFORGE_MODEL         writer/judge model id
+    TRACEFORGE_TRIAGE_MODEL  cheap labeler (default: TRACEFORGE_MODEL)
+    TRACEFORGE_DATA_DIR      store root (default: ./data/space)
+
+Locally: `python app.py` (or `traceforge space serve`).
+"""
+from __future__ import annotations
+
+import os
+
+from traceforge.space.app import CSS, build_app
+
+demo = build_app()
+demo.queue(default_concurrency_limit=4)
+
+if __name__ == "__main__" or os.environ.get("SPACE_ID"):
+    demo.launch(show_error=True, css=CSS)

data/dataset/dataset_card.json

@@ -0,0 +1,167 @@
+{
+  "traces": 3652,
+  "dedup": {
+    "removed": 1,
+    "pct": 0.0,
+    "key": "(task, tool-names + sha1(args))"
+  },
+  "languages": {
+    "en": 3444,
+    "de": 208
+  },
+  "agent_roles": {
+    "subagent": 278,
+    "orchestrator": 11
+  },
+  "tasks": {
+    "review": 1118,
+    "debug": 682,
+    "implement": 620,
+    "research": 468,
+    "other": 293,
+    "test": 165,
+    "ops": 108,
+    "data-analysis": 86,
+    "creative": 54,
+    "refactor": 44,
+    "office-docs": 14
+  },
+  "messages": {
+    "total": 112327,
+    "by_role": {
+      "assistant": 54700,
+      "tool": 53265,
+      "user": 4070,
+      "system": 292
+    },
+    "per_trace": {
+      "mean": 30.8,
+      "median": 20,
+      "p90": 64,
+      "max": 252
+    }
+  },
+  "tool_calls": {
+    "total": 53265,
+    "per_trace": {
+      "mean": 14.6,
+      "median": 9,
+      "p90": 32,
+      "max": 125
+    },
+    "traces_with_none": 1,
+    "distinct_tools": 553,
+    "top_tools": {
+      "Bash": 14327,
+      "exec": 13084,
+      "terminal": 3813,
+      "read_file": 3008,
+      "Read": 2085,
+      "search_files": 1837,
+      "execute_code": 1601,
+      "tool_call": 1512,
+      "Edit": 1497,
+      "skill_view": 918,
+      "Grep": 917,
+      "patch": 758,
+      "unlock_discoverable_agent_tool": 457,
+      "call_discoverable_agent_tool": 457,
+      "wait": 436
+    }
+  },
+  "user_turns": {
+    "per_trace": {
+      "mean": 1.1,
+      "median": 1,
+      "p90": 1,
+      "max": 18
+    },
+    "single_turn_traces": 3352,
+    "multi_turn_traces": 300
+  },
+  "tokens": {
+    "method": "chars/4 (prose proxy). Calibrated against a real tokenizer on this corpus: chars/4 runs ~13% low (median 21.4k est vs 24.5k actual), so scale by ~1.15 for a central estimate; tool output is code/JSON and tokenizes denser than prose.",
+    "budget": 131072,
+    "chars_total": 323084312,
+    "est_total": 80769721,
+    "per_trace": {
+      "mean": 22116.6,
+      "median": 16996,
+      "p90": 53126,
+      "max": 119767
+    },
+    "fits_budget": {
+      "4096": {
+        "traces": 1043,
+        "pct": 28.6
+      },
+      "8192": {
+        "traces": 1218,
+        "pct": 33.4
+      },
+      "16384": {
+        "traces": 1773,
+        "pct": 48.5
+      },
+      "65536": {
+        "traces": 3570,
+        "pct": 97.8
+      },
+      "131072": {
+        "traces": 3652,
+        "pct": 100.0
+      }
+    }
+  },
+  "split": null,
+  "supervision": {
+    "contract": "loss on assistant spans only; tool/user/system spans are masked context",
+    "chars_by_role": {
+      "user": 4777684,
+      "assistant": 41191827,
+      "tool": 276324543,
+      "system": 790258
+    },
+    "assistant_share_pct": 12.7,
+    "tool_share_pct": 85.5,
+    "recovery": {
+      "traces": 47,
+      "pct": 1.3,
+      "cap_pct": 15.0,
+      "over_cap": false,
+      "method": "verified label: exactly one environment-caused failure (flaky endpoint, missing dependency, exhausted resource) followed by exactly one corrective step; model-caused errors and retry loops disqualify (docs/DATA_QUALITY.md §4.1)"
+    }
+  },
+  "difficulty": {
+    "tool_calls": {
+      "mean": 14.6,
+      "median": 9,
+      "p90": 32,
+      "max": 125
+    },
+    "messages": {
+      "mean": 30.8,
+      "median": 20,
+      "p90": 64,
+      "max": 252
+    },
+    "est_tokens": {
+      "mean": 22116.6,
+      "median": 16996,
+      "p90": 53126,
+      "max": 119767
+    },
+    "bands": {
+      "calls_lt_10": 1916,
+      "calls_10_24": 1149,
+      "calls_25_49": 450,
+      "calls_ge_50": 137
+    }
+  },
+  "notes": [
+    "identity/refusal burns gated (traceforge/identity.py)",
+    "clip markers, template leaks, permission-ask finals gated",
+    "residual-defect estimate ~5% (selfhosted cross-exam, triaged)",
+    "reasoning channel dropped per message (keep_reasoning=False)"
+  ]
+}

data/space/reference_census.json

@@ -0,0 +1,23 @@
+{
+  "key": {
+    "mtime": 1793044902,
+    "size": 406218197
+  },
+  "census": {
+    "total": 3652,
+    "classes": {
+      "time_sensitive": 13,
+      "time_anchored": 12,
+      "not_time_sensitive": 3627
+    },
+    "labels": {
+      "time_sensitive": "answer decays: temporal fact, no retrieval tool",
+      "time_anchored": "temporal fact, but a retrieval tool can fetch it",
+      "not_time_sensitive": "no time-dependent fact in the task"
+    },
+    "needs_rewrite": 0,
+    "time_sensitive_share": 0.0036,
+    "source": "data/dataset/train.jsonl",
+    "available": true
+  }
+}

guidelines/IDEAL_TRACE_GUIDELINES.md

@@ -0,0 +1,230 @@
+# Ideal-Trace Guidelines — v0.3
+
+> **Authoritative spec for what an "ideal trace" looks like.** Read at runtime
+> by the improver, digest, long-trace and review stages. The six quality
+> pillars (§3) are owner-decided and binding. §4 states the training-signal
+> contract that makes the pillars necessary; §6 scopes what this reference is
+> for (evaluation, not training). Evidence base: `docs/DATA_QUALITY.md`.
+>
+> **Integrity rule:** this file is a spec, not a notebook. Keep it under
+> 24 KB. Never paste a worked example, a trace, or your own reasoning into it
+> — examples live in `data/traces/improved/`. `tests/test_guidelines.py`
+> enforces this.
+
+## 1. Purpose
+
+An ideal trace is the **ideal-world execution of the same task**: same intent,
+same real environment, flawless execution. It is NOT a re-execution — it is a
+rewrite of the recorded trace, using the trace's own tool outputs as the only
+evidence about the world.
+
+The ideal trace is the **reference standard** the benchmark scores a model
+against: the less the pipeline has to edit a model's trace to reach it, the
+better the model. The training-signal contract in §4 is retained because the
+reference must also be a valid target if the corpus is reused for SFT — but
+the official scope is evaluation, not training.
+
+## 2. Stable structural rules (hard requirements)
+
+1. **Preserve the task intent.** The first user message carries the task. The
+   ideal trace solves *that* task. When the source trace starts with a handoff
+   artifact — a Hermes `[CONTEXT COMPACTION …]` row, or a Claude Code "This
+   session is being continued…" summary row — the first REAL user task message
+   defines the intent: reproduce it verbatim and byte-exact as the ideal
+   trace's opening message.
+2. **Tool-call pairing.** Every assistant tool_call has exactly one matching
+   tool result; every tool result references an existing call; tool arguments
+   are valid JSON strings.
+3. **Message shape.** user / assistant(+calls) / tool / … / final assistant.
+   The last message is the assistant's final answer. No two consecutive user
+   rows, no two consecutive text-only assistant rows.
+4. **Metadata is set by the harness**, not by the improver.
+5. **No scaffold leakage.** Never copy prompt scaffolding, clip markers
+   (`…[+N chars clipped]`), compaction banners, or harness self-talk into the
+   ideal trace.
+
+## 3. The six quality pillars (owner-decided, binding)
+
+**P1 — Efficient but deep reasoning.**
+Reasoning is sharp and high-signal: the key decision logic per turn (why this
+call, what the result implies, what to verify next). No filler, no restating
+the obvious, no rambling. Depth where it matters (diagnoses, trade-offs,
+verification logic) — silence where it doesn't. **Answer focus:** every
+answer, final or mid-trace, gives exactly what its question asked for. A
+follow-up gets only the new piece; never re-present facts the user already
+received. Over-answering is a P1 violation.
+
+**Deliver the substance, not a description of it.** The final answer must
+present the concrete result the task asked for — the actual numbers, names,
+paths, values, or code from the tool evidence — not a summary of what was
+retrieved. "I pulled the nutrition data and the recipe" is poor; the data and
+the recipe ARE the answer. Naming the work instead of showing its result fails
+the task even when every tool call was correct.
+
+**P2 — No committed mistakes (mine the negative signal first).**
+Before writing, identify every mistake in the real trace: failed tool calls
+(error results), retry loops, redundant calls, wrong turns that later needed
+correction, and USER FEEDBACK (corrections, re-asks, complaints). The ideal
+trace commits none of them: the correct route is taken the first time. Where
+the real agent repaired itself, the ideal trace shows only the correct action
+— not the failure, not the repair.
+*Why:* the assistant turns are the loss-bearing target. A mistake in the
+target is imitated; a mistake in the context is not. See §4 for the one
+narrow exception (environment-caused failures).
+
+**P3 — Perfect tool calls, only when really needed.**
+Every call is necessary, correctly parameterized, and the natural next step.
+Independent calls are batched in parallel.
+
+**Batching is structural, not stylistic.** When the real trace issues two
+calls in separate assistant turns but the second does not consume the first's
+output, the ideal path issues them in ONE assistant turn carrying both
+`tool_calls`, with the two tool results following in the same order. Do not
+preserve the source's one-call-per-turn rhythm — that rhythm is the defect.
+Measured 2026-10-25: 75 of 621 repair-exhausted failures are exactly this
+shape. A duplicate call (same tool, same arguments, same turn) is never
+batched; it is dropped. No exploratory calls, no re-reads
+of what is already in context, no call whose result the narrative doesn't
+use. When the ideal path makes the SAME call as the real trace, the tool
+result must be the real trace's output — verbatim or a faithful compression
+of it — never a plausible re-simulation of what the call might have printed.
+
+**P4 — Fact-driven, no hallucinations (two layers, both binding).**
+Layer 1 (trace-internal): reuse the real trace's tool outputs verbatim
+wherever the ideal path makes the same call; never invent files, commands, or
+results; no fake success — the final answer is grounded in real evidence from
+the trace. Layer 2 (external world): statements about the outside world
+(library versions, API behavior, dates, technical facts) must be either (a)
+taken from the trace's own evidence or (b) kept out unless you are certain
+they are correct. The harness fact-checks external claims with live web
+searches after you finish — unsupported claims send the trace back.
+**Construction order:** decide the ideal call path → copy each result
+byte-verbatim from `outputs.jsonl` → only then write reasoning and the final
+answer around those real results. A tool result you cannot quote is not
+writable: reroute the path or drop the claim. Stitching individually-plausible
+fragments is fabrication and is the single most common rejection.
+
+**P5 — Perfect grammar and language quality, single language.**
+One language per trace: the language of the original first user message. No
+code-switching, no stray fragments in other languages. Assistant prose is
+clean, grammatically perfect, professionally concise. The user message is kept
+verbatim — realism of the input matters more than its typos. Never open by
+restating the brief: the first assistant turn must do work (a tool call, a
+decision, a finding), not echo the user's own text back at them.
+
+**P6 — No refusals.**
+The ideal agent completes the task without refusing. If the source task
+genuinely cannot be completed (harmful or impossible), do NOT write an ideal
+version: reply starting with `REJECTED:` plus a one-line reason, and write no
+output file.
+
+## 4. Training-signal contract (why the pillars are shaped this way)
+
+The corpus is consumed by SFT: loss is computed on **assistant target tokens
+only**; tool results, user turns and system prompts are masked context. The
+model imitates what is in the target span.
+
+> **Failures may be context. Failures must never be target.**
+
+Therefore:
+
+1. **A model-caused error must never appear in an assistant turn** — not as
+   "I'll try X", not as a failed call, not as a self-correction narrative.
+   Wrong tool name, wrong arguments, wrong path, bad reasoning: route around
+   it. This is P2.
+2. **An environment-caused error may appear as a tool result** (context), with
+   the correct adaptation in the next assistant turn. This is the only
+   exception to P2, and it exists because a corpus of only first-try-clean
+   paths loses the only examples of "tool returned an error → read it →
+   adapt", and silently skews the corpus toward easy tasks.
+3. **The recovery is the target, never the mistake.** If the ideal path cannot
+   avoid the error, the assistant's next turn is the supervised recovery; the
+   error text stays byte-verbatim in the tool row.
+
+### 4.1 The narrow recovery carve-out
+
+A trace may keep a real failure **only** when ALL of these hold:
+
+- the failure is **environment-caused** — a tool returned an error the ideal
+  agent could not have foreseen (flaky endpoint, missing dependency, a
+  genuinely absent file). Not a wrong tool name, wrong arguments, wrong path,
+  or a reasoning error;
+- the error text is **byte-verbatim** from `outputs.jsonl` — never composed,
+  never paraphrased, never invented;
+- there is **one corrective step per error**; two or more consecutive
+  identical failed calls disqualify the trace;
+- the harness records the trace as a **retained recovery** (manifest + dataset
+  card) so the corpus can report failure density and a trainer can filter it
+  out;
+- the final state is **verified** — a clean-looking recovery whose final
+  answer is wrong is rejected outright.
+
+Everything else follows P2: route around the mistake, show only the correct
+action. When in doubt, route around it — the carve-out is an exception, not a
+license.
+
+### 4.2 What must never be in the target span
+
+- A fabricated or paraphrased error message.
+- A failure the ideal agent could have avoided by reading its own context.
+- A "let me try again" narrative whose first attempt is not shown (the model
+  learns the shape of recovery without the substance).
+- A recovery loop longer than one corrective step.
+- A success claim that the tool results do not support.
+
+## 5. Rewrite procedure (binding)
+
+1. Read these guidelines in full.
+2. Mine the negative signal: list the real trace's failed calls, retries,
+   wrong turns, and user feedback (P2).
+3. Classify each failure: **harness artefact** (wrong cwd, missing env var,
+   stale workdir, tool not installed) → route around it; **environment-caused
+   and unavoidable** → eligible for the §4.1 carve-out; **model-caused** →
+   route around it, always.
+4. Map the ideal path: the minimal set of correct, necessary calls (P3).
+5. Copy every tool result on that path byte-verbatim from `outputs.jsonl`
+   (P4), then write the reasoning and final answer around them (P1, P5).
+6. Write the ideal trace JSON (schema in the task prompt).
+7. Verify: re-read the file; check JSON validity, call/result pairing, every
+   pillar, and that no failure sits in an assistant turn. Fix, then summarize
+   the changes.
+
+## 6. Scope: what this corpus is and is not
+
+This is an **evaluation reference**. It is not a training corpus by official
+scope, not a preference corpus, and not an RL environment.
+
+- **The benchmark** scores a model by how little the pipeline has to edit its
+  traces to reach the reference. The reference is the ruler, not the product.
+- **SFT reuse** is possible — the §4 contract keeps the reference a valid
+  target — but it is a downstream decision, not this repo's purpose.
+- **KTO/DPO** needs rejected samples from the policy being aligned,
+  length-matched to the chosen side, with a bounded negative loss. A
+  hand-written "bad" trace is a strawman and teaches length/style, not
+  preference.
+- **PPO/GRPO** needs rollouts and a verifiable reward; failed rollouts are
+  advantage, not imitation. A static JSONL of ideal traces cannot supply that
+  signal.
+- If those corpora are built later, they are separate artifacts with separate
+  rules. Do not repurpose this corpus by relabelling its rows.
+
+Corpus-level obligations that belong to the harness, not the improver: report
+failure density and difficulty distribution per batch; cap retained recoveries
+at ≤15% of traces until an in-repo ablation says otherwise; state the masking
+contract in the dataset card; dedup at the (task, tool-sequence) level.
+
+## 7. Remaining open items
+
+- System prompt: keep original vs. normalize to a canonical corpus prompt.
+- Corpus-level dedup threshold for batch runs (Jaccard > 0.9 is the current
+  audit default).
+- Whether the §4.1 carve-out improves or degrades held-out recovery rate —
+  needs an in-repo ablation (same traces, with and without retained
+  recoveries, scored on a forced-error recovery eval).
+
+## 8. Provenance
+
+Improved traces carry source session id, parent session id, and generator
+version; the manifest records per-trace status, attempts, and errors. The
+harness records retained recoveries (§4.1) in the manifest and reports failure
+density in the dataset card.

requirements.txt

@@ -0,0 +1,3 @@
+gradio>=6.0
+pyyaml>=6
+huggingface_hub>=0.24

traceforge/__init__.py

@@ -0,0 +1,16 @@
+"""traceforge — an evaluation harness for agent models.
+
+Reads real agent traces (Hermes Agent, Claude Code, Codex), then uses the
+Hermes Agent CLI itself as the improver harness to rewrite them into
+ideal-world references, validated against the guidelines in
+guidelines/IDEAL_TRACE_GUIDELINES.md. The benchmark scores a model by how
+little that rewrite has to change: if the pipeline has nothing to do, the
+model is already perfect.
+"""
+
+__version__ = "0.1.0"
+
+from traceforge.config import Config
+from traceforge.trace import Message, ToolCall, Trace
+
+__all__ = ["Config", "Message", "ToolCall", "Trace", "__version__"]

traceforge/audit.py

@@ -0,0 +1,272 @@
+"""Corpus audit: mechanical, LLM-free quality gates over the PROMOTED corpus.
+
+Supervisor tool — run before any training export. Zero LLM calls; every
+finding is a mechanical pattern with a trace + index citation, so it can
+drive repairs or corpus pruning deterministically.
+
+Audit classes:
+  1. harness self-talk — compaction/continuation narrative, spillover
+     slices, retry language in ASSISTANT prose (raw-session plumbing an
+     ideal trace must never show a model saying)
+  2. clip markers — the static gate, re-checked corpus-wide
+  3. empty/degenerate assistant turns — prose turns with < 20 chars
+     that are not a tool-call carrier
+  4. judge/quality residue — "defect": false entries that leaked into
+     promoted artifacts (should be impossible; proves a gate bypass)
+  5. near-duplicates — normalized assistant-prose shingles > 0.9
+     Jaccard across traces (overfit risk for the trainee)
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+import re
+from collections import Counter
+
+# ---------------------------------------------------------------- patterns
+
+# Harness/plumbing noise in ASSISTANT prose. Deliberately narrower than
+# the raw-trace patterns: words like "harness"/"retry" are legitimate in
+# domain tasks (test harness, git retry), so we anchor on unambiguous
+# self-referential plumbing.
+SELF_TALK_RE = re.compile(
+    r"("
+    r"[Cc]ontext was compacted"
+    r"|\[\s*CONTEXT COMPACTION\s*\]"
+    r"|⟪HERMES-CONTEXT-COMPRESSION"
+    r"|spillover slice"
+    r"|\bcompacted mid-"
+    r"|\bmid context-compression\b"
+    r"|\bcontinuation handoff\b"
+    r"|CONTEXT-COMPRESSION:"
+    r")"
+)
+
+# A task ABOUT harness/crash/database repair legitimately discusses
+# compaction state — that is domain content, not self-talk leakage.
+DOMAIN_TASK_RE = re.compile(
+    r"crash|repair|database|hermes|corrupt|restore|recover|diagnos",
+    re.IGNORECASE,
+)
+
+CLIP_RE = re.compile(r"…?\s*\[\+[\d,.]+ ?(?:chars|bytes) clipped\]")
+
+# same-family judge leak marker: judge entries surviving into artifacts
+JUDGE_LEAK_RE = re.compile(r'"defect"\s*:\s*false|"answer_quality"', re.I)
+
+# identity burns / refusal phrasings — must never reach the reference
+from traceforge.identity import IDENTITY_RE, REFUSAL_RE  # noqa: E402
+from traceforge.splice import _task_demands_json  # noqa: E402
+
+# a final answer that IS a raw JSON dump (not fenced, parses as JSON)
+# teaches the trainee to close sessions with machine output
+def _final_is_blob(msgs: list) -> bool:
+    if not msgs:
+        return False
+    last = msgs[-1]
+    if last.get("role") != "assistant":
+        return False
+    c = (last.get("content") or "").strip()
+    if len(c) < 400 or c.startswith("```"):
+        return False
+    if not (c.startswith("{") or c.startswith("[")):
+        return False
+    try:
+        json.loads(c)
+        return True
+    except json.JSONDecodeError:
+        return False
+
+
+def _shingle(text: str, k: int = 8) -> set[str]:
+    """Word k-gram hashes for near-duplicate detection."""
+    words = re.findall(r"\w+", text.lower())
+    if len(words) < k:
+        return set()
+    return {hashlib.md5(
+        " ".join(words[i:i + k]).encode()).hexdigest()
+        for i in range(len(words) - k + 1)}
+
+
+def _jaccard(a: set, b: set) -> float:
+    if not a or not b:
+        return 0.0
+    return len(a & b) / len(a | b)
+
+
+def audit_trace(t: dict) -> list[dict]:
+    """One promoted trace -> list of findings."""
+    findings = []
+    sid = t.get("parent_session_id") or t.get("session_id", "?")
+    msgs = t.get("messages", [])
+    # opening user task (skip compaction/continuation handoffs)
+    task = next((m.get("content") or "" for m in msgs
+                 if m.get("role") == "user"
+                 and not (m.get("content") or "").startswith(
+                     ("[CONTEXT", "⟪"))), "")
+    task_domain = bool(DOMAIN_TASK_RE.search(task))
+    if _final_is_blob(msgs) and not _task_demands_json(task):
+        findings.append({
+            "cls": "final_blob", "i": len(msgs) - 1,
+            "detail": "final answer is a raw JSON dump, not prose",
+            "ctx": (msgs[-1].get("content") or "")[:160].replace("\n", " "),
+        })
+    for i, m in enumerate(msgs):
+        role = m.get("role")
+        c = m.get("content") or ""
+        if role == "assistant":
+            mo = SELF_TALK_RE.search(c)
+            if mo:
+                findings.append({
+                    "cls": "self_talk_warn" if task_domain else "self_talk",
+                    "i": i,
+                    "detail": mo.group(0)[:80],
+                    "ctx": c[max(0, mo.start() - 30):
+                            mo.end() + 50].replace("\n", " ")[:160],
+                })
+        # clip markers must not survive in ANY role's content (tool
+        # rows carry them when a rewrite copied the window rendering)
+        mo = CLIP_RE.search(c)
+        if mo:
+            findings.append({
+                "cls": "clip_marker", "i": i,
+                "detail": mo.group(0)[:80], "ctx": c[
+                    max(0, mo.start() - 30):
+                    mo.end() + 50].replace("\n", " ")[:160],
+            })
+        if role == "assistant":
+            has_calls = bool(m.get("tool_calls"))
+            if not has_calls and len(c.strip()) < 20:
+                findings.append({
+                    "cls": "degenerate_turn", "i": i,
+                    "detail": f"prose turn {len(c.strip())} chars",
+                    "ctx": c.strip()[:160],
+                })
+            mo = JUDGE_LEAK_RE.search(c)
+            if mo:
+                findings.append({
+                    "cls": "judge_residue", "i": i,
+                    "detail": mo.group(0)[:80],
+                    "ctx": c[max(0, mo.start() - 30):
+                            mo.end() + 50].replace("\n", " ")[:160],
+                })
+            mo = IDENTITY_RE.search(c)
+            if mo:
+                findings.append({
+                    "cls": "identity_burn", "i": i,
+                    "detail": mo.group(0)[:80],
+                    "ctx": c[max(0, mo.start() - 30):
+                            mo.end() + 50].replace("\n", " ")[:160],
+                })
+            mo = REFUSAL_RE.search(c)
+            if mo:
+                findings.append({
+                    "cls": "refusal", "i": i,
+                    "detail": mo.group(0)[:80],
+                    "ctx": c[max(0, mo.start() - 30):
+                            mo.end() + 50].replace("\n", " ")[:160],
+                })
+    return findings
+
+
+def audit_corpus(improved_dir: str, *, max_dupes: int = 20) -> dict:
+    """Whole-corpus audit: per-trace findings + near-duplicate clusters."""
+    per_trace: dict[str, list] = {}
+    shingles: dict[str, set] = {}
+    sizes = []
+    for f in sorted(os.listdir(improved_dir)):
+        if not f.endswith(".json"):
+            continue
+        sid = f[:-5]
+        try:
+            t = json.load(open(os.path.join(improved_dir, f)))
+        except Exception as e:
+            per_trace[sid] = [{"cls": "unparseable", "i": -1,
+                               "detail": str(e)[:120], "ctx": ""}]
+            continue
+        per_trace[sid] = audit_trace(t)
+        prose = " ".join((m.get("content") or "")
+                          for m in t.get("messages", [])
+                          if m.get("role") == "assistant")
+        sh = _shingle(prose)
+        if sh:
+            shingles[sid] = sh
+        sizes.append((sid, len(json.dumps(t))))
+
+    # Near-duplicate clusters: greedy grouping against cluster reps.
+    #
+    # Comparing every trace against every cluster is O(n^2) Jaccard on sets of
+    # thousands of shingles: at 2,983 traces it ran for over 20 minutes and
+    # blocked every export. Two sets with Jaccard > 0.9 almost certainly share
+    # their smallest shingle hashes, so bucket by the 3 smallest and only
+    # compare inside a bucket. That is the standard LSH banding trick and it
+    # turns the pass into O(n) buckets with a handful of comparisons each.
+    buckets: dict[str, list[str]] = {}
+    for sid, sh in shingles.items():
+        for h in sorted(sh)[:3]:
+            buckets.setdefault(h, []).append(sid)
+
+    clusters: list[list[str]] = []
+    assigned: dict[str, list[str]] = {}
+    for a in shingles:
+        placed = False
+        # candidate clusters: any cluster whose rep shares a bucket with `a`
+        cands: list[list[str]] = []
+        seen_ids: set[int] = set()
+        for h in sorted(shingles[a])[:3]:
+            for other in buckets.get(h, ()):
+                cl = assigned.get(other)
+                if cl is not None and id(cl) not in seen_ids:
+                    seen_ids.add(id(cl))
+                    cands.append(cl)
+        for cl in cands:
+            if _jaccard(shingles[a], shingles[cl[0]]) > 0.9:
+                cl.append(a)
+                assigned[a] = cl
+                placed = True
+                break
+        if not placed:
+            cl = [a]
+            clusters.append(cl)
+            assigned[a] = cl
+    dupes = [cl for cl in clusters if len(cl) > 1][:max_dupes]
+
+    return {"per_trace": per_trace, "dupes": dupes,
+            "total": len(per_trace)}
+
+
+def main(improved_dir: str, *, verbose: bool = False) -> int:
+    rep = audit_corpus(improved_dir)
+    counts: Counter = Counter()
+    flagged = {}
+    for sid, findings in rep["per_trace"].items():
+        for f in findings:
+            counts[f["cls"]] += 1
+        if findings:
+            flagged[sid] = findings
+    print(f"audited {rep['total']} promoted traces")
+    for cls, n in sorted(counts.items()):
+        print(f"  {cls}: {n}")
+    for sid, fs in list(flagged.items())[:10 if verbose else 5]:
+        print(f"\n--- {sid}")
+        for f in fs[:4]:
+            print(f"  [{f['i']}] {f['cls']}: {f['detail']}")
+            print(f"      …{f['ctx'][:120]}…")
+    if rep["dupes"]:
+        print(f"\nnear-duplicate clusters (J>0.9): {len(rep['dupes'])}")
+        for cl in rep["dupes"][:8]:
+            print(f"  {' ~ '.join(cl)}")
+    else:
+        print("\nno near-duplicate clusters (J>0.9)")
+    return 1 if any(counts[c] for c in
+                    ("self_talk", "clip_marker", "unparseable",
+                     "judge_residue", "identity_burn", "refusal",
+                     "final_blob")) else 0
+
+
+if __name__ == "__main__":
+    import sys
+    raise SystemExit(main(sys.argv[1] if len(sys.argv) > 1
+                          else "data/traces/improved",
+                          verbose="-v" in sys.argv))

traceforge/benchmark.py

@@ -0,0 +1,811 @@
+"""Benchmark suite — score a model by how little the pipeline had to edit.
+
+The pipeline turns a real agent trace into an ideal-world reference. That
+makes it an evaluation instrument: **the amount of work the pipeline has to
+do on a model's trace is an inverse measure of how close the model already
+is to the reference.** If the pipeline has nothing to do, the model is
+already perfect.
+
+Two independent signals, both "less is better":
+
+1. **Fidelity** — similarity between the model's trace and the ideal
+   rewrite. A trace the pipeline can promote unchanged scores 1.0; every
+   dropped, rephrased or added message lowers it. Computed as a Dice
+   coefficient over aligned message pairs, so a rewrite that deletes half
+   the trace cannot score high by matching the surviving half perfectly.
+2. **Labels** — the triage pass labels every message ``keep`` / ``drop`` /
+   ``rephrase`` / ``fact_check`` / ``summarize``. ``keep`` is a positive
+   judgement, ``drop`` a negative one, and the shape edits sit in between.
+   The label score is graded (keep 1.0, shape edit 0.5, drop 0.0) and is
+   computed over ASSISTANT rows only: those are the model's own turns, while
+   a long tool result the triage wants compressed is the environment's
+   verbosity, not a model defect.
+
+Composite score (0-100)::
+
+    score = 100 * (w_sim * fidelity + w_label * label_score)
+
+Defaults are 0.6 / 0.4. Weights are a policy choice, not a measurement;
+they are recorded in every report so a score is never quoted without them.
+
+Why Dice and not a plain mean: a mean over aligned pairs rewards a rewrite
+that keeps one message and deletes nine. Dice divides by the sum of both
+lengths, so deletions and additions are edits and cost score.
+
+Normalisation: the readers prepend pipeline framing markers to user rows
+(``[trace segment 2 of 2 …]``, ``[subagent brief — role: …]``). The model
+never wrote those, so they are stripped before comparison — otherwise every
+split segment and subagent trace is penalised for a reader artifact.
+
+Caveats the report states explicitly: triage labels come from a cheap
+model and are a judgement, not ground truth; similarity measures agreement
+with the reference, not correctness of the task outcome; and a corpus
+smaller than ~30 traces cannot support a stable mean.
+"""
+from __future__ import annotations
+
+import difflib
+import json
+import math
+import os
+import re
+import time
+from collections import Counter, defaultdict
+from dataclasses import dataclass, field
+from typing import Any, Iterable, Optional
+
+from traceforge.trace import Message, Trace
+from traceforge.triage import ACTIONS
+from traceforge.validate import strip_scaffold_markers
+
+BENCHMARK_VERSION = "1.0"
+
+# Policy weights, not measurements. Recorded in every report.
+DEFAULT_WEIGHTS: dict[str, float] = {"similarity": 0.6, "labels": 0.4}
+
+# A pair at or above this is "kept verbatim" (normalised); below it the
+# pipeline rewrote the message. 0.55 is the alignment floor: two messages
+# that share less than half their text are different messages, not edits.
+VERBATIM = 0.98
+REPHRASE = 0.55
+
+_WS_RE = re.compile(r"\s+")
+
+
+# --------------------------------------------------------------- similarity
+
+
+def _norm(text: Optional[str]) -> str:
+    """Collapse whitespace and case so formatting noise is not an edit."""
+    return _WS_RE.sub(" ", (text or "").strip().lower())
+
+
+def text_similarity(a: str, b: str, *, head: int = 800, tail: int = 200) -> float:
+    """Similarity of two message texts in [0, 1].
+
+    Long tool outputs are compared on head + tail: a rewrite that preserves
+    an output preserves both ends, and running difflib over 50 KB strings is
+    quadratic for no extra signal. The length-ratio prefilter is what keeps
+    a full-corpus run under two minutes — a message half the length of its
+    counterpart is a compression, not a match.
+    """
+    a, b = _norm(a), _norm(b)
+    if a == b:
+        return 1.0
+    if not a or not b:
+        return 0.0
+    la, lb = len(a), len(b)
+    if min(la, lb) / max(la, lb) < 0.5:
+        return 0.0
+    if la > head + tail or lb > head + tail:
+        a = a[:head] + "\n…\n" + a[-tail:]
+        b = b[:head] + "\n…\n" + b[-tail:]
+    sm = difflib.SequenceMatcher(None, a, b, autojunk=False)
+    # quick_ratio is an upper bound on ratio; below the alignment floor the
+    # exact value cannot change any decision, so skip the quadratic pass.
+    if sm.quick_ratio() < 0.4:
+        return 0.0
+    return sm.ratio()
+
+
+def _msg_text(m: Message) -> str:
+    """Everything a message carries: prose plus its tool calls."""
+    parts = [m.content or ""]
+    for tc in m.tool_calls:
+        parts.append(f"{tc.name} {tc.arguments}")
+    return "\n".join(p for p in parts if p)
+
+
+def _call_sig(tc) -> tuple[str, str]:
+    return (tc.name, _norm(tc.arguments)[:300])
+
+
+def _seq_similarity(a: list, b: list) -> float:
+    """Dice coefficient over two sequences (1.0 when both are empty)."""
+    if not a and not b:
+        return 1.0
+    if not a or not b:
+        return 0.0
+    sm = difflib.SequenceMatcher(None, a, b, autojunk=False)
+    return sm.ratio()
+
+
+def message_similarity(raw_m: Message, imp_m: Message) -> float:
+    """Similarity of one aligned message pair.
+
+    A tool-only assistant turn carries no prose, so its calls ARE the
+    message; weighting prose and calls equally would score every such pair
+    as a perfect match on the empty string.
+    """
+    if raw_m.role != imp_m.role:
+        return 0.0
+    text_sim = text_similarity(_msg_text(raw_m), _msg_text(imp_m))
+    raw_calls = [_call_sig(tc) for tc in raw_m.tool_calls]
+    imp_calls = [_call_sig(tc) for tc in imp_m.tool_calls]
+    if raw_calls or imp_calls:
+        call_sim = _seq_similarity(raw_calls, imp_calls)
+        if not _norm(raw_m.content) and not _norm(imp_m.content):
+            return call_sim
+        return 0.5 * text_sim + 0.5 * call_sim
+    return text_sim
+
+
+def align_messages(raw_msgs: list[Message],
+                   imp_msgs: list[Message]) -> list[tuple[int, int]]:
+    """Monotonic alignment of raw -> improved messages.
+
+    Roles are the stable skeleton: a rewrite keeps the user/assistant/tool
+    shape and drops or adds whole turns. SequenceMatcher over the role
+    sequences finds the equal runs and the replaced blocks; inside a block
+    messages pair by position. That is O(n) pairs instead of the O(n·window)
+    of a content-driven search, and it is the right model of the edit: the
+    pipeline does not reorder turns, it removes and rewrites them.
+    """
+    raw_roles = [m.role for m in raw_msgs]
+    imp_roles = [m.role for m in imp_msgs]
+    sm = difflib.SequenceMatcher(None, raw_roles, imp_roles, autojunk=False)
+    pairs: list[tuple[int, int]] = []
+    for tag, i1, i2, j1, j2 in sm.get_opcodes():
+        if tag in ("equal", "replace"):
+            for k in range(min(i2 - i1, j2 - j1)):
+                pairs.append((i1 + k, j1 + k))
+    return pairs
+
+
+# ------------------------------------------------------------------- labels
+
+
+def label_stats(triage: Optional[dict],
+                messages: Optional[list[Message]] = None) -> dict[str, Any]:
+    """Counts and rates over one triage label file.
+
+    ``keep`` is the positive judgement; ``drop`` the negative one.
+    ``rephrase``/``summarize`` are shape edits (right content, wrong form),
+    ``fact_check`` is a verification flag — neither is a negative label, so
+    they are reported separately instead of being folded into the keep rate.
+
+    When ``messages`` is given, the same counts are also computed over
+    ASSISTANT rows only. That is the number the composite uses: the model
+    wrote its own turns, not the tool outputs, and a long tool result the
+    triage wants compressed is the environment's verbosity, not a model
+    defect.
+    """
+    labels = (triage or {}).get("labels") or []
+    valid = [lab for lab in labels
+             if isinstance(lab, dict) and lab.get("action") in ACTIONS]
+    counts = Counter(lab["action"] for lab in valid)
+    total = sum(counts.values())
+    out: dict[str, Any] = {"total": total}
+    for action in ACTIONS:
+        out[action] = counts.get(action, 0)
+    out["keep_rate"] = (counts.get("keep", 0) / total) if total else None
+    out["negative_rate"] = (counts.get("drop", 0) / total) if total else None
+    out["edit_rate"] = (
+        (counts.get("rephrase", 0) + counts.get("summarize", 0)) / total
+        if total else None
+    )
+    # Graded positive/negative score: keep = 1.0, a shape edit (rephrase /
+    # summarize / fact_check) = 0.5, drop = 0.0. The strict keep rate is
+    # dominated by the triage model's compression preference — 58% of
+    # assistant labels on the real corpus are `summarize` because assistant
+    # turns carry large tool-call arguments — so the composite uses the
+    # graded score and the report shows both.
+    out["label_score"] = (
+        (counts.get("keep", 0)
+         + 0.5 * (counts.get("rephrase", 0) + counts.get("summarize", 0)
+                  + counts.get("fact_check", 0))) / total
+        if total else None
+    )
+    if messages is not None:
+        roles = {i: m.role for i, m in enumerate(messages)}
+        a_counts = Counter(
+            lab["action"] for lab in valid
+            if isinstance(lab.get("i"), int)
+            and roles.get(lab["i"]) == "assistant"
+        )
+        a_total = sum(a_counts.values())
+        out["assistant_total"] = a_total
+        for action in ACTIONS:
+            out[f"assistant_{action}"] = a_counts.get(action, 0)
+        out["assistant_keep_rate"] = (
+            a_counts.get("keep", 0) / a_total if a_total else None)
+        out["assistant_negative_rate"] = (
+            a_counts.get("drop", 0) / a_total if a_total else None)
+        out["assistant_label_score"] = (
+            (a_counts.get("keep", 0)
+             + 0.5 * (a_counts.get("rephrase", 0) + a_counts.get("summarize", 0)
+                      + a_counts.get("fact_check", 0))) / a_total
+            if a_total else None
+        )
+    return out
+
+
+def load_triage(path: str) -> Optional[dict]:
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            return json.load(f)
+    except (OSError, ValueError):
+        return None
+
+
+# ------------------------------------------------------------------ scoring
+
+
+@dataclass
+class TraceScore:
+    """One model trace scored against its ideal rewrite."""
+
+    session_id: str
+    task: Optional[str] = None
+    source: Optional[str] = None
+    model: Optional[str] = None
+
+    # fidelity
+    similarity: float = 0.0
+    tool_similarity: float = 0.0
+    kept_verbatim: int = 0
+    rephrased: int = 0
+    dropped: int = 0
+    added: int = 0
+    raw_messages: int = 0
+    improved_messages: int = 0
+    raw_chars: int = 0
+
+    # labels
+    labels_total: int = 0
+    keep: int = 0
+    drop: int = 0
+    rephrase: int = 0
+    fact_check: int = 0
+    summarize: int = 0
+    keep_rate: Optional[float] = None
+    negative_rate: Optional[float] = None
+    label_score: Optional[float] = None
+    # assistant-only: the model's own turns (the loss-bearing target)
+    assistant_labels_total: int = 0
+    assistant_keep: int = 0
+    assistant_drop: int = 0
+    assistant_keep_rate: Optional[float] = None
+    assistant_negative_rate: Optional[float] = None
+    assistant_label_score: Optional[float] = None
+
+    # gates (best effort — only traces with a manifest attempts row have it)
+    attempts: Optional[int] = None
+    first_pass: Optional[bool] = None
+
+    # composite
+    score: float = 0.0
+    pipeline_idle: bool = False
+
+    @property
+    def edits(self) -> int:
+        return self.dropped + self.added + self.rephrased
+
+    def to_dict(self) -> dict[str, Any]:
+        d = {
+            "session_id": self.session_id,
+            "task": self.task,
+            "source": self.source,
+            "model": self.model,
+            "score": round(self.score, 2),
+            "similarity": round(self.similarity, 4),
+            "tool_similarity": round(self.tool_similarity, 4),
+            "keep_rate": (None if self.keep_rate is None
+                          else round(self.keep_rate, 4)),
+            "negative_rate": (None if self.negative_rate is None
+                              else round(self.negative_rate, 4)),
+            "assistant_keep_rate": (None if self.assistant_keep_rate is None
+                                    else round(self.assistant_keep_rate, 4)),
+            "assistant_negative_rate": (
+                None if self.assistant_negative_rate is None
+                else round(self.assistant_negative_rate, 4)),
+            "label_score": (None if self.label_score is None
+                            else round(self.label_score, 4)),
+            "assistant_label_score": (
+                None if self.assistant_label_score is None
+                else round(self.assistant_label_score, 4)),
+            "edits": {
+                "kept_verbatim": self.kept_verbatim,
+                "rephrased": self.rephrased,
+                "dropped": self.dropped,
+                "added": self.added,
+                "total": self.edits,
+            },
+            "messages": {
+                "raw": self.raw_messages,
+                "improved": self.improved_messages,
+                "raw_chars": self.raw_chars,
+            },
+            "labels": {
+                "total": self.labels_total,
+                "keep": self.keep,
+                "drop": self.drop,
+                "rephrase": self.rephrase,
+                "fact_check": self.fact_check,
+                "summarize": self.summarize,
+            },
+            "attempts": self.attempts,
+            "first_pass": self.first_pass,
+            "pipeline_idle": self.pipeline_idle,
+        }
+        return d
+
+
+def score_pair(raw: Trace, improved: Trace, *, triage: Optional[dict] = None,
+               attempts: Optional[int] = None,
+               weights: Optional[dict[str, float]] = None) -> TraceScore:
+    """Score one (model trace, ideal rewrite) pair.
+
+    ``raw`` is mutated in place by the scaffold strip — pass a freshly
+    loaded trace, which is what the CLI does.
+    """
+    w = {**DEFAULT_WEIGHTS, **(weights or {})}
+    strip_scaffold_markers(raw)
+
+    pairs = align_messages(raw.messages, improved.messages)
+    sims = [message_similarity(raw.messages[i], improved.messages[j])
+            for i, j in pairs]
+    n_raw, n_imp = len(raw.messages), len(improved.messages)
+    similarity = (2.0 * sum(sims) / (n_raw + n_imp)) if (n_raw + n_imp) else 1.0
+
+    kept = sum(1 for s in sims if s >= VERBATIM)
+    rephrased = sum(1 for s in sims if REPHRASE <= s < VERBATIM)
+    dropped = n_raw - len(pairs)
+    added = n_imp - len(pairs)
+
+    tool_similarity = _seq_similarity(
+        [_call_sig(tc) for m in raw.messages for tc in m.tool_calls],
+        [_call_sig(tc) for m in improved.messages for tc in m.tool_calls],
+    )
+
+    lab = label_stats(triage, raw.messages)
+    keep_rate = lab["keep_rate"]
+    # The composite judges the model on its OWN turns. Tool outputs are
+    # context the model did not write; a long result the triage wants
+    # compressed is the environment's verbosity, not a model defect.
+    label_score = (lab.get("assistant_label_score")
+                   if lab.get("assistant_label_score") is not None
+                   else lab.get("label_score"))
+    # Idle is strict: any non-keep label means the pipeline had work to do.
+    strict_keep = (lab.get("assistant_keep_rate")
+                   if lab.get("assistant_keep_rate") is not None else keep_rate)
+
+    # Composite: renormalise over the components that exist. A trace with no
+    # triage file is scored on fidelity alone rather than being penalised
+    # for a missing label pass.
+    parts = [(w["similarity"], similarity)]
+    if label_score is not None:
+        parts.append((w["labels"], label_score))
+    total_w = sum(p[0] for p in parts)
+    score = (100.0 * sum(p[0] * p[1] for p in parts) / total_w) if total_w else 0.0
+
+    idle = (similarity >= 0.999 and dropped == 0 and added == 0
+            and (strict_keep is None or strict_keep >= 0.999))
+
+    return TraceScore(
+        session_id=raw.session_id,
+        task=improved.task or raw.task,
+        source=raw.source,
+        model=improved.model or raw.model,
+        similarity=similarity,
+        tool_similarity=tool_similarity,
+        kept_verbatim=kept,
+        rephrased=rephrased,
+        dropped=dropped,
+        added=added,
+        raw_messages=n_raw,
+        improved_messages=n_imp,
+        raw_chars=sum(len(m.content or "") for m in raw.messages),
+        labels_total=lab["total"],
+        keep=lab["keep"],
+        drop=lab["drop"],
+        rephrase=lab["rephrase"],
+        fact_check=lab["fact_check"],
+        summarize=lab["summarize"],
+        keep_rate=keep_rate,
+        negative_rate=lab["negative_rate"],
+        label_score=lab.get("label_score"),
+        assistant_labels_total=lab.get("assistant_total", 0),
+        assistant_keep=lab.get("assistant_keep", 0),
+        assistant_drop=lab.get("assistant_drop", 0),
+        assistant_keep_rate=lab.get("assistant_keep_rate"),
+        assistant_negative_rate=lab.get("assistant_negative_rate"),
+        assistant_label_score=lab.get("assistant_label_score"),
+        attempts=attempts,
+        first_pass=(attempts == 1) if attempts is not None else None,
+        score=score,
+        pipeline_idle=idle,
+    )
+
+
+# --------------------------------------------------------------- statistics
+
+
+def _dist(vals: Iterable[float]) -> dict[str, Any]:
+    """Distribution summary. Percentiles are nearest-rank, not interpolated."""
+    v = sorted(float(x) for x in vals)
+    if not v:
+        return {"n": 0, "mean": None, "median": None, "p10": None,
+                "p90": None, "min": None, "max": None}
+
+    def pct(q: float) -> float:
+        idx = min(len(v) - 1, max(0, int(round(q * (len(v) - 1)))))
+        return v[idx]
+
+    return {
+        "n": len(v),
+        "mean": round(sum(v) / len(v), 4),
+        "median": round(pct(0.5), 4),
+        "p10": round(pct(0.1), 4),
+        "p90": round(pct(0.9), 4),
+        "min": round(v[0], 4),
+        "max": round(v[-1], 4),
+    }
+
+
+def _pearson(xs: list[float], ys: list[float]) -> Optional[float]:
+    n = len(xs)
+    if n < 3 or n != len(ys):
+        return None
+    mx, my = sum(xs) / n, sum(ys) / n
+    num = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
+    dx = math.sqrt(sum((x - mx) ** 2 for x in xs))
+    dy = math.sqrt(sum((y - my) ** 2 for y in ys))
+    if dx == 0 or dy == 0:
+        return None
+    return round(num / (dx * dy), 3)
+
+
+def _histogram(vals: list[float], edges: list[float]) -> dict[str, int]:
+    """Bucket counts; the last bucket is inclusive of its upper edge."""
+    out: dict[str, int] = {}
+    last = len(edges) - 2
+    for i, (lo, hi) in enumerate(zip(edges, edges[1:])):
+        out[f"{lo:g}-{hi:g}"] = sum(
+            1 for v in vals if lo <= v < hi or (i == last and v == hi))
+    return out
+
+
+def _group(scores: list[TraceScore], key) -> dict[str, dict[str, Any]]:
+    buckets: dict[str, list[TraceScore]] = defaultdict(list)
+    for s in scores:
+        buckets[key(s) or "unknown"].append(s)
+    return {
+        k: {"n": len(v), "score_mean": round(sum(x.score for x in v) / len(v), 2),
+            "similarity_mean": round(sum(x.similarity for x in v) / len(v), 3),
+            "keep_rate_mean": (
+                round(sum(x.keep_rate for x in v if x.keep_rate is not None)
+                      / max(1, sum(1 for x in v if x.keep_rate is not None)), 3)
+                if any(x.keep_rate is not None for x in v) else None)}
+        for k, v in sorted(buckets.items(), key=lambda kv: -len(kv[1]))
+    }
+
+
+def _size_band(chars: int) -> str:
+    kb = chars / 1000.0
+    if kb < 60:
+        return "<60KB"
+    if kb < 120:
+        return "60-120KB"
+    if kb < 180:
+        return "120-180KB"
+    if kb < 240:
+        return "180-240KB"
+    if kb < 300:
+        return "240-300KB"
+    return "300KB+"
+
+
+def build_report(scores: list[TraceScore], *, label: Optional[str] = None,
+                 weights: Optional[dict[str, float]] = None,
+                 corpus: Optional[dict[str, Any]] = None,
+                 generated_at: Optional[str] = None) -> dict[str, Any]:
+    """Corpus-level statistics over a set of scored trace pairs."""
+    w = {**DEFAULT_WEIGHTS, **(weights or {})}
+    n = len(scores)
+    sims = [s.similarity for s in scores]
+    keeps = [s.keep_rate for s in scores if s.keep_rate is not None]
+    a_keeps = [s.assistant_keep_rate for s in scores
+               if s.assistant_keep_rate is not None]
+    a_scores = [s.assistant_label_score for s in scores
+                if s.assistant_label_score is not None]
+    negs = [s.negative_rate for s in scores if s.negative_rate is not None]
+    a_negs = [s.assistant_negative_rate for s in scores
+              if s.assistant_negative_rate is not None]
+    comp = [s.score for s in scores]
+    edits = [s.edits for s in scores]
+    tools = [s.tool_similarity for s in scores]
+
+    label_totals = Counter()
+    for s in scores:
+        for action in ACTIONS:
+            label_totals[action] += getattr(s, action)
+    label_total = sum(label_totals.values())
+
+    edit_totals = {
+        "kept_verbatim": sum(s.kept_verbatim for s in scores),
+        "rephrased": sum(s.rephrased for s in scores),
+        "dropped": sum(s.dropped for s in scores),
+        "added": sum(s.added for s in scores),
+    }
+    edit_total = sum(edit_totals.values()) or 1
+
+    idle = [s for s in scores if s.pipeline_idle]
+    gated = [s for s in scores if s.attempts is not None]
+    first_pass = [s for s in gated if s.first_pass]
+
+    # Fidelity and labels are independent instruments; where they disagree
+    # the pipeline is inconsistent (a trace promoted unchanged although the
+    # triage wanted edits, or a rewrite of turns the triage called clean).
+    # That is a pipeline diagnostic, not a model defect — report it.
+    def _lab(s: TraceScore) -> float:
+        return s.assistant_label_score if s.assistant_label_score is not None else 0.0
+
+    fid_perfect = [s for s in scores if s.similarity >= 0.999]
+    lab_perfect = [s for s in scores if _lab(s) >= 0.999]
+    both_perfect = [s for s in scores
+                    if s.similarity >= 0.999 and _lab(s) >= 0.999]
+
+    corr = _pearson(
+        [s.similarity for s in scores if s.assistant_label_score is not None],
+        [s.assistant_label_score for s in scores
+         if s.assistant_label_score is not None],
+    )
+
+    worst = sorted(scores, key=lambda s: s.score)[:10]
+    best = sorted(scores, key=lambda s: -s.score)[:10]
+
+    report: dict[str, Any] = {
+        "benchmark_version": BENCHMARK_VERSION,
+        "generated_at": generated_at or time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+        "label": label,
+        "weights": w,
+        "corpus": {
+            "scored": n,
+            "with_labels": len(keeps),
+            "with_gates": len(gated),
+            **(corpus or {}),
+        },
+        "score": {
+            **_dist(comp),
+            "histogram": _histogram(comp, [0, 20, 40, 60, 80, 100]),
+        },
+        "components": {
+            "similarity": _dist(sims),
+            "keep_rate": _dist(keeps),
+            "assistant_keep_rate": _dist(a_keeps),
+            "label_score": _dist(a_scores),
+            "negative_rate": _dist(negs),
+            "assistant_negative_rate": _dist(a_negs),
+            "tool_sequence_similarity": _dist(tools),
+        },
+        "edits": {
+            "totals": edit_totals,
+            "share_pct": {k: round(100.0 * v / edit_total, 1)
+                          for k, v in edit_totals.items()},
+            "per_trace": _dist(edits),
+        },
+        "labels": {
+            "totals": dict(label_totals),
+            "total": label_total,
+            "share_pct": {k: round(100.0 * v / label_total, 1)
+                          for k, v in label_totals.items()} if label_total else {},
+        },
+        "pipeline_idle": {
+            "traces": len(idle),
+            "pct": round(100.0 * len(idle) / n, 1) if n else 0.0,
+            "definition": "similarity >= 0.999, no dropped/added message, "
+                          "keep_rate >= 0.999 — the pipeline had nothing to do",
+        },
+        "agreement": {
+            "both_perfect": len(both_perfect),
+            "fidelity_perfect_labels_edit": len(fid_perfect) - len(both_perfect),
+            "labels_perfect_fidelity_edit": len(lab_perfect) - len(both_perfect),
+            "note": "fidelity and labels are independent instruments; a "
+                    "disagreement is a pipeline inconsistency (promoted "
+                    "unchanged although triage wanted edits, or rewritten "
+                    "although triage called it clean), not a model defect",
+        },
+        "gates": {
+            "traces_with_attempts": len(gated),
+            "first_pass": len(first_pass),
+            "first_pass_pct": (round(100.0 * len(first_pass) / len(gated), 1)
+                               if gated else None),
+            "attempts_mean": (round(sum(s.attempts for s in gated) / len(gated), 2)
+                              if gated else None),
+        },
+        "by_task": _group(scores, lambda s: s.task),
+        "by_source": _group(scores, lambda s: s.source),
+        "by_size": _group(scores, lambda s: _size_band(s.raw_chars)),
+        "correlation": {"similarity_vs_label_score": corr},
+        "worst": [s.to_dict() for s in worst],
+        "best": [s.to_dict() for s in best],
+        "notes": [
+            "score = 100 * (w_similarity * fidelity + w_labels * label_score); "
+            "weights are policy, not measurement",
+            "fidelity is a Dice coefficient over aligned message pairs, so "
+            "dropped and added messages both cost score",
+            "label_score is graded: keep=1.0, rephrase/summarize/fact_check=0.5, "
+            "drop=0.0, over ASSISTANT rows only (the model's own turns)",
+            "triage labels come from a cheap model and are a judgement, not "
+            "ground truth; label_score is agreement with the pipeline's judge",
+            "similarity measures agreement with the reference, not whether "
+            "the task outcome was correct",
+            "pipeline framing markers are stripped from raw user rows before "
+            "comparison (the model never wrote them)",
+            "n < 30 traces cannot support a stable mean — treat the score as "
+            "a smoke signal, not a ranking",
+        ],
+    }
+    return report
+
+
+# ------------------------------------------------------------------ render
+
+
+def _fmt(v: Optional[float], nd: int = 2) -> str:
+    return "—" if v is None else f"{v:.{nd}f}"
+
+
+def render_report(report: dict[str, Any]) -> str:
+    """Human-readable report. The JSON is the artifact; this is the glance."""
+    c = report["corpus"]
+    sc = report["score"]
+    comp = report["components"]
+    lines: list[str] = []
+    title = f"traceforge benchmark v{report['benchmark_version']}"
+    if report.get("label"):
+        title += f" — {report['label']}"
+    lines.append(title)
+    lines.append(
+        f"scored {c['scored']} trace pairs (model trace -> ideal rewrite) · "
+        f"{c['with_labels']} with triage labels · "
+        f"{c['with_gates']} with gate attempts"
+    )
+    lines.append("")
+    lines.append(f"score (0-100)      mean {_fmt(sc['mean'], 1)}   "
+                 f"median {_fmt(sc['median'], 1)}   "
+                 f"p10 {_fmt(sc['p10'], 1)}   p90 {_fmt(sc['p90'], 1)}")
+    lines.append(f"  fidelity         mean {_fmt(comp['similarity']['mean'], 3)}   "
+                 f"median {_fmt(comp['similarity']['median'], 3)}   "
+                 f"p10 {_fmt(comp['similarity']['p10'], 3)}   "
+                 f"p90 {_fmt(comp['similarity']['p90'], 3)}")
+    lines.append(f"  label score      mean {_fmt(comp['label_score']['mean'], 3)}   "
+                 f"median {_fmt(comp['label_score']['median'], 3)}   "
+                 f"p10 {_fmt(comp['label_score']['p10'], 3)}   "
+                 f"p90 {_fmt(comp['label_score']['p90'], 3)}")
+    lines.append(f"  keep rate (asst) mean {_fmt(comp['assistant_keep_rate']['mean'], 3)}   "
+                 f"median {_fmt(comp['assistant_keep_rate']['median'], 3)}   "
+                 f"p10 {_fmt(comp['assistant_keep_rate']['p10'], 3)}   "
+                 f"p90 {_fmt(comp['assistant_keep_rate']['p90'], 3)}")
+    lines.append(f"  negative (asst)  mean {_fmt(comp['assistant_negative_rate']['mean'], 3)}   "
+                 f"median {_fmt(comp['assistant_negative_rate']['median'], 3)}   "
+                 f"p90 {_fmt(comp['assistant_negative_rate']['p90'], 3)}")
+    lines.append(f"  tool sequence    mean {_fmt(comp['tool_sequence_similarity']['mean'], 3)}")
+    lines.append("")
+    hist = sc["histogram"]
+    lines.append("score histogram    " + " · ".join(
+        f"{k}: {v}" for k, v in hist.items()))
+    lines.append("")
+    e = report["edits"]
+    lines.append(f"edits per trace    mean {_fmt(e['per_trace']['mean'], 1)}   "
+                 f"median {_fmt(e['per_trace']['median'], 1)}   "
+                 f"p90 {_fmt(e['per_trace']['p90'], 1)}")
+    lines.append(
+        "  messages         "
+        f"kept {e['totals']['kept_verbatim']} ({e['share_pct']['kept_verbatim']}%) · "
+        f"rephrased {e['totals']['rephrased']} ({e['share_pct']['rephrased']}%) · "
+        f"dropped {e['totals']['dropped']} ({e['share_pct']['dropped']}%) · "
+        f"added {e['totals']['added']} ({e['share_pct']['added']}%)")
+    lab = report["labels"]
+    if lab["total"]:
+        lines.append("labels             " + " · ".join(
+            f"{k} {v}" for k, v in lab["totals"].items()))
+    idle = report["pipeline_idle"]
+    lines.append(f"pipeline idle      {idle['traces']} traces "
+                 f"({idle['pct']}%) — nothing to edit")
+    ag = report.get("agreement") or {}
+    if ag:
+        lines.append(f"instrument agree   both perfect {ag['both_perfect']} · "
+                     f"fidelity-only {ag['fidelity_perfect_labels_edit']} · "
+                     f"labels-only {ag['labels_perfect_fidelity_edit']}")
+    g = report["gates"]
+    if g["traces_with_attempts"]:
+        lines.append(f"first-pass gates   {g['first_pass']}/"
+                     f"{g['traces_with_attempts']} ({g['first_pass_pct']}%) · "
+                     f"mean attempts {g['attempts_mean']}")
+    lines.append("")
+    for name, key in (("by task", "by_task"), ("by source", "by_source"),
+                      ("by size", "by_size")):
+        groups = report[key]
+        if not groups:
+            continue
+        parts = [f"{k} {v['n']} ({_fmt(v['score_mean'], 1)})"
+                 for k, v in list(groups.items())[:8]]
+        lines.append(f"{name:<18} " + " · ".join(parts))
+    corr = report["correlation"]["similarity_vs_label_score"]
+    lines.append(f"correlation        similarity vs label score: r={_fmt(corr, 3)}")
+    lines.append("")
+    lines.append("worst traces (inspect these first):")
+    for s in report["worst"][:5]:
+        lines.append(f"  {s['session_id'][:44]:<44} score {s['score']:>5.1f}  "
+                     f"sim {s['similarity']:.2f}  "
+                     f"label {_fmt(s['assistant_label_score'], 2)}  "
+                     f"{s['task'] or '—'}")
+    return "\n".join(lines)
+
+
+# -------------------------------------------------------------- leaderboard
+
+
+def append_leaderboard(path: str, report: dict[str, Any]) -> None:
+    """Append one compact row per run; the leaderboard keeps the latest per label."""
+    os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True)
+    row = {
+        "label": report.get("label") or "unlabeled",
+        "generated_at": report["generated_at"],
+        "benchmark_version": report["benchmark_version"],
+        "n": report["corpus"]["scored"],
+        "score_mean": report["score"]["mean"],
+        "score_median": report["score"]["median"],
+        "similarity_mean": report["components"]["similarity"]["mean"],
+        "label_score_mean": report["components"]["label_score"]["mean"],
+        "keep_rate_mean": report["components"]["assistant_keep_rate"]["mean"],
+        "pipeline_idle_pct": report["pipeline_idle"]["pct"],
+        "weights": report["weights"],
+    }
+    with open(path, "a", encoding="utf-8") as f:
+        f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def load_leaderboard(path: str) -> list[dict[str, Any]]:
+    rows: dict[str, dict[str, Any]] = {}
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            for line in f:
+                try:
+                    r = json.loads(line)
+                except ValueError:
+                    continue
+                if r.get("label"):
+                    rows[r["label"]] = r  # latest run wins
+    except OSError:
+        return []
+    return sorted(rows.values(), key=lambda r: -(r.get("score_mean") or 0))
+
+
+def render_leaderboard(rows: list[dict[str, Any]]) -> str:
+    if not rows:
+        return "leaderboard: empty"
+    lines = ["leaderboard (latest run per label, sorted by mean score):",
+             f"  {'label':<28} {'n':>5} {'score':>7} {'fidelity':>9} "
+             f"{'label':>6} {'idle%':>6}  when"]
+    for r in rows:
+        lines.append(
+            f"  {r['label'][:28]:<28} {r['n']:>5} "
+            f"{_fmt(r.get('score_mean'), 1):>7} "
+            f"{_fmt(r.get('similarity_mean'), 3):>9} "
+            f"{_fmt(r.get('label_score_mean'), 3):>6} "
+            f"{_fmt(r.get('pipeline_idle_pct'), 1):>6}  "
+            f"{r.get('generated_at', '')[:19]}")
+    return "\n".join(lines)

traceforge/chunker.py

@@ -0,0 +1,90 @@
+"""Splits a Trace into LLM-digestible chunks at message boundaries.
+
+A 10M-token session can never be rendered into a single prompt. Chunks are
+cut at message boundaries under a char budget (≈4 chars/token), with a
+small tail-overlap of previous-chunk messages prepended as continuity
+context for the digester.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from traceforge.trace import Trace
+
+
+@dataclass
+class Chunk:
+    index: int          # 1-based
+    start: int          # first message index (0-based, inclusive)
+    end: int            # last message index (exclusive)
+    rendered: str       # compact rendering incl. overlap context header
+    approx_chars: int   # size estimate of the chunk's own messages
+
+
+def _msg_size(m) -> int:
+    return len(m.content or "") + sum(
+        len(tc.arguments) for tc in m.tool_calls
+    ) + 64
+
+
+def chunk_trace(
+    trace: Trace,
+    *,
+    max_chars: int = 100_000,
+    overlap_messages: int = 2,
+    render_kwargs: dict | None = None,
+) -> list[Chunk]:
+    """Cut the trace into chunks of ≤ max_chars (single messages never split).
+
+    Returns chunks with per-chunk overlap context prepended (marked as
+    continuity-only so the digester does not double-count).
+    """
+    rk = render_kwargs or {}
+    msgs = trace.messages
+    n = len(msgs)
+    chunks: list[Chunk] = []
+
+    def render_slice(lo: int, hi: int) -> str:
+        sub = Trace(
+            session_id=trace.session_id,
+            model=trace.model,
+            messages=msgs[lo:hi],
+        )
+        out = sub.render_compact(**rk)
+        # drop the synthetic "=== TRACE ... ===" header line
+        return out.split("\n", 1)[1] if "\n" in out else out
+
+    i = 0
+    while i < n:
+        j = i
+        total = 0
+        while j < n and (j == i or total + _msg_size(msgs[j]) <= max_chars):
+            total += _msg_size(msgs[j])
+            j += 1
+        body = render_slice(i, j)
+        header = (
+            f"CHUNK {len(chunks) + 1} of ~N (messages {i + 1}-{j} of {n})"
+        )
+        block = f"{header}\n{body}"
+        if i > 0 and overlap_messages > 0:
+            ov = render_slice(max(0, i - overlap_messages), i)
+            block = (
+                "CONTEXT — tail of the previous chunk (continuity only;"
+                " already digested, do not re-report):\n"
+                f"{ov}\n\n{block}"
+            )
+        chunks.append(
+            Chunk(
+                index=len(chunks) + 1,
+                start=i,
+                end=j,
+                rendered=block,
+                approx_chars=total,
+            )
+        )
+        i = j
+    # fix the "of ~N" placeholders now that we know the count
+    for c in chunks:
+        if "~N" in c.rendered:
+            c.rendered = c.rendered.replace("~N", str(len(chunks)))
+    return chunks

traceforge/classify.py

@@ -0,0 +1,394 @@
+"""Task/topic classification for traces.
+
+Labels every trace with one task bucket so export can split the corpus by
+topic (``train.coding.jsonl``, ``train.research.jsonl``, ...). Two tiers:
+
+  heuristic  tool signature + first-task-message keywords. Free,
+             deterministic, whole corpus in seconds. Confidence is the
+             margin between the winning bucket and the runner-up.
+  llm        one structured call per trace, used only where the heuristic
+             is unsure (``--llm``). Same endpoint/grammar ladder as
+             triage.py, so no new infrastructure.
+
+The label is persisted on the trace (``Trace.task``) so export is
+deterministic and every assignment is inspectable and reviewable.
+
+Taxonomy is deliberately closed: a small set the trainer can weight
+against, not free-form tags. Coding is split because the corpus is ~95%
+coding and a single bucket would be useless for balancing.
+"""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any, Optional
+
+from traceforge.trace import Trace
+
+# --------------------------------------------------------------- taxonomy
+
+CODING_TASKS = ("debug", "implement", "refactor", "test", "review")
+NONCODING_TASKS = (
+    "research", "data-analysis", "office-docs", "creative", "ops", "other",
+)
+TASKS = CODING_TASKS + NONCODING_TASKS
+
+TASK_DESCRIPTIONS = {
+    "debug": "diagnose and fix a failure: error, traceback, crash, regression",
+    "implement": "add or build a feature, file, endpoint, or capability",
+    "refactor": "restructure existing code without changing behaviour",
+    "test": "write or run tests, coverage, fixtures, CI test jobs",
+    "review": "read-only audit, code review, diff inspection, critique",
+    "research": "gather external information: web, papers, docs, comparison",
+    "data-analysis": "analyse data: metrics, SQL, CSV, statistics, dashboards",
+    "office-docs": "produce office-suite deliverables: .xlsx spreadsheets, "
+                   ".pptx decks, .docx documents (NOT markdown docs or code docs)",
+    "creative": "creative writing or design: prose, lyrics, branding, visuals",
+    "ops": "deploy, configure, or operate infrastructure and services",
+    "other": "none of the above, or too ambiguous to call",
+}
+
+# ------------------------------------------------------------- heuristics
+
+# Keyword rules over the first task message. Weight = how diagnostic the
+# term is; a single strong term ("traceback") should beat several weak ones.
+_KEYWORDS: dict[str, list[tuple[str, int]]] = {
+    "debug": [
+        (r"\btraceback\b", 4), (r"\bstack ?trace\b", 4),
+        (r"\bsegfault\b", 4), (r"\bpanic\b", 3), (r"\bexception\b", 3),
+        (r"\bregression\b", 3), (r"\bcrash(es|ed|ing)?\b", 3),
+        (r"\bbug\b", 3), (r"\bbroken\b", 3), (r"\bfail(s|ed|ing)?\b", 2),
+        (r"\berror\b", 2), (r"\bfix\b", 2), (r"\bwhy (does|is|did|do)\b", 2),
+        (r"\bdebug\b", 3), (r"\bhang(s|ing)?\b", 2), (r"\btimeout\b", 2),
+    ],
+    "test": [
+        (r"\bpytest\b", 4), (r"\bunittest\b", 4), (r"\bcoverage\b", 3),
+        (r"\bfixture\b", 3), (r"\bmock(s|ed|ing)?\b", 2),
+        (r"\bassert(ion)?s?\b", 2), (r"\be2e\b", 3),
+        (r"\bintegration test", 3), (r"\btest suite\b", 3),
+        (r"\bwrite (a |the )?tests?\b", 3), (r"\btests?\b", 1),
+    ],
+    "refactor": [
+        (r"\brefactor\b", 5), (r"\brestructur", 4), (r"\brename\b", 3),
+        (r"\bextract (a |the )?(function|method|class|module)", 4),
+        (r"\bdedup", 3), (r"\bsimplif", 3), (r"\bclean ?up\b", 2),
+        (r"\bmove .{0,30} into\b", 2), (r"\bsplit .{0,30} into\b", 2),
+        (r"\btidy\b", 2), (r"\bconsolidat", 3),
+    ],
+    "review": [
+        (r"\bcode review\b", 5), (r"\breview\b", 3), (r"\baudit\b", 4),
+        (r"\binspect\b", 3), (r"\bcritique\b", 3), (r"\bpull request\b", 3),
+        (r"\bdiff\b", 2), (r"\bcheck .{0,25} for\b", 2),
+        (r"\bread[- ]only\b", 3), (r"\bassess\b", 2),
+    ],
+    "implement": [
+        (r"\bimplement\b", 4), (r"\badd (a |an |the )?", 2),
+        (r"\bbuild\b", 2), (r"\bcreate\b", 2), (r"\bwrite (a |an |the )?", 2),
+        (r"\bfeature\b", 3), (r"\bsupport for\b", 3), (r"\bintroduce\b", 2),
+        (r"\bwire up\b", 3), (r"\bhook up\b", 3), (r"\bendpoint\b", 2),
+        (r"\bmake .{0,25} work\b", 2), (r"\bscaffold\b", 3),
+    ],
+    "research": [
+        (r"\bresearch\b", 4), (r"\binvestigate\b", 3), (r"\bcompare\b", 3),
+        (r"\bsurvey\b", 3), (r"\barxiv\b", 5), (r"\bpapers?\b", 3),
+        (r"\bliterature\b", 4), (r"\bfind out\b", 3), (r"\blook up\b", 3),
+        (r"\bwhat is\b", 2), (r"\bhow does\b", 2), (r"\bexplain\b", 2),
+        (r"\bsources?\b", 2), (r"\bstate of the art\b", 4),
+    ],
+    "data-analysis": [
+        (r"\banaly[sz]e\b", 3), (r"\banalysis\b", 3), (r"\bmetrics?\b", 3),
+        (r"\bstatistic", 3), (r"\bcsv\b", 4), (r"\bsql\b", 4),
+        (r"\bdashboard\b", 4), (r"\bdataset\b", 3),
+        (r"\baggregat", 3), (r"\bpivot\b", 4), (r"\bcorrelat", 3),
+    ],
+    "office-docs": [
+        (r"\bexcel\b", 5), (r"\bxlsx\b", 5), (r"\bspreadsheet\b", 5),
+        (r"\bpowerpoint\b", 5), (r"\bpptx\b", 5), (r"\bslide deck\b", 5),
+        (r"\bpresentation\b", 4),
+        (r"\bdocx\b", 5), (r"\bword document\b", 5), (r"\boutlook\b", 4),
+        (r"\bcalendar\b", 3), (r"\binvoice\b", 3),
+    ],
+    "creative": [
+        (r"\bpoem\b", 5), (r"\bstory\b", 4), (r"\bsong\b", 5),
+        (r"\blyrics\b", 5), (r"\bblog post\b", 4), (r"\bessay\b", 4),
+        (r"\bcreative\b", 4), (r"\blogo\b", 4), (r"\billustration\b", 4),
+        (r"\binfographic\b", 4), (r"\bbrand(ing)?\b", 3),
+        (r"\bnarrative\b", 3), (r"\bdesign a\b", 2),
+    ],
+    "ops": [
+        (r"\bdeploy(ment)?\b", 4), (r"\bdocker\b", 4), (r"\bkubernetes\b", 5),
+        (r"\bk8s\b", 5), (r"\bterraform\b", 5), (r"\bci/cd\b", 4),
+        (r"\binfra(structure)?\b", 4),
+        (r"\bnginx\b", 4), (r"\bsystemd\b", 4),
+        (r"\bmigrat", 2), (r"\bbackup\b", 3),
+        (r"\bmonitor(ing)?\b", 2), (r"\balert(s|ing)?\b", 2),
+    ],
+}
+
+_COMPILED: dict[str, list[tuple[re.Pattern[str], int]]] = {
+    task: [(re.compile(pat, re.IGNORECASE), w) for pat, w in rules]
+    for task, rules in _KEYWORDS.items()
+}
+
+# Tool-signature boosts. The corpus is coding-heavy, so tool evidence is
+# what separates the coding sub-buckets from each other.
+_EDIT_TOOLS = {"patch", "edit", "write_file", "write", "apply_patch",
+               "str_replace_editor", "multiedit", "notebookedit"}
+_WEB_TOOLS = {"web_search", "web_extract", "webfetch", "websearch", "fetch",
+              "browser", "browser_exec"}
+_SHELL_TOOLS = {"exec", "bash", "terminal", "exec_command", "powershell",
+                "shell", "run_command"}
+_READ_TOOLS = {"read_file", "read", "search_files", "grep", "glob",
+               "list_dir", "ls"}
+_TEST_RUNNER_RE = re.compile(
+    r"\b(pytest|py\.test|npm (run )?test|yarn test|pnpm test|jest|vitest|"
+    r"go test|cargo test|mvn test|gradle test|ctest|tox|nox)\b", re.IGNORECASE)
+_OFFICE_TOOL_RE = re.compile(
+    r"\b(xlsx|pptx|docx|openpyxl|python-pptx|python-docx|excel|powerpoint)\b",
+    re.IGNORECASE)
+
+# Splice/reader scaffolding prepended to the first user row. It names the
+# subagent role ("root/whole_branch_review") and would otherwise dominate
+# keyword scoring — strip it before classifying.
+_SCAFFOLD_RE = re.compile(
+    r"^\s*(?:\[subagent brief[^\]]*\]"
+    r"|\[trace segment \d+ of \d+[^\]]*\]"
+    r"|\[continued[^\]]*\])\s*", re.IGNORECASE)
+
+# German task vocabulary — the corpus carries ~8% German traces and the
+# English keyword sets score them as "other".
+_KEYWORDS_DE: dict[str, list[tuple[str, int]]] = {
+    "debug": [(r"\bfehler\b", 3), (r"\bbeheben\b", 3), (r"\bdebuggen\b", 4),
+              (r"\bkaputt\b", 3), (r"\bschlägt fehl\b", 3),
+              (r"\bwarum\b", 2), (r"\bproblem\b", 2)],
+    "test": [(r"\btests?\b", 2), (r"\btestabdeckung\b", 4),
+             (r"\bprüf(?:en|e|ung)\b", 1)],
+    "refactor": [(r"\bumbauen\b", 4), (r"\brefaktor", 5),
+                 (r"\bvereinfach", 3), (r"\baufräum", 3)],
+    "review": [(r"\bprüf(?:en|e|ung)\b", 3), (r"\breview\b", 3),
+               (r"\baudit\b", 4), (r"\bbewert", 3)],
+    "implement": [(r"\beinbauen\b", 4), (r"\bimplementier", 4),
+                  (r"\berstell", 3), (r"\bhinzufüg", 3), (r"\bumsetzen\b", 3)],
+    "research": [(r"\brecherch", 4), (r"\bvergleich", 3),
+                 (r"\buntersuch", 3), (r"\banalysier", 2)],
+    "data-analysis": [(r"\banalysier", 3), (r"\bauswert", 4),
+                      (r"\bkennzahlen\b", 4), (r"\bstatistik\b", 3)],
+    "office-docs": [(r"\bexcel\b", 5), (r"\bpräsentation\b", 5),
+                    (r"\bfolien\b", 4), (r"\bbericht\b", 2),
+                    (r"\btabelle\b", 3)],
+    "creative": [(r"\bgedicht\b", 5), (r"\bgeschichte\b", 4),
+                 (r"\btexten\b", 3), (r"\bgestalt", 3)],
+    "ops": [(r"\bdeploy", 4), (r"\bbereitstell", 3), (r"\binfrastruktur\b", 4),
+            (r"\bserver\b", 2), (r"\bkonfigurier", 3)],
+}
+_COMPILED_DE: dict[str, list[tuple[re.Pattern[str], int]]] = {
+    task: [(re.compile(pat, re.IGNORECASE), w) for pat, w in rules]
+    for task, rules in _KEYWORDS_DE.items()
+}
+
+# --- language detection -------------------------------------------------
+# The corpus is EN/DE only, so function-word density beats a real detector
+# here: no model call, no dependency, and it is stable on short task texts.
+# The two sets are deliberately disjoint ("war" is dropped: it is a common
+# English noun and a common German verb, so it carries no signal).
+_DE_WORDS = frozenset(
+    "der die das den dem des und oder nicht kein keine ist sind wird werden "
+    "wurde wurden bitte für mit auf von zu im ein eine einen einem einer auch "
+    "noch schon aber wenn dass wie wir ich du sie es hat haben kann können "
+    "muss müssen soll sollen nach über unter zwischen durch bei aus als nur "
+    "mehr sehr jetzt dann hier dort diese dieser dieses diesem sowie bereits "
+    "jeweils".split()
+)
+_EN_WORDS = frozenset(
+    "the and or not is are be been to of for with from by at this that these "
+    "those it its we you they he she i have has had can could should would "
+    "must may do does did but if then than only more very now here there what "
+    "which who how when where why all any each every some no yes will was were "
+    "so in on as an a".split()
+)
+_WORD_RE = re.compile(r"[A-Za-zÄÖÜäöüß]+")
+_UMLAUT_RE = re.compile(r"[äöüß]", re.IGNORECASE)
+
+
+def language_scores(text: str) -> tuple[int, int]:
+    """(de, en) function-word counts for a text, umlaut bonus included.
+
+    Public so the structural validator can require a CONFIDENT language
+    call (a margin and a ratio) instead of detect_language's tie-break.
+    """
+    words = _WORD_RE.findall(text.lower())
+    de = sum(1 for w in words if w in _DE_WORDS)
+    en = sum(1 for w in words if w in _EN_WORDS)
+    if _UMLAUT_RE.search(text):
+        de += 2  # strong signal that survives very short texts
+    return de, en
+
+
+def detect_language(text: str) -> str:
+    """Return 'de' or 'en' for a task text. Defaults to 'en' on a tie."""
+    if not text:
+        return "en"
+    de, en = language_scores(text)
+    return "de" if de > en else "en"
+
+
+LANGUAGE_NAMES = {"en": "English", "de": "German"}
+
+
+def _tool_names(t: Trace) -> set[str]:
+    names: set[str] = set()
+    for m in t.messages:
+        for tc in m.tool_calls:
+            if tc.name:
+                names.add(tc.name.lower())
+    return names
+
+
+def _tool_args_blob(t: Trace, limit: int = 20000) -> str:
+    parts: list[str] = []
+    total = 0
+    for m in t.messages:
+        for tc in m.tool_calls:
+            a = tc.arguments or ""
+            parts.append(a)
+            total += len(a)
+            if total > limit:
+                return " ".join(parts)
+    return " ".join(parts)
+
+
+def heuristic_scores(t: Trace) -> dict[str, float]:
+    """Raw per-task scores from keywords + tool signature."""
+    scores = {task: 0.0 for task in TASKS}
+    raw = (t.first_task_message() or "")[:6000]
+    text = _SCAFFOLD_RE.sub("", raw)
+    for rules in (_COMPILED, _COMPILED_DE):
+        for task, task_rules in rules.items():
+            for rx, weight in task_rules:
+                hits = len(rx.findall(text))
+                if hits:
+                    # diminishing returns: one strong term beats a keyword soup
+                    scores[task] += weight * (1 + 0.25 * min(hits - 1, 4))
+
+    tools = _tool_names(t)
+    if tools & _EDIT_TOOLS:
+        scores["implement"] += 3
+        scores["refactor"] += 1.5
+    if tools & _WEB_TOOLS:
+        scores["research"] += 4
+    if tools & _SHELL_TOOLS:
+        scores["ops"] += 1
+        scores["debug"] += 1
+    if tools and not (tools & _EDIT_TOOLS) and (tools & _READ_TOOLS):
+        scores["review"] += 2
+    if _TEST_RUNNER_RE.search(_tool_args_blob(t)):
+        scores["test"] += 4
+    if _OFFICE_TOOL_RE.search(_tool_args_blob(t)):
+        scores["office-docs"] += 5
+    return scores
+
+
+def classify_heuristic(t: Trace) -> dict[str, Any]:
+    """Deterministic label + confidence + the evidence that produced it."""
+    scores = heuristic_scores(t)
+    ranked = sorted(scores.items(), key=lambda kv: kv[1], reverse=True)
+    top_task, top = ranked[0]
+    second = ranked[1][1] if len(ranked) > 1 else 0.0
+    if top <= 0:
+        return {"task": "other", "confidence": 0.0, "scores": scores,
+                "method": "heuristic"}
+    if top == second:
+        # a tie is not a label — hand it to the LLM tier or "other"
+        return {"task": "other", "confidence": 0.0, "scores": scores,
+                "method": "heuristic", "tie": True}
+    # confidence: share of the top two, so a clear winner scores near 1.0
+    confidence = (top - second) / top
+    return {"task": top_task, "confidence": round(confidence, 3),
+            "scores": scores, "method": "heuristic"}
+
+
+# ------------------------------------------------------------- LLM tier
+
+_CLASSIFY_SCHEMA: dict[str, Any] = {
+    "type": "object",
+    "properties": {
+        "task": {"type": "string", "enum": list(TASKS)},
+        "reason": {"type": "string"},
+    },
+    "required": ["task"],
+}
+
+
+def _classify_prompt(t: Trace) -> str:
+    task_text = (t.first_task_message() or "")[:2500]
+    tools = sorted(_tool_names(t))
+    # a few assistant prose lines carry the intent when the user message is
+    # terse ("continue", "go on")
+    prose = [
+        (m.content or "").strip()[:200]
+        for m in t.messages
+        if m.role == "assistant" and (m.content or "").strip()
+    ][:4]
+    taxonomy = "\n".join(f"- {k}: {v}" for k, v in TASK_DESCRIPTIONS.items())
+    return (
+        "Classify this agent session into exactly one task bucket.\n\n"
+        f"Buckets:\n{taxonomy}\n\n"
+        f"Tools used: {', '.join(tools) or '(none)'}\n\n"
+        f"User task:\n{task_text}\n\n"
+        f"Assistant prose samples:\n" + "\n".join(f"- {p}" for p in prose) +
+        "\n\nAnswer with the single best bucket and a one-line reason."
+    )
+
+
+def classify_llm(t: Trace, cfg) -> Optional[dict[str, Any]]:
+    """One structured call. Returns None when the endpoint is unavailable."""
+    from traceforge import triage
+
+    try:
+        base, key, model = triage._resolve_endpoint(cfg)
+    except Exception:
+        return None
+    payload = {
+        "model": model,
+        "messages": [
+            {"role": "system",
+             "content": "You label agent sessions. Answer with JSON only."},
+            {"role": "user", "content": _classify_prompt(t)},
+        ],
+        "temperature": 0.0,
+        "max_tokens": 200,
+        "response_format": {
+            "type": "json_schema",
+            "json_schema": {"name": "task_label",
+                            "schema": _CLASSIFY_SCHEMA, "strict": True},
+        },
+    }
+    try:
+        data = triage._post_json(
+            triage._chat_url(base), payload, key, cfg.triage_timeout_s)
+        raw = triage._extract_content(data)
+        obj = json.loads(raw) if raw else None
+    except Exception:
+        return None
+    if not isinstance(obj, dict) or obj.get("task") not in TASKS:
+        return None
+    return {"task": obj["task"], "confidence": 1.0,
+            "reason": obj.get("reason", ""), "method": "llm"}
+
+
+def classify_trace(
+    t: Trace, cfg=None, *, use_llm: bool = True, min_confidence: float = 0.35,
+) -> dict[str, Any]:
+    """Label one trace.
+
+    LLM first when an endpoint is configured (it reads intent, which the
+    keyword prior cannot); the heuristic is the fallback and is always
+    computed so the result carries the evidence for review.
+    """
+    heuristic = classify_heuristic(t)
+    if use_llm and cfg is not None:
+        llm = classify_llm(t, cfg)
+        if llm:
+            llm["heuristic"] = heuristic
+            return llm
+    return heuristic

traceforge/config.py

@@ -0,0 +1,276 @@
+"""traceforge configuration: defaults + optional YAML overrides."""
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass, field
+from typing import Any, Optional
+
+DEFAULT_HERMES_HOME = os.environ.get("HERMES_HOME", os.path.expanduser("~/.hermes"))
+
+
+@dataclass
+class Config:
+    # ---------------------------------------------------------------- paths
+    hermes_home: str = DEFAULT_HERMES_HOME
+    state_db: str = ""  # derived from hermes_home when empty
+    data_dir: str = ""  # derived from repo root when empty
+    guidelines_path: str = ""  # derived from repo root when empty
+    # Seed stores (2026-10-13): the repo ships portable trace archives —
+    # harness-traces/{hermes,claude,codex} (a snapshot of all three
+    # live stores) and exported-seed-datas/* (codex rollouts from other
+    # machines). resolve() auto-appends them to the reader path lists
+    # below when the folders exist, so `traceforge generate` /
+    # scripts/generate.sh processes them with zero configuration.
+    # Lists are ADDITIVE to the live stores; empty = live store only.
+    hermes_state_dbs: list[str] = field(default_factory=list)
+    claude_projects_dirs: list[str] = field(default_factory=list)
+    # Published SFT corpora normalized into the pool by
+    # scripts/fetch_nemotron.py (one <subset>.jsonl per dataset subset).
+    # Auto-discovered from seed-datasets/nemotron/ when it exists.
+    nemotron_roots: list[str] = field(default_factory=list)
+
+    # -------------------------------------------------------- trace picking
+    # min_messages / min_tool_calls are HARD floors: they drop degenerate
+    # stubs (2-9 message probes), not long work.
+    # max_messages is a SOFT cap (2026-10-23): a session above it is still
+    # a candidate, flagged `oversize` in the listing row. The hard cap it
+    # replaced silently discarded 3,026 real agentic sessions (563 hermes,
+    # 1,246 codex, 1,217 claude — 35% of the viable pool) before they
+    # could reach split_long, which is the stage that actually bounds what
+    # the pipeline processes: by rendered size (split_max_chars), not by
+    # message count. A 200-message session is exactly the long tool-call
+    # chain the corpus wants.
+    # NOTE: `traceforge list|pull|generate` take these from argparse
+    # defaults in cli.py, not from this file — see the report's cli.py
+    # patch note.
+    min_messages: int = 10
+    max_messages: int = 120
+    min_tool_calls: int = 2
+
+    # ------------------------------------------------------- prompt budgets
+    user_chars: int = 4000
+    assistant_chars: int = 4000
+    tool_result_chars: int = 1200
+    tool_args_chars: int = 600
+    include_reasoning: bool = False
+
+    # ------------------------------------------------------ improver harness
+    model: Optional[str] = None  # None = hermes default profile model
+    provider: Optional[str] = None
+    max_turns: Optional[int] = 16  # improver: read → write → validate → reply, with headroom
+    run_budget: Optional[int] = None
+    harness_timeout_s: int = 1200
+    improver_workdir: str = ""  # derived: data_dir/work/ when empty
+    skills: list[str] = field(default_factory=list)  # -s preloads for improver
+
+    # ------------------------------------------------- external verification
+    verify: bool = True  # web-search fact-check stage (adversarial verifier)
+    verifier_max_turns: int = 20
+    verifier_timeout_s: int = 900
+
+    # ---------------------------------------------------- review rounds
+    # Never trust one reviewer: quality-pillar review runs independently
+    # this many times; ALL rounds must PASS for corpus admission.
+    review: bool = True
+    review_rounds: int = 2
+    review_max_turns: int = 16
+    review_timeout_s: int = 900  # deep-thinker reviewers verify byte-exact;
+                                 # 600s timed out in practice (2026-09-26)
+
+    # verification rounds: fact-check runs this many independent times
+    verify_rounds: int = 2
+
+    # ------------------------------------------------- cross-family confirm
+    # Second-family confirmation on promotion (residual-defect squeeze):
+    # after the primary judge PASSes, ONE selfhosted model from a
+    # different family re-judges the final window. VETO blocks promotion
+    # (the trace fails honestly and lands on the retry queue).
+    confirm_model: Optional[str] = None  # e.g. "gemma4-31b-selfhosted"
+
+    # ---------------------------------------------------------- long traces
+    # Traces whose compact rendering exceeds chunk_max_chars go through the
+    # map-digest-write pipeline instead of single-shot improvement.
+    chunk_max_chars: int = 100_000      # ≈25k tokens per chunk prompt
+    chunk_overlap_messages: int = 2     # continuity tail between chunks
+    digest_workers: int = 3             # parallel digester sessions
+    digest_max_turns: int = 8
+    digest_timeout_s: int = 600
+    digest_output_clip: int = 6000      # max chars per verbatim output kept
+
+    # ------------------------------------------------------- live telemetry
+    # LiveStats polls state.db child-session counters while batch runs
+    # are in flight. TTY: per-trace sublines + tok/s postfix. Pipes: one
+    # heartbeat line every live_heartbeat_s. 0 disables the poller.
+    live_poll_s: float = 2.0
+    live_heartbeat_s: float = 30.0
+
+    # ------------------------------------------------------------- triage
+    # Cheap LLM pass (structured generation, NOT a hermes one-shot) that
+    # labels every message keep/drop/rephrase/fact_check/summarize BEFORE
+    # the expensive improver runs. Output: data/triage/<sid>.json.
+    triage: bool = False
+    triage_base_url: str = ""   # empty = hermes active-profile endpoint
+    triage_model: str = ""      # empty = server default
+    triage_max_chars: int = 24_000  # per-window char budget (≈6k tokens)
+    triage_overlap_messages: int = 4
+    triage_workers: int = 8     # parallel window requests
+    # Parallel traces in the pre-pass (2026-10-23). The pre-pass used to be
+    # serial over traces, so a doubled pool meant hours of silent log before
+    # the first promotion. Each trace is self-contained, so the outer loop
+    # runs this many traces at once; total in-flight requests ≈
+    # triage_prepass_workers × triage_workers.
+    triage_prepass_workers: int = 4
+    triage_timeout_s: int = 300
+    triage_temperature: float = 0.0
+
+    # ------------------------------------------------- windowed splice engine
+    # Direct-HTTP windowed improve path (traceforge/splice.py): no agent
+    # runtime, stdlib only. Triage labels drive a mechanical splice; only
+    # windows with rephrase/summarize/fact_check edits pay a rewrite call;
+    # admission is a windowed judge instead of agent verify/review rounds.
+    splice: bool = True            # default improve path (False = hermes agents)
+    splice_max_chars: int = 24_000  # per-window char budget for rewrites/judge
+    splice_workers: int = 8         # parallel window HTTP requests
+    splice_temperature: float = 0.2
+    rewrite_max_tokens: int = 8000  # per-window rewrite emit ceiling
+    rewrite_timeout_s: int = 300   # per rewrite HTTP call
+    judge_model: str = ""           # empty = same as cfg.model
+    judge_max_tokens: int = 3000
+    judge_timeout_s: int = 300
+    allow_task_reframe: bool = True  # intent-only FAIL + good answer ->
+                                     # rewrite the task message once
+
+    # --------------------------------------------------------------- extras
+    raw_cache: bool = True  # cache normalized raw traces under data/traces/raw
+    overwrite: bool = False
+    # Long-trace splitting (2026-10-07, owner directive): raws whose
+    # rendered size exceeds split_max_chars become N segment traces
+    # <sid>#p1..N at pull time — each ~≤120k tokens, each its own
+    # pipeline candidate (triage/splice/judge/promotion per segment).
+    split_long: bool = True
+    split_max_chars: int = 480_000   # ≈120k tokens per output trace
+    # Split TRIGGER (2026-10-24): only traces above this rendered size are
+    # split at all; 0 = same as split_max_chars (split everything over the
+    # segment ceiling, the pre-2026-10-24 behaviour).
+    #
+    # Measured on 300KB+ raws re-split three ways (same 3 parents, 19
+    # segments, scratch/exp): 480k segments promoted 0/2 at 650s and ~197k
+    # tokens each; 240k segments 1/6 (16.7%) at 418s; 120k segments 5/11
+    # (45.5%) at 364s and ~76k tokens. Cost per promotion: 802s at 120k vs
+    # 2507s at 240k vs never at 480k. The historical corpus agrees — last
+    # segments promote 47% at 60-120KB, 42% at 120-180KB, 4.8% at 420-480KB.
+    # Recommended pair for the 300KB+ backlog: split_max_chars=120_000 with
+    # split_trigger_chars=240_000 (leave the 60-240KB cohort whole: it
+    # already promotes at 27-32% and splitting it only doubles the attempts).
+    split_trigger_chars: int = 0
+    # A tail smaller than this merges into the previous segment instead of
+    # becoming its own trace (2026-10-23: the merge branch was a no-op, so
+    # 192 of 1,044 segments were under this floor, 31 under 5k, the
+    # smallest 714 chars / 2 messages).
+    min_segment_chars: int = 60_000
+    # Hard ceiling on segments per parent session (2026-10-23). The soft
+    # max_messages cap admits long sessions, and a few are pathological:
+    # measured, one codex rollout is 50,114 messages / 155M chars, which
+    # splits into 317 segments of the same task. At ~10-20 min per
+    # segment that is days of batch time for one session, and it makes
+    # the pool count and ETA meaningless. Above this ceiling the split
+    # keeps the FIRST max-1 segments (task setup + early work) and the
+    # LAST one (the real final answer), dropping the middle — the same
+    # standalone-window contract every later segment already has. Set 0
+    # to disable the ceiling.
+    max_segments_per_trace: int = 16
+    # Codex rollout stores to read (live store + extracted seed exports).
+    # Extra roots are merged into one multi-root index; thread ids are
+    # globally unique (uuid v7) so collisions cannot occur.
+    codex_roots: list[str] = field(default_factory=list)
+    # Claude subagent sidechains (<uuid>/subagents/agent-*.jsonl) as
+    # separate candidate traces (MASS arXiv 2610.12176: multi-agent
+    # trajectories are the densest training signal; see export._agent_role).
+    claude_sidechains: bool = True
+
+    def resolve(self, repo_root: str) -> "Config":
+        self.state_db = self.state_db or os.path.join(self.hermes_home, "state.db")
+        self.data_dir = self.data_dir or os.path.join(repo_root, "data")
+        self.guidelines_path = self.guidelines_path or os.path.join(
+            repo_root, "guidelines", "IDEAL_TRACE_GUIDELINES.md"
+        )
+        self.improver_workdir = self.improver_workdir or os.path.join(
+            self.data_dir, "work"
+        )
+        self.codex_roots = [os.path.expanduser(r) for r in self.codex_roots]
+        # ---- seed-store auto-discovery (see field docs above) --------
+        # harness-traces/<store> + exported-seed-datas/* are appended to
+        # the reader path lists when present; already-listed paths are
+        # never duplicated (YAML overrides + auto-discovery merge).
+        def _add_unique(lst: list[str], item: str) -> None:
+            if item not in lst:
+                lst.append(item)
+
+        seed_hermes = os.path.join(repo_root, "harness-traces", "hermes", "state.db")
+        if os.path.exists(seed_hermes):
+            _add_unique(self.hermes_state_dbs, seed_hermes)
+        seed_claude = os.path.join(repo_root, "harness-traces", "claude")
+        if os.path.isdir(seed_claude):
+            _add_unique(self.claude_projects_dirs, seed_claude)
+        seed_codex = os.path.join(repo_root, "harness-traces", "codex")
+        if os.path.isdir(seed_codex):
+            _add_unique(self.codex_roots, seed_codex)
+        seed_rollouts = os.path.join(repo_root, "exported-seed-datas")
+        if os.path.isdir(seed_rollouts):
+            _add_unique(self.codex_roots, seed_rollouts)
+        seed_nemotron = os.path.join(repo_root, "seed-datasets", "nemotron")
+        if os.path.isdir(seed_nemotron):
+            _add_unique(self.nemotron_roots, seed_nemotron)
+        self.hermes_state_dbs = [os.path.expanduser(d) for d in self.hermes_state_dbs]
+        self.claude_projects_dirs = [
+            os.path.expanduser(d) for d in self.claude_projects_dirs
+        ]
+        self.nemotron_roots = [os.path.expanduser(d) for d in self.nemotron_roots]
+        return self
+
+    @property
+    def raw_dir(self) -> str:
+        return os.path.join(self.data_dir, "traces", "raw")
+
+    @property
+    def improved_dir(self) -> str:
+        return os.path.join(self.data_dir, "traces", "improved")
+
+    @property
+    def prompts_dir(self) -> str:
+        return os.path.join(self.data_dir, "prompts")
+
+    @property
+    def triage_dir(self) -> str:
+        return os.path.join(self.data_dir, "triage")
+
+    @property
+    def manifest_path(self) -> str:
+        return os.path.join(self.data_dir, "manifest.jsonl")
+
+    # ------------------------------------------------------------------ load
+
+    @classmethod
+    def load(cls, path: Optional[str] = None, repo_root: str = ".") -> "Config":
+        cfg = cls()
+        if path:
+            if not os.path.exists(path):
+                raise FileNotFoundError(f"config file not found: {path}")
+            try:
+                import yaml  # optional dependency
+            except ImportError as e:
+                raise RuntimeError(
+                    "PyYAML is required for --config files: uv pip install pyyaml"
+                ) from e
+            with open(path, "r", encoding="utf-8") as f:
+                overrides: dict[str, Any] = yaml.safe_load(f) or {}
+            unknown = set(overrides) - {f for f in cls.__dataclass_fields__}
+            if unknown:
+                raise ValueError(f"unknown config keys: {sorted(unknown)}")
+            cfg = cls(**{k: v for k, v in overrides.items() if v is not None})
+        return cfg.resolve(os.path.abspath(repo_root))
+
+
+def repo_root_from_pkg() -> str:
+    """Repo root as located relative to this package (traceforge/../)."""
+    return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

traceforge/defects.py

@@ -0,0 +1,186 @@
+"""Stable defect codes for a failed improve attempt.
+
+Why this module exists
+----------------------
+The pipeline used to record a failure as a free-text `stage` plus an `errors`
+list. That is enough to read one trace and useless for a funnel: "repair 2" and
+"gates fail (2)" are strings, so every report re-derived its own grouping and
+two reports of the same corpus disagreed.
+
+A defect code is the smallest thing that makes a loss countable. It is stable
+across runs, it is what the yield table is broken down by, and it is what tells
+you whether a bad week was the writer, the source, or the gate.
+
+The codes follow the shared taxonomy in the agentic-sft-synth skill suite
+(`agentic-verify` §"Defect codes") where a code already exists, and add the
+repo-specific ones the suite has no name for. Do not invent a new code for a
+failure that already has one; extend the mapping instead.
+
+    from traceforge.defects import classify, census
+    code = classify(stage="repair 2", errors=["max repair rounds exceeded"])
+    # -> "D-REP-01"
+"""
+
+from __future__ import annotations
+
+import collections
+import json
+import os
+import re
+from typing import Iterable, Optional
+
+# code -> (short name, layer, what it means, what to do about it)
+DEFECTS: dict[str, tuple[str, str, str, str]] = {
+    # --- source / task ---------------------------------------------------
+    "D-TASK-04": ("no_task", "init",
+                  "the source session has no user task message",
+                  "not rewritable; the task lived in another session"),
+    "D-TASK-05": ("harness_brief", "init",
+                  "the only opening user message is a harness dispatch brief",
+                  "unwrap the brief to the real task, or drop"),
+    "D-TASK-02": ("reframe_failed", "init",
+                  "the task could not be reframed into a single intent",
+                  "usually several tasks in one session; split it"),
+    # --- writer / repair -------------------------------------------------
+    "D-REP-01": ("repair_exhausted", "repair",
+                 "the repair budget ran out before the trace passed",
+                 "the dominant loss; a third round rarely helps (R-85)"),
+    "D-REP-02": ("repair_round_1", "repair",
+                 "failed after the first repair round",
+                 "inspect the first violation; often a single bad window"),
+    "D-REP-03": ("no_writer_artifact", "writer",
+                 "the writer produced nothing to verify",
+                 "check the writer call, not the trace"),
+    # --- gates -----------------------------------------------------------
+    "D-GATE-01": ("gate_reject", "gate",
+                  "a hard gate rejected the trace",
+                  "read the violation; a gate with low precision is R-88"),
+    "D-VER-01": ("judge_violation", "judge",
+                 "the judge found an unsupported claim or a broken pillar",
+                 "the trace is fluent and wrong; do not ship it"),
+    "D-VER-02": ("fabricated_observation", "gate",
+                 "a tool result shares no content with the call that produced it",
+                 "fabrication; the trace is unusable, not repairable"),
+    # --- format (shared with the suite) ----------------------------------
+    "D-FMT-01": ("pairing", "format",
+                 "tool call/result pairing or turn structure is broken",
+                 "structural; cheap to detect, never ship"),
+    "D-FMT-03": ("final_blob", "format",
+                 "the final answer is a dump of file contents, not a synthesis",
+                 "rewrite the closing turn or drop"),
+    "D-FMT-04": ("self_talk", "format",
+                 "the assistant narrates a plan and never acts on it",
+                 "drop the step; it teaches narration"),
+    "D-FMT-05": ("judge_residue", "format",
+                 "verifier text leaked into the trace",
+                 "strip it; it is not the model's output"),
+    "D-FMT-06": ("clip_marker", "format",
+                 "a truncation marker sits inside a message",
+                 "the trace was clipped; drop or re-pull"),
+    "D-SAFE-03": ("identity_burn", "safety",
+                  "the assistant claims a harness identity in a trainable span",
+                  "strip the claim mechanically before the gates"),
+    # --- infra -----------------------------------------------------------
+    # 2026-10-26: 191 traces failed triage inside a 37-minute window
+    # (01:50-02:27) and every one was recorded as D-UNK-01, because the
+    # per-window error text was discarded and no code existed for "the
+    # endpoint failed". An infra loss is not a trace defect: it must be
+    # retried, never counted against the pool's yield.
+    "D-INF-01": ("llm_unavailable", "infra",
+                 "the model endpoint failed every call for this trace",
+                 "not a trace defect; retry it, do not count it as a loss"),
+    # --- unknown ---------------------------------------------------------
+    "D-UNK-01": ("unclassified", "unknown",
+                 "a failure with no matching rule",
+                 "extend classify(); an unclassified loss is a reporting bug"),
+}
+
+# Ordered rules: first match wins. Order matters — the specific before the
+# general, because "max repair rounds exceeded" also contains "repair".
+_RULES: list[tuple[str, re.Pattern[str]]] = [
+    # infra first: when a record carries both a network error and a repair
+    # verdict, the network error is the cause and the verdict is its shadow.
+    ("D-INF-01", re.compile(
+        r"network:|urlopen|WinError 10054|connection (?:reset|refused|aborted)"
+        r"|infra error|failed for every window|rate limit", re.I)),
+    ("D-TASK-05", re.compile(r"harness dispatch brief|dispatch brief", re.I)),
+    ("D-TASK-04", re.compile(r"no user task message|no user task", re.I)),
+    ("D-TASK-02", re.compile(r"reframe", re.I)),
+    ("D-VER-02", re.compile(r"fabricated tool result|shares no content", re.I)),
+    ("D-FMT-01", re.compile(r"multi-turn:.*dropped|pairing|orphan", re.I)),
+    ("D-FMT-03", re.compile(r"final_blob|final answer is a (?:file )?dump", re.I)),
+    ("D-FMT-04", re.compile(r"self_talk|self-talk", re.I)),
+    ("D-FMT-05", re.compile(r"judge_residue|judge residue", re.I)),
+    ("D-FMT-06", re.compile(r"clip_marker|truncated", re.I)),
+    ("D-SAFE-03", re.compile(r"identity_burn|co-authored-by", re.I)),
+    ("D-REP-01", re.compile(r"max repair rounds|repair budget", re.I)),
+    ("D-REP-03", re.compile(r"no writer artifact|writer produced nothing", re.I)),
+    ("D-VER-01", re.compile(r"judge violations?|unsupported claim", re.I)),
+    ("D-GATE-01", re.compile(r"^gate:|gate fail", re.I)),
+]
+
+
+def classify(stage: Optional[str] = None,
+             errors: Optional[Iterable[str]] = None,
+             violations: Optional[Iterable[dict]] = None) -> str:
+    """Map a failure record to one stable code.
+
+    Errors are checked before the stage, because the error text is the specific
+    signal and the stage is the bucket. A record with neither is D-UNK-01 —
+    which is a bug in the caller, not a property of the trace.
+    """
+    text = " | ".join(str(e) for e in (errors or []))
+    for code, pat in _RULES:
+        if pat.search(text):
+            return code
+
+    # violations carry a pillar; a judge violation is the only one we can name
+    for v in (violations or []):
+        if str(v.get("pillar", "")).upper() in {"P2", "ANSWER"}:
+            return "D-VER-01"
+
+    st = (stage or "").strip().lower()
+    if st.startswith("repair 2"):
+        return "D-REP-01"
+    if st.startswith("repair 1"):
+        return "D-REP-02"
+    if st.startswith("gates fail"):
+        return "D-GATE-01"
+    if st.startswith("init"):
+        return "D-TASK-04"
+    if st.startswith("reframe"):
+        return "D-TASK-02"
+    return "D-UNK-01"
+
+
+def census(workdir: str) -> dict[str, int]:
+    """Count failure.json records by defect code, newest verdict per trace.
+
+    Reads only `failure.json` (the latest verdict), not the append-only log, so
+    a trace that failed three times counts once.
+    """
+    counts: collections.Counter[str] = collections.Counter()
+    try:
+        entries = os.listdir(workdir)
+    except OSError:
+        return {}
+    for name in entries:
+        path = os.path.join(workdir, name, "failure.json")
+        if not os.path.exists(path):
+            continue
+        try:
+            with open(path, encoding="utf-8") as f:
+                row = json.load(f)
+        except (OSError, ValueError):
+            counts["D-UNK-01"] += 1
+            continue
+        code = row.get("defect") or classify(
+            row.get("stage"), row.get("errors"), row.get("violations"))
+        counts[code] += 1
+    return dict(counts.most_common())
+
+
+def describe(code: str) -> str:
+    """One line for a report: code, name, meaning."""
+    name, layer, meaning, _ = DEFECTS.get(code, DEFECTS["D-UNK-01"])
+    return f"{code} {name} [{layer}] — {meaning}"

traceforge/difficulty.py

@@ -0,0 +1,66 @@
+"""Difficulty bands, measured from the trace itself.
+
+The card reported topic and language but not difficulty, so a reader could not
+tell whether the corpus was 2,000 one-shot edits or 2,000 twenty-call debugging
+sessions. Those are different datasets.
+
+The bands are the shared ones from the agentic-sft-synth suite
+(`agentic-task-design` §"Difficulty"): the number of tool calls in the ideal
+path, which is the only difficulty signal that is cheap, deterministic, and
+already in the record. It is a proxy — a 20-call mechanical rename is easier
+than a 6-call race-condition hunt — so the card reports it as a distribution,
+never as a score.
+
+    from traceforge.difficulty import band, census
+    band(7)          # -> "D3"
+    census(traces)   # -> {"D1": 12, "D2": 88, ...}
+"""
+
+from __future__ import annotations
+
+import collections
+from typing import Iterable
+
+# (code, lo, hi, label) — hi is inclusive; the last band is open-ended.
+BANDS: list[tuple[str, int, int, str]] = [
+    ("D1", 0, 2, "one-shot / single edit"),
+    ("D2", 3, 5, "short loop"),
+    ("D3", 6, 10, "multi-step"),
+    ("D4", 11, 20, "long session"),
+    ("D5", 21, 10**9, "very long session"),
+]
+
+# A trace below this is not a task worth training on: one call and an answer is
+# a lookup, and it teaches the model to stop early.
+MIN_CALLS = 1
+
+
+def band(n_calls: int) -> str:
+    """Map a tool-call count to its band code."""
+    for code, lo, hi, _ in BANDS:
+        if lo <= n_calls <= hi:
+            return code
+    return "D5"
+
+
+def label(code: str) -> str:
+    for c, _, _, lab in BANDS:
+        if c == code:
+            return lab
+    return "?"
+
+
+def count_calls(trace: dict) -> int:
+    """Tool calls in an exported record (`{"messages": [...], "meta": ...}`)."""
+    n = 0
+    for m in trace.get("messages") or []:
+        n += len(m.get("tool_calls") or [])
+    return n
+
+
+def census(traces: Iterable[dict]) -> dict[str, int]:
+    """Count exported records by band, in band order."""
+    counts: collections.Counter[str] = collections.Counter()
+    for t in traces:
+        counts[band(count_calls(t))] += 1
+    return {code: counts.get(code, 0) for code, _, _, _ in BANDS}

traceforge/enrich.py

@@ -0,0 +1,334 @@
+"""Recover the system prompt and tool definitions the harness actually sent.
+
+The readers drop both, so an exported row has tool_calls naming tools nothing
+defines — a model trained on it learns to emit `exec` without ever seeing what
+`exec` accepts. The data is not lost, it is just not in the Trace:
+
+- claude  ``~/.claude/projects/*/<sid>.jsonl`` records ``api-request-shape``
+          whose ``shape`` holds the full ``system`` blocks AND the ``tools``
+          array the request was made with.
+- codex   ``~/.codex/sessions/**/rollout-*-<sid>.jsonl`` opens with
+          ``session_meta.payload.base_instructions.text`` — the prompt, but
+          not the schemas (those live in the codex binary).
+- hermes  ``state.db``: ``sessions.system_prompt_hash`` -> ``system_prompts``.
+- nemotron already embeds a ``## Available tools`` block in its system row.
+
+Where the harness did not record schemas, the export falls back to
+``data/toolspec.json`` — schemas derived from the arguments the corpus
+actually passes, with per-property evidence counts. That is evidence, not
+invention, and ``meta.tools_source`` says which one a row got.
+"""
+from __future__ import annotations
+
+import glob
+import json
+import os
+import sqlite3
+from typing import Any, Optional
+
+CLAUDE_GLOB = os.path.expanduser("~/.claude/projects/*/*.jsonl")
+CODEX_GLOB = os.path.expanduser("~/.codex/sessions/**/*.jsonl")
+CODEX_ZST_GLOB = os.path.expanduser("~/.codex/sessions/**/*.jsonl.zst")
+HERMES_DB = os.path.expanduser("~/AppData/Local/hermes/state.db")
+TOOLSPEC = "data/toolspec.json"
+RAW_DIR = "data/traces/raw"
+
+
+def _sid_from_codex(path: str) -> Optional[str]:
+    """rollout-<iso>-<uuid>.jsonl -> <uuid>."""
+    base = os.path.basename(path)
+    if not base.startswith("rollout-"):
+        return None
+    stem = base
+    for suf in (".jsonl.zst", ".jsonl"):
+        if stem.endswith(suf):
+            stem = stem[: -len(suf)]
+            break
+    parts = stem.split("-")
+    # uuid is the last 5 dash groups
+    if len(parts) < 5:
+        return None
+    return "-".join(parts[-5:])
+
+
+def _read_claude_shape(path: str) -> tuple[Optional[str], Optional[list]]:
+    """(system_text, tools) from the first api-request-shape in the file."""
+    try:
+        with open(path, encoding="utf-8") as f:
+            for line in f:
+                if "api-request-shape" not in line:
+                    continue
+                try:
+                    r = json.loads(line)
+                except Exception:
+                    continue
+                if r.get("type") != "api-request-shape":
+                    continue
+                shape = r.get("shape") or {}
+                blocks = shape.get("system") or []
+                text = "\n\n".join(
+                    b.get("text", "") for b in blocks
+                    if isinstance(b, dict) and b.get("text"))
+                tools = shape.get("tools") or None
+                return (text or None), tools
+    except OSError:
+        pass
+    return None, None
+
+
+def _read_codex_prompt(path: str) -> Optional[str]:
+    # 587 of the rollouts on this host are zstd-compressed in place; the codex
+    # reader already knows how to open them, so reuse it rather than
+    # reimplementing the decompression here.
+    try:
+        from traceforge.readers.codex import _open_rollout
+        opener = _open_rollout
+    except Exception:
+        opener = lambda p: open(p, encoding="utf-8")
+    try:
+        with opener(path) as f:
+            for line in f:
+                if "session_meta" not in line:
+                    continue
+                try:
+                    r = json.loads(line)
+                except Exception:
+                    continue
+                if r.get("type") != "session_meta":
+                    continue
+                bi = (r.get("payload") or {}).get("base_instructions")
+                if isinstance(bi, dict):
+                    return bi.get("text") or None
+                if isinstance(bi, str):
+                    return bi or None
+                return None
+    except OSError:
+        pass
+    return None
+
+
+def _hermes_prompts() -> dict[str, str]:
+    """session_id -> system prompt, via the hash join."""
+    out: dict[str, str] = {}
+    if not os.path.exists(HERMES_DB):
+        return out
+    try:
+        c = sqlite3.connect(f"file:{HERMES_DB}?mode=ro", uri=True)
+        c.row_factory = sqlite3.Row
+        prompts = {r["hash"]: r["prompt"] for r in
+                   c.execute("select hash, prompt from system_prompts")}
+        for r in c.execute(
+                "select id, system_prompt_hash from sessions "
+                "where system_prompt_hash is not null"):
+            p = prompts.get(r["system_prompt_hash"])
+            if p:
+                out[r["id"]] = p
+        c.close()
+    except sqlite3.Error:
+        pass
+    return out
+
+
+_INDEX_CACHE: Optional[dict] = None
+
+
+def load_index(verbose: bool = False) -> dict[str, dict]:
+    """session_id -> {system, tools, source}.
+
+    Memoized: `export --by-topic` calls export_chat_jsonl once per topic, and
+    rebuilding this index each time re-reads 587 compressed rollouts and 401
+    claude transcripts — ten times the I/O for one export.
+    """
+    global _INDEX_CACHE
+    if _INDEX_CACHE is not None:
+        return _INDEX_CACHE
+    idx: dict[str, dict] = {}
+
+    for p in glob.glob(CLAUDE_GLOB):
+        sid = os.path.basename(p)[:-6]   # strip ".jsonl" (6 chars)
+        sys_text, tools = _read_claude_shape(p)
+        if sys_text or tools:
+            idx[sid] = {"system": sys_text, "tools": tools,
+                        "source": "claude-request-shape"}
+
+    codex_files = (glob.glob(CODEX_GLOB, recursive=True)
+                   + glob.glob(CODEX_ZST_GLOB, recursive=True))
+    for p in codex_files:
+        sid = _sid_from_codex(p)
+        if not sid or sid in idx:
+            continue
+        sys_text = _read_codex_prompt(p)
+        if sys_text:
+            idx[sid] = {"system": sys_text, "tools": None,
+                        "source": "codex-base-instructions"}
+
+    for sid, prompt in _hermes_prompts().items():
+        if sid in idx:
+            continue
+        idx[sid] = {"system": prompt, "tools": None,
+                    "source": "hermes-system-prompts"}
+
+    if verbose:
+        print(f"enrich index: {len(idx)} sessions "
+              f"({sum(1 for v in idx.values() if v['tools'])} with real tools)")
+    _INDEX_CACHE = idx
+    return idx
+
+
+_SPEC_CACHE: Optional[dict] = None
+
+
+def load_toolspec(path: str = TOOLSPEC) -> dict[str, dict]:
+    global _SPEC_CACHE
+    if _SPEC_CACHE is not None:
+        return _SPEC_CACHE
+    if not os.path.exists(path):
+        _SPEC_CACHE = {}
+        return _SPEC_CACHE
+    try:
+        _SPEC_CACHE = (json.load(open(path, encoding="utf-8")) or {}).get("tools") or {}
+    except Exception:
+        _SPEC_CACHE = {}
+    return _SPEC_CACHE
+
+
+_RAW_SYS_CACHE: dict[str, Optional[str]] = {}
+
+
+def _raw_system(sid: str, raw_dir: str = RAW_DIR) -> Optional[str]:
+    """The system row from the RAW trace, if it has one.
+
+    The improver drops the system message, so a nemotron trace arrives with no
+    prompt even though its raw carries a `## Available tools` block (12,878
+    raws on this host have one). Recover it here rather than re-generating.
+    """
+    if sid in _RAW_SYS_CACHE:
+        return _RAW_SYS_CACHE[sid]
+    out = None
+    path = os.path.join(raw_dir, sid + ".json")
+    if os.path.exists(path):
+        try:
+            t = json.load(open(path, encoding="utf-8"))
+            for m in t.get("messages") or []:
+                if m.get("role") == "system" and (m.get("content") or "").strip():
+                    out = m["content"]
+                    break
+        except Exception:
+            out = None
+    _RAW_SYS_CACHE[sid] = out
+    return out
+
+
+def _called_names(msgs: list[dict]) -> list[str]:
+    seen: list[str] = []
+    for m in msgs:
+        for tc in (m.get("tool_calls") or []):
+            fn = tc.get("function") or {}
+            name = fn.get("name") or tc.get("name")
+            if name and name not in seen:
+                seen.append(name)
+    return seen
+
+
+def _derived_tool(name: str, spec: dict) -> Optional[dict]:
+    """OpenAI-format tool definition from the observed-call evidence."""
+    s = spec.get(name)
+    if not s:
+        return None
+    props = {}
+    for k, v in (s.get("properties") or {}).items():
+        props[k] = {"type": v.get("type", "string")}
+        ex = v.get("examples") or []
+        if ex:
+            props[k]["description"] = f"e.g. {ex[0][:120]}"
+    return {
+        "type": "function",
+        "function": {
+            "name": name,
+            "description": (f"Derived from {s.get('calls', 0)} observed calls "
+                            f"in this corpus; property types and required "
+                            f"list come from the arguments actually passed."),
+            "parameters": {
+                "type": "object",
+                "properties": props,
+                "required": list(s.get("required") or []),
+            },
+        },
+    }
+
+
+def _to_openai_tool(t: dict) -> Optional[dict]:
+    """Normalize one recorded tool schema to OpenAI `{type, function}`.
+
+    The claude reader keeps the request's own shape, which is Anthropic:
+    `{name, description, input_schema}`. Every other schema in the export
+    (derived ones, the nemotron block) is OpenAI chat format, so a trainer
+    applying the chat template would meet two incompatible shapes in one
+    file. Convert at export time rather than at capture: the raw store
+    should keep what the harness actually sent.
+    """
+    if not isinstance(t, dict):
+        return None
+    if t.get("type") == "function" and isinstance(t.get("function"), dict):
+        return t  # already OpenAI
+    name = t.get("name")
+    if not name:
+        return None
+    params = t.get("input_schema") or t.get("parameters")
+    if not isinstance(params, dict):
+        params = {"type": "object", "properties": {}}
+    return {
+        "type": "function",
+        "function": {
+            "name": name,
+            "description": t.get("description") or "",
+            "parameters": params,
+        },
+    }
+
+
+def enrich(rec: dict, sid: str, index: dict, spec: dict) -> dict:
+    """Attach `tools` and a leading system message to one export record."""
+    msgs = rec.get("messages") or []
+    rec.setdefault("meta", {})
+    entry = index.get(sid) or {}
+
+    # ---- tools: real schemas if the harness recorded them, else derived
+    tools = entry.get("tools")
+    if tools:
+        norm = [x for x in (_to_openai_tool(t) for t in tools) if x]
+        if norm:
+            # The recorded request can list fewer tools than the trace
+            # actually called (a deferred tool, or one the harness added
+            # after the request was captured). Fill the gap from the
+            # observed-call evidence so no call names a tool nothing
+            # defines, and mark the additions so a derived schema is never
+            # mistaken for a recorded one.
+            have = {(t.get("function") or {}).get("name") for t in norm}
+            extra = [d for d in (_derived_tool(n, spec)
+                                 for n in _called_names(msgs))
+                     if d and (d.get("function") or {}).get("name") not in have]
+            rec["tools"] = norm + extra
+            rec["meta"]["tools_source"] = entry.get("source")
+            if extra:
+                rec["meta"]["tools_derived_extra"] = [
+                    (d.get("function") or {}).get("name") for d in extra]
+    else:
+        names = _called_names(msgs)
+        derived = [d for d in (_derived_tool(n, spec) for n in names) if d]
+        if derived:
+            rec["tools"] = derived
+            rec["meta"]["tools_source"] = "derived-from-observed-calls"
+            rec["meta"]["tools_undefined"] = [
+                n for n in names if n not in spec]
+
+    # ---- system prompt: prepend unless the trace already opens with one
+    sys_text = entry.get("system")
+    if not sys_text:
+        sys_text = _raw_system(sid)
+        if sys_text:
+            entry = dict(entry, source="raw-system-row")
+    if sys_text and not (msgs and msgs[0].get("role") == "system"):
+        rec["messages"] = [{"role": "system", "content": sys_text}] + msgs
+        rec["meta"]["system_prompt_source"] = entry.get("source")
+    return rec

traceforge/export.py

@@ -0,0 +1,564 @@
+"""Export improved traces to training formats.
+
+chat_jsonl: OpenAI-style chat fine-tune JSONL — one line per trace:
+  {"messages": [{"role","content"[,"tool_calls"][,"tool_call_id"...]}]}
+Reasoning is dropped by default (keep_reasoning=False): the corpus trains on
+the assistant's visible turns, and the reasoning channel is not part of the
+served format. Pass keep_reasoning=True to retain it under "reasoning".
+"""
+from __future__ import annotations
+
+import difflib
+import hashlib
+import json
+import os
+import re
+from typing import Optional
+
+from traceforge.trace import Trace
+
+# Progress narration: the agent announcing what it is about to do instead of
+# doing it. Same shape the SFT reframe gate rejects (_AGENT_NARRATION_RE in
+# splice.py); duplicated here so export does not import the splice module.
+_NARRATION_RE = re.compile(
+    r"\b(?:I'll|I will|Let me|Next,? I|Now I"
+    r"|Ich (?:prüfe|führe|werde|setze|beginne|starte|nehme))\b",
+    re.I,
+)
+
+
+def _msg_to_chat(m, keep_reasoning: bool = False) -> Optional[dict]:
+    d: dict = {"role": m.role}
+    if m.content is not None:
+        d["content"] = m.content
+    if keep_reasoning and m.reasoning:
+        d["reasoning"] = m.reasoning
+    if m.tool_calls:
+        d["tool_calls"] = [
+            {
+                "id": tc.id,
+                "type": "function",
+                "function": {"name": tc.name, "arguments": tc.arguments},
+            }
+            for tc in m.tool_calls
+        ]
+    if m.role == "tool":
+        d["tool_call_id"] = m.tool_call_id or ""
+        if m.tool_name:
+            d["name"] = m.tool_name
+    return d
+
+
+# Delegation tool names across the reader corpus (Claude 'Task', codex
+# thread_spawn, hermes delegate_task): a trace calling one of these is
+# an ORCHESTRATOR — it divides work across subagents. MASS (arXiv
+# 2610.12176) trains on orchestrator AND subagent conversations as
+# separate sequences; tagging lets the trainer balance the two roles
+# instead of sampling blind.
+_DELEGATION_TOOLS = {
+    "task", "delegate_task", "launchsubagent", "spawn_subagent",
+    "thread_spawn", "agent",
+}
+
+
+def _agent_role(t: Trace) -> Optional[str]:
+    """'orchestrator' | 'subagent' | None, derived mechanically.
+
+    subagent: the sid carries the reader's sidechain namespace
+    ('<parent>#sa-<agent>'), or provenance says the trace IS a
+    subagent's own thread (source claude-sidechain, or a codex child
+    rollout). The sid check matters because the improve stage rewrites
+    traces and overwrites source with 'traceforge-splice', while the
+    namespaced sid is preserved verbatim.
+    orchestrator: the trace issues a delegation tool call.
+    """
+    if "#sa-" in (t.session_id or ""):
+        return "subagent"
+    src = (t.source or "").lower()
+    if "sidechain" in src:
+        return "subagent"
+    for m in t.messages:
+        for tc in m.tool_calls:
+            if (tc.name or "").lower() in _DELEGATION_TOOLS:
+                return "orchestrator"
+    return None
+
+
+def export_chat_jsonl(
+    traces: list[Trace], out_path: str, *, append: bool = False,
+    keep_reasoning: bool = False, enrich: bool = True,
+) -> int:
+    """Write improved traces as chat-format JSONL. Returns lines written.
+
+    keep_reasoning=False (default) drops per-message reasoning: harness
+    self-talk is not training signal. Flip on when the training target
+    includes a reasoning channel.
+
+    enrich=True (default) attaches the `tools` array and, where the harness
+    recorded one, the system prompt. Without it a row names tools nothing
+    defines — see traceforge/enrich.py.
+    """
+    os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True)
+    mode = "a" if append else "w"
+    n = 0
+    _idx = None
+    _spec = None
+    if enrich:
+        from traceforge.enrich import load_index, load_toolspec
+        _idx = load_index()
+        _spec = load_toolspec()
+    with open(out_path, mode, encoding="utf-8") as f:
+        for t in traces:
+            msgs = [ _msg_to_chat(m, keep_reasoning) for m in t.messages ]
+            msgs = [m for m in msgs if m and (m.get("content") or m.get("tool_calls") or m.get("tool_call_id"))]
+            if not msgs:
+                continue
+            meta = {
+                "traceforge_session_id": t.session_id,
+                "parent_session_id": t.parent_session_id,
+                "source": t.source,
+            }
+            role = _agent_role(t)
+            if role:
+                meta["agent_role"] = role
+            # docs/DATA_QUALITY.md §4.1: a retained recovery is labelled so a
+            # trainer can filter it out and the corpus can report density.
+            if recovery_label(t):
+                meta["recovered"] = True
+            rec = {"messages": msgs, "meta": meta}
+            if enrich:
+                from traceforge.enrich import enrich as _enrich
+                sid = t.session_id
+                if sid.startswith("ideal-"):
+                    sid = sid[6:]
+                sid = sid.split("#")[0]
+                rec = _enrich(rec, sid, _idx or {}, _spec or {})
+            # Time sensitivity: measured on the RECORD, because the enriched
+            # row carries the `tools` array and "did the agent have a
+            # retrieval tool" is what separates a decaying answer from a
+            # durable procedure. Absent = not time-sensitive.
+            try:
+                from traceforge.timesensitivity import detect as _ts_detect
+                ts = _ts_detect(rec)
+                if ts["class"] != "not_time_sensitive":
+                    rec["meta"]["time_sensitivity"] = ts["class"]
+                    if ts.get("temporal_facts"):
+                        rec["meta"]["time_sensitivity_facts"] = ts["temporal_facts"]
+                    if ts.get("needs_rewrite"):
+                        rec["meta"]["time_sensitivity_needs_rewrite"] = True
+            except Exception:
+                pass
+            f.write(json.dumps(rec, ensure_ascii=False) + "\n")
+            n += 1
+    return n
+
+
+# --------------------------------------------------------------------------
+# Corpus-level obligations (docs/DATA_QUALITY.md §4.3, rules 6 and 9)
+# --------------------------------------------------------------------------
+
+# An ENVIRONMENT-caused failure is one the ideal agent could not have
+# foreseen: a flaky endpoint, a missing dependency, an exhausted resource.
+# A MODEL-caused failure (wrong tool, wrong args, wrong path, bad reasoning)
+# is never eligible for the §4.1 carve-out — the ideal path routes around it.
+#
+# The match is anchored to the HEAD of the tool output, because a harness
+# reports status there. Scanning the whole output matches error text quoted
+# inside source code and file listings: measured on this corpus, a loose
+# pattern labelled 32% of traces "recovered" where the real figure is ~12%,
+# with `502`/`429` matching line numbers (`502-  capture_output=True`) and
+# `timeout` matching documentation prose. Bare status codes are therefore
+# excluded; only an HTTP-context form counts.
+_ENV_ERROR_RE = re.compile(
+    r"(?i)(command not found"
+    r"|modulenotfounderror|cannot find module"
+    r"|connection (refused|reset|timed out|aborted)"
+    r"|(?:error|fatal|wait|request|operation|connection)[^\n]{0,40}"
+    r"(?:timed out|timeout)"
+    r"|temporarily unavailable|service unavailable"
+    r"|bad gateway|too many requests"
+    r"|network is unreachable|name or service not known"
+    r"|no space left on device|resource temporarily unavailable"
+    r"|too many open files)"
+)
+_MODEL_ERROR_RE = re.compile(
+    r"(?i)(invalid (argument|json|parameter|request)|unknown tool"
+    r"|tool not found|jsondecodeerror|unexpected token"
+    r"|missing required|schema validation|permission denied"
+    r"|operation not permitted|not a git repository"
+    r"|is a directory|no such file or directory)"
+)
+# How much of a tool output counts as its status head.
+_ERR_HEAD = 200
+# A grep/nl/`cat -n` listing starts with a line number. Its text is file
+# content, not a status line — `39:const RESOLVE_TIMEOUT_DETAIL: &str =
+# "Timed out opening chat event stream"` is a string literal, not a failure.
+_LISTING_RE = re.compile(r"^\s*\d+[:-]")
+
+
+def _call_sig(tc) -> tuple[str, str]:
+    return ((tc.name or "").lower(),
+            re.sub(r"\s+", " ", (tc.arguments or "")).strip()[:300])
+
+
+def recovery_label(t: Trace) -> bool:
+    """True when the trace retains exactly one environment-caused failure
+    followed by exactly one corrective step (docs/DATA_QUALITY.md §4.1).
+
+    Disqualified by a model-caused error, by a repeated identical failed call
+    (a retry loop, not a correction), or by an error with no corrective step
+    after it. Conservative by design: the carve-out is an exception, not a
+    license, so more than one retained error also disqualifies.
+    """
+    msgs = t.messages
+    env_hits = 0
+    for i, m in enumerate(msgs):
+        if m.role != "tool":
+            continue
+        c = (m.content or "")[:_ERR_HEAD]
+        if _LISTING_RE.match(c):
+            continue  # file content, not a status line
+        if not _ENV_ERROR_RE.search(c) or _MODEL_ERROR_RE.search(c):
+            continue
+        call = None
+        for j in range(i - 1, -1, -1):
+            if msgs[j].role == "assistant" and msgs[j].tool_calls:
+                call = msgs[j].tool_calls[-1]
+                break
+        if call is None:
+            continue
+        nxt = None
+        for j in range(i + 1, len(msgs)):
+            if msgs[j].role == "assistant" and msgs[j].tool_calls:
+                nxt = msgs[j].tool_calls[0]
+                break
+        if nxt is None:
+            continue
+        if _call_sig(nxt) == _call_sig(call):
+            return False  # retry loop: disqualified outright
+        env_hits += 1
+    return env_hits == 1
+
+
+def dedup_key(t: Trace) -> tuple[str, str]:
+    """(task, tool-sequence) identity — docs/DATA_QUALITY.md rule 9.
+
+    Task text alone over-merges (one brief solved two ways is two examples);
+    the tool sequence alone over-merges (unrelated tasks share exec>exec).
+    The pair is what repeats when a session is re-pulled or a split segment
+    is re-improved.
+
+    The sequence carries a digest of the call ARGUMENTS, not just the tool
+    names (2026-10-23). Name-only was measurably destructive: on 1,246
+    improved traces it removed 18, and every one was a DISTINCT session —
+    0 with an identical body, 0 with an identical first assistant turn.
+    Nine separate "watch the CI and report" sessions all reduce to
+    ``exec>exec>exec`` and were collapsed into one; the arguments are what
+    distinguishes them (a different job id per session). Arguments survive
+    re-improvement verbatim except on the splice engine's malformed-args
+    repair path, so a genuine re-pull still matches; a repaired duplicate
+    that slips through is one extra near-duplicate, which is far cheaper
+    than deleting real sessions.
+    """
+    task = ""
+    for m in t.messages:
+        if m.role == "user":
+            task = re.sub(r"\s+", " ", (m.content or "")).strip().lower()[:400]
+            break
+    names: list[str] = []
+    digest = hashlib.sha1()
+    for m in t.messages:
+        for c in (m.tool_calls or []):
+            name = (c.name or "").lower()
+            names.append(name)
+            digest.update(name.encode("utf-8", "replace"))
+            digest.update(b"\x00")
+            digest.update((c.arguments or "").encode("utf-8", "replace"))
+            digest.update(b"\x00")
+    return task, ">".join(names) + "#" + digest.hexdigest()[:16]
+
+
+def dedup_traces(traces: list[Trace]) -> tuple[list[Trace], list[Trace]]:
+    """Keep the most complete trace per (task, tool-sequence).
+
+    Returns (kept, removed). Deterministic — more messages wins, then more
+    tool calls, then session id — so re-exporting a grown corpus keeps the
+    same survivor instead of reshuffling the split.
+    """
+    best: dict[tuple[str, str], Trace] = {}
+    for t in traces:
+        k = dedup_key(t)
+        cur = best.get(k)
+        if cur is None:
+            best[k] = t
+            continue
+        a = (len(t.messages), sum(len(m.tool_calls or []) for m in t.messages))
+        b = (len(cur.messages), sum(len(m.tool_calls or []) for m in cur.messages))
+        if a > b or (a == b and (t.session_id or "") < (cur.session_id or "")):
+            best[k] = t
+    kept = list(best.values())
+    removed = [t for t in traces if best.get(dedup_key(t)) is not t]
+    return kept, removed
+
+
+# --------------------------------------------------------------------------
+# Preference corpus (KTO) — docs/DATA_QUALITY.md §4.4, rule 10
+# --------------------------------------------------------------------------
+# This is NOT a relabelling of the SFT corpus. KTO needs rejected samples
+# from the policy being aligned: a negative the policy would never produce is
+# a length/style gradient, not a preference signal. So the flow is staged:
+#
+#   1. extract_prompts() -> kto_prompts.jsonl: rollout seeds taken from the
+#      SFT corpus (the cleaned prefix at each assistant turn boundary).
+#   2. after SFT, sample N completions per prompt from the SFT model, verify
+#      each against the trace's own tool evidence, and label it.
+#   3. build_kto() -> kto.jsonl: length-matched, bounded-loss contract.
+#
+# Stage 2 needs a served model and is deliberately not in this module.
+
+def extract_prompts(traces, *, max_prompts: int = 0, turns: str = "final",
+                    max_prefix_chars: int = 0) -> list[dict]:
+    """Rollout seeds: (cleaned prefix, reference completion).
+
+    The prefix is the ideal trace's own context, so a rollout starts from the
+    distribution the model sees at inference — not from the raw trace's
+    messier prefix.
+
+    turns="final" (default) emits one seed per trace: the final assistant
+    turn, whose outcome the trace's own verification already covers. That is
+    the turn a KTO label can be trusted on. turns="all" emits every assistant
+    turn — 14k seeds and ~940 MB of prefixes on the current corpus — for
+    step-level preference work where each step has its own verifier.
+
+    max_prefix_chars > 0 keeps the opening task message and the most recent
+    turns, with an explicit elision marker, so a seed stays a bounded size.
+    """
+    out: list[dict] = []
+    for t in traces:
+        msgs = t.messages
+        idx = [i for i, m in enumerate(msgs)
+               if m.role == "assistant" and (m.content or m.tool_calls)]
+        if turns == "final":
+            idx = idx[-1:]
+        for i in idx:
+            prefix = msgs[:i]
+            if not any(p.role == "user" for p in prefix):
+                continue
+            chat = [m for m in (_msg_to_chat(p) for p in prefix) if m]
+            if max_prefix_chars:
+                chat = _clip_prefix(chat, max_prefix_chars)
+            out.append({
+                "session_id": t.session_id,
+                "turn": i,
+                "prompt": chat,
+                "reference": _msg_to_chat(msgs[i]),
+            })
+            if max_prompts and len(out) >= max_prompts:
+                return out
+    return out
+
+
+def _clip_prefix(chat: list[dict], limit: int) -> list[dict]:
+    """Keep the opening task and the most recent turns under `limit` chars."""
+    def size(m: dict) -> int:
+        return len(json.dumps(m, ensure_ascii=False))
+    if sum(size(m) for m in chat) <= limit:
+        return chat
+    head = chat[:1]
+    used = size(head[0])
+    tail: list[dict] = []
+    for m in reversed(chat[1:]):
+        n = size(m)
+        if used + n > limit:
+            break
+        tail.append(m)
+        used += n
+    tail.reverse()
+    return head + [{"role": "system",
+                    "content": "[earlier turns elided for length]"}] + tail
+
+
+def build_kto(rollouts, *, max_len_ratio: float = 1.5) -> tuple[list[dict], dict]:
+    """Turn labelled rollouts into KTO rows, length-matched.
+
+    Each rollout: {"prompt": [...], "completion": {...}, "label": bool}.
+    A negative whose completion is more than `max_len_ratio` times the
+    shortest desirable completion for the same prompt is dropped — otherwise
+    the only signal is length (arXiv:2403.19159). A prompt with no desirable
+    sample contributes nothing: an unpaired negative is noise, not a label.
+    """
+    by_prompt: dict[str, list[dict]] = {}
+    for r in rollouts:
+        key = json.dumps(r.get("prompt"), sort_keys=True, ensure_ascii=False)
+        by_prompt.setdefault(key, []).append(r)
+    rows: list[dict] = []
+    dropped_len = dropped_unpaired = 0
+    for group in by_prompt.values():
+        pos = [r for r in group if r.get("label")]
+        neg = [r for r in group if not r.get("label")]
+        if not pos:
+            dropped_unpaired += len(neg)
+            continue
+        ref_len = min(len(json.dumps(r["completion"], ensure_ascii=False))
+                      for r in pos)
+        for r in pos:
+            rows.append({"prompt": r["prompt"],
+                         "completion": r["completion"], "label": True})
+        for r in neg:
+            n = len(json.dumps(r["completion"], ensure_ascii=False))
+            if n > max_len_ratio * ref_len:
+                dropped_len += 1
+                continue
+            rows.append({"prompt": r["prompt"],
+                         "completion": r["completion"], "label": False})
+    stats = {
+        "rows": len(rows),
+        "desirable": sum(1 for r in rows if r["label"]),
+        "undesirable": sum(1 for r in rows if not r["label"]),
+        "prompts": len(by_prompt),
+        "dropped_length_mismatch": dropped_len,
+        "dropped_unpaired": dropped_unpaired,
+    }
+    return rows, stats
+
+
+def _defect_shape(text: str, task: str) -> Optional[str]:
+    """Name the defect a dropped assistant turn actually exhibits, or None.
+
+    The triage judge drops a turn for two very different reasons: it is
+    *wrong* (narration instead of work, a restatement of the brief, a bare
+    acknowledgement) or merely *redundant* (it repeats something already
+    said). Only the first is an instructive negative — training against
+    redundancy teaches the model to be terse, not to be correct. Measured on
+    the 1,456 dropped turns available 2026-10-22: 37% are defect shapes,
+    63% are redundancy.
+    """
+    body = (text or "").strip()
+    if not body:
+        return None
+    head = body[:200]
+    if _NARRATION_RE.match(body) or _NARRATION_RE.search(head):
+        return "narration"
+    if task:
+        a = re.sub(r"\s+", " ", task).strip().lower()[:400]
+        b = re.sub(r"\s+", " ", body).strip().lower()[:400]
+        if a and b and difflib.SequenceMatcher(None, a, b).ratio() > 0.6:
+            return "restates_task"
+    if len(body) < 120:
+        return "short_ack"
+    return None
+
+
+def build_kto_provisional(traces, raw_dir: str, triage_dir: str, *,
+                          max_rows: int = 0) -> tuple[list[dict], dict]:
+    """Provisional KTO rows for exercising the trainer — NOT on-policy.
+
+    Positives: each improved trace's final assistant turn, prompt = the
+    cleaned prefix. Negatives: assistant turns the triage judge labelled
+    ``drop``, prompt = that raw trace's own prefix.
+
+    ``docs/DATA_QUALITY.md`` §4.4 requires negatives sampled from the policy
+    being aligned. These come from the original harness agent, so this
+    artifact is a smoke-test set and a baseline, not the corpus a real KTO
+    run trains on — that one is built from SFT-model rollouts via
+    ``build_kto``. The negatives are at least *verified* bad (the triage
+    judge dropped them) rather than merely different, which is why this is
+    preferable to relabelling raw final answers: measured 2026-10-22, raw and
+    improved finals differ materially in 229 of 991 traces but are usually
+    just differently worded, not worse.
+    """
+    rows: list[dict] = []
+    seen: set[str] = set()
+
+    def _key(prompt, completion) -> str:
+        return json.dumps([prompt, completion], sort_keys=True, ensure_ascii=False)
+
+    for t in traces:
+        msgs = t.messages
+        idx = [i for i, m in enumerate(msgs)
+               if m.role == "assistant" and (m.content or m.tool_calls)]
+        if not idx:
+            continue
+        i = idx[-1]
+        prefix = msgs[:i]
+        if not any(p.role == "user" for p in prefix):
+            continue
+        prompt = [m for m in (_msg_to_chat(p) for p in prefix) if m]
+        comp = _msg_to_chat(msgs[i])
+        if not comp:
+            continue
+        k = _key(prompt, comp)
+        if k in seen:
+            continue
+        seen.add(k)
+        rows.append({"prompt": prompt, "completion": comp, "label": True,
+                     "session_id": t.session_id, "source": "improved"})
+        if max_rows and len(rows) >= max_rows:
+            return rows, _provisional_stats(rows, 0, 0, 0)
+
+    raws = {f[:-5]: os.path.join(raw_dir, f)
+            for f in os.listdir(raw_dir) if f.endswith(".json")}
+    n_neg = n_skipped = n_redundant = 0
+    for tf in sorted(os.listdir(triage_dir)):
+        if not tf.endswith(".json"):
+            continue
+        try:
+            with open(os.path.join(triage_dir, tf), encoding="utf-8") as f:
+                d = json.load(f)
+        except (OSError, json.JSONDecodeError):
+            continue
+        rp = raws.get(d.get("session_id"))
+        if not rp:
+            continue
+        try:
+            rt = Trace.load(rp)
+        except Exception:
+            continue
+        for lab in d.get("labels") or []:
+            if lab.get("action") != "drop":
+                continue
+            i = lab.get("i")
+            if not isinstance(i, int) or i >= len(rt.messages):
+                continue
+            m = rt.messages[i]
+            if m.role != "assistant" or not m.content or len(m.content) < 20:
+                continue
+            prefix = rt.messages[:i]
+            if not any(p.role == "user" for p in prefix):
+                continue
+            prompt = [c for c in (_msg_to_chat(p) for p in prefix) if c]
+            comp = _msg_to_chat(m)
+            if not comp:
+                continue
+            task = next((p.content or "" for p in prefix if p.role == "user"), "")
+            if not _defect_shape(m.content, task):
+                n_redundant += 1
+                continue
+            k = _key(prompt, comp)
+            if k in seen:
+                n_skipped += 1
+                continue
+            seen.add(k)
+            rows.append({"prompt": prompt, "completion": comp, "label": False,
+                         "session_id": rt.session_id, "source": "triage_drop"})
+            n_neg += 1
+            if max_rows and len(rows) >= max_rows:
+                return rows, _provisional_stats(rows, n_neg, n_skipped, n_redundant)
+    return rows, _provisional_stats(rows, n_neg, n_skipped, n_redundant)
+
+
+def _provisional_stats(rows: list[dict], n_neg: int, n_skipped: int,
+                       n_redundant: int = 0) -> dict:
+    pos = sum(1 for r in rows if r["label"])
+    neg = len(rows) - pos
+    return {
+        "rows": len(rows),
+        "desirable": pos,
+        "undesirable": neg,
+        "ratio": round(pos / neg, 2) if neg else None,
+        "duplicates_skipped": n_skipped,
+        "skipped_redundant": n_redundant,
+        "on_policy": False,
+        "note": ("negatives come from the original harness agent, not the SFT "
+                 "model — smoke-test artifact only (DATA_QUALITY.md §4.4)"),
+    }

traceforge/grounding.py

@@ -0,0 +1,256 @@
+"""Grounding validation: every salient fact in an improved trace must exist
+in its source trace. This is the mechanical no-hallucination gate (P4).
+
+Salient facts = numbers-with-units, file paths, hex ids, quoted technical
+strings. Sound but cheap: a trace that invents paths or counts fails here.
+
+Tool-result fidelity: an ideal trace that makes the same tool call must
+NOT carry a fabricated result. For each ideal tool row whose call signature
+(tool name + argument keys) matches a real call, the result must share
+content with the real output — exact line match first, then 25-char window
+containment (the improver sees a clipped rendering and may compress, so a
+wholly-unfamiliar output is fabrication, but paraphrase/reformat is not).
+"""
+from __future__ import annotations
+
+import json
+import re
+
+from traceforge.trace import Trace
+
+FACT_PATTERNS = [
+    # counts with units (verdicts, row/line counts, repeats)
+    re.compile(r"\b\d{1,6}\s+(?:PASS|SUSPECT|FAIL|CLEAN|lines|rows|files|chars|bytes|errors)\b"),
+    # file paths
+    re.compile(r"(?:[\w./-]+/)?[\w.-]+\.(?:py|ts|tsx|js|rs|go|svelte|json|jsonl|yaml|yml|toml|md|txt|csv|sql|sh)\b"),
+    # hex ids (commit shas, call ids)
+    re.compile(r"\b[0-9a-f]{7,40}\b"),
+]
+
+# facts the improver may legitimately introduce (its own bookkeeping)
+ALLOWLIST_PATTERN = re.compile(r"^(?:call_|ideal-)")
+
+# rendering artifacts, not world claims: the compact renderer's clip
+# markers (also matches the "+N chars clipped" tail of read_file-style
+# outputs). Stripped from the ideal blob before fact extraction.
+CLIP_MARKER_RE = re.compile(r"…?\[\+[\d,.]+ ?(?:chars|bytes) clipped\]")
+
+# unit claims that describe a rendering's compression, not the world.
+# "48613 chars" in an ideal trace is the improver mirroring its own
+# input rendering — never a salient fact.
+NON_WORLD_UNIT_RE = re.compile(r"^\d{1,6}\s+(?:chars|bytes)$")
+
+# fuzzy-containment window for tool-result fidelity
+_CONTAIN_WINDOW = 25
+
+# Verified on live promoted/attempted corpora (2026-09-28): fact patterns
+# extract GLUED tokens from ideal traces whose underlying path IS in the
+# source corpus — the top grounding false-kill shapes:
+#   'a///Users/x/y.py'  — unified-diff prefix "a/" glued onto a path that
+#                          sits after a literal \n in a JSON-embedded diff
+#   'nbackend/x/y.py'   — a literal \n escape's 'n' glued onto the next
+#                          path segment of a JSON-embedded tool output
+# The real corpus stores the same output with REAL newlines, so it
+# extracts the clean path and the glued variant looks invented. Before a
+# fact is declared missing, glue-stripped variants are re-tested; only
+# variants long enough to remain distinctive are considered.
+def _glue_variants(fact: str) -> list[str]:
+    """The fact itself plus glue-stripped spelling variants.
+
+    The BASE fact is always included (it is self-grounding when present
+    in the corpus); only the stripped variants must be distinctive
+    (len >= 8) to count, so short facts are never grounded by a
+    meaninglessly-short fragment.
+    """
+    out = [fact]
+    if "/" in fact:
+        stripped = re.sub(r"^[a-z]?/+", "", fact)  # glue char + slash run
+        if stripped != fact and len(stripped) >= 8:
+            out.append(stripped)
+        if len(fact) > 1 and len(fact) - 1 >= 8:
+            out.append(fact[1:])  # single leading char (n-glue, a/, b/)
+    # count+unit facts: the corpus may spell the same claim with the
+    # unit BEFORE the numbers ("lines old/new: 12875 12875" — verified
+    # false-kill 1fa6a3e5: identical bytes in ideal and raw, the
+    # NUMBER+UNIT pattern simply creates a token order the corpus lacks).
+    m = re.match(r"^(\d{1,6})\s+(lines|rows|files|errors)$", fact)
+    if m:
+        out.append(f"{m.group(2)} old/new: {m.group(1)} {m.group(1)}")
+        out.append(f"{m.group(2)}: {m.group(1)}")
+    return out
+
+
+def _ideal_blob(ideal: Trace) -> str:
+    """Content + tool args of the ideal trace, clip markers stripped."""
+    blob = "\n".join(
+        (m.content or "") for m in ideal.messages
+    ) + "\n" + "\n".join(
+        tc.arguments for m in ideal.messages for tc in m.tool_calls
+    )
+    return CLIP_MARKER_RE.sub("", blob)
+
+
+def _real_corpus(real: Trace) -> str:
+    parts = [m.content or "" for m in real.messages]
+    parts += [tc.arguments for m in real.messages for tc in m.tool_calls]
+    return "\n".join(parts)
+
+
+def grounding_coverage(ideal: Trace, real: Trace) -> tuple[float, list[str]]:
+    """Return (fraction_grounded, missing_facts). 1.0 == fully grounded."""
+    corpus = _real_corpus(real)
+    ideal_blob = _ideal_blob(ideal)
+    missing: list[str] = []
+    total = 0
+    grounded = 0
+    for pat in FACT_PATTERNS:
+        for f in set(pat.findall(ideal_blob)):
+            if ALLOWLIST_PATTERN.match(f):
+                continue
+            if NON_WORLD_UNIT_RE.match(f):
+                continue  # rendering artifact (mirrored clip marker)
+            total += 1
+            # JSON-embedded outputs glue a stray char onto the head of a
+            # real path (verified false-kill shapes, see _glue_variants).
+            # A fact is grounded when IT or a glue-stripped variant sits
+            # in the real corpus.
+            if any(v in corpus for v in _glue_variants(f)):
+                grounded += 1
+            else:
+                missing.append(f)
+    if total == 0:
+        return 1.0, []
+    return grounded / total, missing
+
+
+def validate_grounding(
+    ideal: Trace, real: Trace, min_coverage: float = 0.95
+) -> list[str]:
+    """Gate: fail when salient-fact grounding is below the threshold."""
+    errs: list[str] = []
+    cov, missing = grounding_coverage(ideal, real)
+    if cov < min_coverage:
+        shown = ", ".join(m[:80] for m in missing[:8])
+        errs.append(
+            f"fact grounding {cov:.0%} < {min_coverage:.0%}; "
+            f"facts not in source trace: {shown}"
+        )
+    errs.extend(tool_result_fidelity_errors(ideal, real))
+    return errs
+
+
+def _tool_output_lines(raw: str) -> list[str]:
+    """Substantive lines of a tool output; unwraps execute_code-style JSON."""
+    try:
+        d = json.loads(raw)
+        if isinstance(d, dict) and isinstance(d.get("output"), str):
+            raw = d["output"]
+    except (json.JSONDecodeError, TypeError):
+        pass
+    return [l.strip() for l in raw.split("\n") if len(l.strip()) >= 25]
+
+
+def _call_signature(name: str, arguments: str) -> str:
+    """Stable-ish call fingerprint: tool name + sorted argument string."""
+    try:
+        a = json.loads(arguments or "{}")
+        keys = ",".join(sorted(a)) if isinstance(a, dict) else str(sorted(a))
+    except json.JSONDecodeError:
+        keys = (arguments or "")[:80]
+    return f"{name}({keys})"
+
+
+def _windows(line: str, size: int = _CONTAIN_WINDOW):
+    """All size-length windows of a line, stepped every quarter-window."""
+    if len(line) < size:
+        return [line]
+    step = max(1, size // 4)
+    return [line[i:i + size] for i in range(0, len(line) - size + 1, step)]
+
+
+def _result_grounded(lines: list[str], rlines: list[str]) -> bool:
+    """Do the ideal result's lines share content with the real output?
+
+    Two tiers: exact stripped-line match (verbatim reuse), then 25-char
+    window containment (compressed/paraphrased reuse of real evidence).
+    """
+    rset = set(rlines)
+    if any(l in rset for l in lines):
+        return True
+    rjoin = "\n".join(rlines)
+    for l in lines:
+        for w in _windows(l):
+            if w in rjoin:
+                return True
+    return False
+
+
+def _real_output_index(real: Trace) -> dict[str, list[tuple[str, list[str]]]]:
+    """Per tool name: [(call signature, substantive output lines)]."""
+    idx: dict[str, list[tuple[str, list[str]]]] = {}
+    for m in real.messages:
+        for tc in m.tool_calls:
+            res = next(
+                (r for r in real.messages
+                 if r.role == "tool" and r.tool_call_id == tc.id),
+                None,
+            )
+            if res is not None and res.content:
+                sig = _call_signature(tc.name, tc.arguments)
+                idx.setdefault(tc.name, []).append(
+                    (sig, _tool_output_lines(res.content))
+                )
+    return idx
+
+
+def _ideal_call_sigs(ideal: Trace) -> dict[str, str]:
+    """tool_call_id -> signature, for the ideal trace's own calls."""
+    out: dict[str, str] = {}
+    for m in ideal.messages:
+        for tc in m.tool_calls:
+            out[tc.id] = _call_signature(tc.name, tc.arguments)
+    return out
+
+
+def tool_result_fidelity_errors(ideal: Trace, real: Trace) -> list[str]:
+    """Fabricated tool results: same-call results sharing no content.
+
+    An ideal tool row whose tool was really called fails when its result
+    shares neither an exact substantive line nor a 25-char window with
+    any real output of that tool. The error names the ideal call's
+    signature, shows what the real trace's output actually looked like,
+    and gives a concrete rewrite target — a vague "be more grounded"
+    never repairs anything.
+    """
+    idx = _real_output_index(real)
+    sigs = _ideal_call_sigs(ideal)
+    errs: list[str] = []
+    for m in ideal.messages:
+        if m.role != "tool":
+            continue
+        cands = idx.get(m.tool_name or "", [])
+        if not cands:
+            continue  # tool never called in real trace: coverage handles it
+        lines = _tool_output_lines(m.content or "")
+        if not lines:
+            continue
+        for _sig, rlines in cands:
+            if _result_grounded(lines, rlines):
+                break  # grounded against (at least) one real result
+        else:
+            sig = sigs.get(m.tool_call_id or "", "?")
+            # prefer the real output of the SAME call when one exists,
+            # else the tool's first real output — concrete repair target
+            same = [rl for s, rl in cands if s == sig]
+            sample = (same or [rl for _s, rl in cands])[0]
+            head_txt = " | ".join(l[:70] for l in sample[:2]) or "(empty)"
+            head = lines[0][:90]
+            errs.append(
+                f"fabricated tool result [{m.tool_name}]: result shares no "
+                f"content with any real {m.tool_name} output "
+                f"(e.g. {head!r}). The ideal call is {sig}; the real trace's "
+                f"output looks like: {head_txt!r}. Rewrite this result to "
+                "quote the real trace's actual output (compressed is fine, "
+                "but every substantive line must come from the real output)."
+            )
+    return errs

traceforge/identity.py

@@ -0,0 +1,131 @@
+"""Identity-neutrality and refusal gates.
+
+The reference must not burn in an assistant identity ("I am Claude",
+"as an AI made by …") and must not teach refusals — a model scored against
+it should neither claim a vendor self-identity nor reproduce refusal
+phrasings.  These gates are deterministic regex checks over ASSISTANT
+prose (system/user rows are never rewritten by the pipeline anyway and
+tool results are verbatim facts, so only assistant content can carry a
+learnable self-claim).
+
+Domain content rule: a trace whose TASK is about a model/vendor (e.g.
+"compare chatgpt vs claude APIs", "debug Claude-Code traces") may
+LEGITIMATELY contain those names — that is content, not identity.  Only
+SELF-referential claims are gated: first-person identity patterns
+("I am Claude", "I'm an AI", "trained by Anthropic", …) and refusal
+patterns ("I can't help with that", "against my guidelines", …).
+Mentions like "CLAUDE.md exists in the repo" are never touched.
+"""
+from __future__ import annotations
+
+import json
+import re
+from typing import Optional
+
+# First-person identity self-claims. Anchored on first-person verbs so
+# "the trace contains Claude-Code traces" (third person) never matches.
+IDENTITY_RE = re.compile(
+    r"("
+    r"co-?authored-?by:\s*"
+    r"(claude|chatgpt|gpt-?[45o]|glm|gemini|qwen|gemma|llama|"
+    r"deepseek|copilot|sonnet|opus|anthropic|openai)\b"
+    r"|\b(i am|i'?m|my name is|call me)\s+"
+    r"(claude|chatgpt|gpt-?[45o]|glm|gemini|qwen|gemma|llama|"
+    r"deepseek|copilot|sonnet|opus)\b"
+    r"|\b(i am|i'?m)\s+(an?|the)\s+"
+    r"(ai|llm|large language model|language model|chatbot|assistant "
+    r"(?:model|developed|made|trained|created)\b[^.]{0,40})"
+    r"|\b(trained|made|created|developed|built)\s+(by|at)\s+"
+    r"(anthropic|openai|google|deepmind|meta|mistral|alibaba|z\.ai|"
+    r"zhipu|nvidia|x\.?ai)\b[^.]{0,30}\bby me\b"
+    r"|\bi\s+(was\s+)?(trained|fine-?tuned)\s+(by|on|with)\b"
+    r"|\bas an?\s+(ai|llm|language model)(?=[,.])"
+    r"|\bclaude\b\s+(here|speaking|at your service)"
+    r"|\boperated by\s+(anthropic|openai|z\.ai)\b"
+    r"|\bich\s+(bin|heiße|werde genannt)\s+"
+    r"(claude|chatgpt|gpt-?[45o]?|glm|gemini|qwen|gemma|llama|"
+    r"deepseek)\b"
+    r"|\bals\s+ki\b[^.]{0,40}\bich\b"
+    r"|\bich\s+(bin|werde)\s+(eine?|ein)\s+"
+    r"(ki|künstliche?\s+intelligenz|sprachmodell)\b"
+    r")",
+    re.IGNORECASE,
+)
+
+# Refusal phrasings that would teach the model to refuse. Anchored on
+# first-person refusal verbs; a task ABOUT moderation policy may quote
+# refusals third-person — those don't match.
+REFUSAL_RE = re.compile(
+    r"("
+    r"\bi\s+(can'?t|cannot|can not|won'?t|will not|am unable to|"
+    r"'?m not able to)\s+"
+    r"(help|assist|comply|provide|create|generate|write|build)\b"
+    r"(\s+(that|this|with (that|this)))?"
+    r"|\bi\s+(don'?t|do not)\s+feel\s+comfortable\b"
+    r"|\b(against|outside)\s+my\s+(guidelines|principles|"
+    r"programming|policies)\b"
+    r"|\bi'?m\s+sorry,?\s+but\s+i\s+(can'?t|cannot|won'?t)\b"
+    r"|\bi\s+must\s+(respect|follow)\s+my\s+"
+    r"(guidelines|principles|programming)\b"
+    r"|\bich\s+(kann|kann nicht|werde)\s+.*(nicht helfen|"
+    r"nicht unterstützen|nicht unterstützen bei)\b"
+    r"|\bich\s+(kann|mag)\s+das\s+nicht\b"
+    r"|\bdas\s+verstößt\s+gegen\s+meine\b"
+    r"|\bmeine\s+richtlinien\b[^.]{0,20}\bverbieten\b"
+    r")",
+    re.IGNORECASE,
+)
+
+# Refusals in the opening task (user explicitly asks for something
+# harmful and the source model refused) are visible in USER rows too —
+# but we never edit user rows, so a harmful-request trace is not
+# promotable at all; that class dies at triage/judge level instead.
+
+
+def _find_hits(content: str, rx: re.Pattern) -> list[str]:
+    return [m.group(0)[:100] for m in rx.finditer(content)]
+
+
+def validate_identity(
+    messages: list,
+    *,
+    identity_rx: Optional[re.Pattern] = None,
+    refusal_rx: Optional[re.Pattern] = None,
+) -> list[str]:
+    """Return gate errors for assistant identity/refusal text ([] = ok).
+
+    ``messages`` may be Trace.messages (Message objects with .role/
+    .content) or plain dicts (role/content keys) — the audit calls this
+    with dicts, the improver with Messages.
+    """
+    irx = identity_rx or IDENTITY_RE
+    rrx = refusal_rx or REFUSAL_RE
+    errors: list[str] = []
+    for i, m in enumerate(messages):
+        role = m.get("role") if isinstance(m, dict) else m.role
+        if role != "assistant":
+            continue
+        c = (m.get("content") if isinstance(m, dict) else m.content) or ""
+        tc = m.get("tool_calls") if isinstance(m, dict) else m.tool_calls
+        texts = [c] if c else []
+        for call in tc or []:
+            if isinstance(call, dict):
+                args = call.get("arguments", call.get("args"))
+            else:
+                args = getattr(call, "arguments", None)
+                if args is None:
+                    args = getattr(call, "args", None)
+            if isinstance(args, str):
+                texts.append(args)
+            elif args:
+                texts.append(json.dumps(args, ensure_ascii=False))
+        for c2 in texts:
+            if not c2:
+                continue
+            for hit in _find_hits(c2, irx):
+                errors.append(
+                    f"[{i}] identity self-claim in assistant text/tool args: {hit!r}")
+            for hit in _find_hits(c2, rrx):
+                errors.append(
+                    f"[{i}] refusal phrasing in assistant text/tool args: {hit!r}")
+    return errors

traceforge/improver/__init__.py

@@ -0,0 +1,4 @@
+from traceforge.improver.harness import ImproverResult, improve_trace
+from traceforge.improver.prompt import build_prompt
+
+__all__ = ["ImproverResult", "improve_trace", "build_prompt"]

traceforge/improver/digest_prompts.py

@@ -0,0 +1,105 @@
+"""Prompts for the long-trace (map-digest-write) pipeline.
+
+Heavy sessions (up to ~10M tokens) can never fit into one context. The
+pipeline: digest each chunk in parallel -> merge digests into a chunk map
+-> writer builds the ideal trace from the map, looking up exact verbatim
+tool outputs from on-disk chunk files by reference.
+"""
+from __future__ import annotations
+
+CHUNK_DIGEST_HEADER = """You are a trace-digest agent. You see ONE chunk of a
+long real agent session. boil it down for a downstream writer agent that
+will rewrite the whole session as a flawless "ideal trace".
+
+Produce JSON with EXACTLY these keys, written to: {digest_path}
+{{
+  "chunk": <chunk number>,
+  "task_scope": "<what part of the overall task this chunk covers>",
+  "narrative": "<compact ordered summary of what happened: decisions,
+                 tool calls made (name + key args), results seen,
+                 user feedback received>",
+  "mistakes": [
+    {{"kind": "failed_tool_call|retry_loop|wrong_turn|redundant|user_feedback",
+      "detail": "<what went wrong>", "turn_ref": "<[chunk-local msg idx]>"}}
+  ],
+  "key_outputs": [
+    {{"ref": "chunk{n}-{i}", "tool": "<name>", "significance": "<why it
+      matters>", "loc": "<file/section the output is about>"}}
+  ],
+  "claim_worthy": ["<external-world facts asserted (versions, prices,
+      paper ids) worth fact-checking>"]
+}}
+
+RULES:
+- Story-first: the downstream writer only sees your digest, not the chunk.
+  Everything load-bearing must appear in narrative/ key_outputs.
+- Mistakes matter most: every failed call, retry, wrong turn, and user
+  correction must be listed — those are what the ideal path must avoid.
+- Do NOT include raw tool outputs in digest. Reference them as refs
+  (chunk{n}-{i}); the writer resolves refs against on-disk chunk files.
+- ideal_path: describe in "narrative" what SHOULD have happened in this
+  chunk's scope (perfect first-time route per guidelines).
+- No preamble: write the JSON, validate it parses when re-read, then reply
+  "DIGEST DONE chunk {n}".
+"""
+
+MAP_MERGE_HEADER = """You are a trace-map merge agent. You receive digests
+of ALL chunks of one long real agent session, in order. Build the unified
+session map for the ideal-trace writer.
+
+Write JSON to: {map_path}
+{{
+  "task": "<the one-sentence task the user wanted, verbatim from the
+           first user message>",
+  "first_user_message": "<the EXACT, COMPLETE first user message text>",
+  "structure": [ "<ordered story beats: what the ideal path needs to do,
+                  step by step, referencing key_outputs refs>" ],
+  "mistakes": [ "<ALL mistakes across chunks, deduplicated, in order>" ],
+  "key_outputs": [ "<all key_output refs, deduplicated, with tool+loc>" ],
+  "claim_worthy": [ "<all claim-worthy external facts>" ],
+  "chunk_index": [ "<per-chunk digest files, in order>" ]
+}}
+
+RULES:
+- Deduplicate conservative: keep every load-bearing item; drop only
+  literal repeats introduced by chunk overlap.
+- first_user_message must be VERBATIM and COMPLETE — the writer will
+  reproduce it byte-exact in the ideal trace.
+- ideal story: structure = the perfect-first-time route through the whole
+  session (per guidelines: mistakes removed, minimal necessary calls).
+- Write the JSON, validate it parses, reply "MAP DONE".
+"""
+
+WRITER_LONG_HEADER = """You are the ideal-trace writer for a LONG session.
+You get: (1) the session map (digests merged), (2) access to on-disk chunk
+files that hold the raw rendered chunks, (3) the guidelines.
+
+Your job: write the ideal-world version of the whole session as JSON to:
+{output_path}
+
+Procedure:
+1. Read the guidelines file: {guidelines_path}
+2. Read the session map at: {map_path}
+3. Read guidelines §3 pillars. Then look up chunk data on demand:
+   - The chunk files live at: {chunk_dir} (files chunk_001.txt, ...).
+   - Use refs from the map's key_outputs to locate the exact tool outputs
+     you want to reuse VERBATIM (search the chunk files).
+   - Read a chunk file only when you need exact content from it. Do not
+     read all chunk files — only what refs point you to.
+4. Write the ideal trace (same JSON schema as the improver contract:
+   messages array with role/content/tool_calls/tool_call_id/tool_name).
+   Apply ALL quality pillars. Reuse tool outputs verbatim when the ideal
+   path makes the same call (P4); resolve via refs.
+5. Verify: re-read what you wrote; check JSON; check tool pairing; check
+   the first user message matches the map's first_user_message byte-exact.
+6. Reply "WRITER DONE" + a summary paragraph.
+
+Scope limits (unchanged): no environment archaeology, no subagents —
+chunk files and guidelines are your only inputs.
+"""
+
+DIGEST_SUFFIX = """
+
+REAL TRACE CHUNK {n}:
+{chunk}
+"""

traceforge/improver/harness.py

@@ -0,0 +1,850 @@
+"""Runs the improver as a Hermes Agent one-shot session (the harness).
+
+Design: for each trace we spawn
+
+    hermes chat --query-file <prompt.txt> --in <workdir> -Q
+        [--improver-model M] [--max-turns N] [--pass-session-id]
+
+The improver is a full agent: it can read the guidelines file, inspect the
+workdir, write its output with a tool, and re-validate it — agentic
+self-improvement inside the generation loop. The harness never trusts the
+improver's chat reply; it reads back the JSON file the improver wrote,
+then structural validation + repair loop run outside the model.
+"""
+from __future__ import annotations
+
+import dataclasses
+import json
+import os
+import shutil
+import subprocess
+import time
+from typing import Optional
+
+from traceforge import chunker
+from traceforge.config import Config
+from traceforge.grounding import CLIP_MARKER_RE as CLIP_MARKER_PROBE
+from traceforge.improver.prompt import build_prompt, write_prompt_file
+from traceforge.trace import Message, Trace, json_loads_lenient
+
+_TOOL_TAIL_ERROR = "last message role is 'tool' (expected assistant)"
+_EMPTY_ROW_RE = "message with empty content and no tool calls"
+
+
+def _only_tool_tail(errs: list[str]) -> bool:
+    """True when the artifact's ONLY defect is that it ends on a tool row."""
+    return bool(errs) and all(_TOOL_TAIL_ERROR in e for e in errs)
+
+
+def _drop_empty_assistant_rows(improved: Trace) -> bool:
+    """Remove assistant rows with no content and no calls. Returns changed.
+
+    The writer emits these as separators when it loses the thread. They
+    carry zero signal and a trainee must never learn to produce them, so
+    dropping is strictly better than paying a repair round to rewrite the
+    whole trace around one blank row.
+    """
+    kept = [m for m in improved.messages
+            if not (m.role == "assistant" and not m.tool_calls
+                    and not (m.content or "").strip())]
+    if len(kept) == len(improved.messages):
+        return False
+    improved.messages = kept
+    return True
+
+
+def _append_synth_final(cfg: Config, improved: Trace,
+                        source: Trace) -> Optional[Trace]:
+    """Close a trace that the writer left mid-call with a synthesized answer.
+
+    The writer stops after a tool call in ~3% of traces (output cap or turn
+    cap), leaving an artifact that is one row short of valid. A full repair
+    round rewrites the whole trace and usually reproduces the same cut, so
+    the cheap fix is the targeted one: ask for the closing answer only,
+    grounded in the tail that already exists. Same helper the splice path
+    uses for raw traces that were cut off mid-run.
+    """
+    from traceforge.splice import _resolve_endpoint, _synthesize_final_answer
+
+    try:
+        base, key = _resolve_endpoint(cfg)
+    except Exception as e:
+        print(f"  tool-tail synth skipped: endpoint {type(e).__name__}: {e}")
+        return None
+    ans, err = _synthesize_final_answer(
+        cfg, improved, source.first_task_message() or "", base, key)
+    if not ans:
+        print(f"  tool-tail synth failed: {(err or 'empty answer')[:120]}")
+        return None
+    improved.messages.append(Message(role="assistant", content=ans))
+    return improved
+
+CLIP_MARKER_ERROR = (
+    "artifact contains rendering clip markers (…[+N chars clipped]) "
+    "with no verifiable counts — reviewers reject these. Rewrite the tool "
+    "results verbatim from outputs.jsonl (contiguous portions fine)."
+)
+
+# Per-workdir file recording which gate rounds already PASSED for which
+# artifact content digest (see _load_gate_cache).
+GATE_CACHE_FILE = "gate_cache.json"
+
+# Bump whenever a JUDGE/GATE STANDARD changes (new pillar rule, ANSWER
+# note wording that changes verdicts, regex gate changes) — cached
+# passes recorded under an older standard must not replay. The value
+# rides in the cache file; _load_gate_cache discards stale files.
+GATE_CACHE_STANDARDS = "2026-10-08.1"  # +answer-focus rule (P1 FOCUS, over-answering = poor)  # +write-JSON deliverable forms, tool-tail carry, exact-args dup reads  # +mixed-turn surgical drops, usage-error detect, planner >=2 + write-fix chains  # +identity scans tool args (Co-Authored-By), degenerate prose gate  # +judge sees reasoning channel
+
+
+def _clip_markers_present(candidate) -> bool:
+    """True if any message content carries a rendering clip marker."""
+    blob = "\n".join((m.content or "") for m in candidate.messages)
+    return bool(CLIP_MARKER_PROBE.search(blob))
+
+
+def _artifact_digest(candidate) -> str:
+    """Stable content hash of an improved artifact.
+
+    Gate passes are keyed on this: structural/grounding/verify/review
+    PASSes are only replayed from cache when the artifact byte-content is
+    unchanged (the file may still have been rewritten with identical
+    bytes — that's fine, it's the same artifact).
+    """
+    import hashlib
+
+    h = hashlib.sha256()
+    for m in candidate.messages:
+        h.update((m.role or "").encode("utf-8"))
+        h.update(b"\x00")
+        h.update((m.content or "").encode("utf-8"))
+        h.update(b"\x01")
+        if m.tool_call_id:
+            h.update(m.tool_call_id.encode("utf-8"))
+        h.update(b"\x01")
+        if m.tool_name:
+            h.update(m.tool_name.encode("utf-8"))
+        h.update(b"\x02")
+        if m.tool_calls:
+            for tc in m.tool_calls:
+                h.update((tc.id or "").encode("utf-8"))
+                h.update((tc.name or "").encode("utf-8"))
+                h.update((tc.arguments or "").encode("utf-8"))
+                h.update(b"\x03")
+    return h.hexdigest()
+
+
+class ImproverResult:
+    def __init__(
+        self,
+        *,
+        session_id: str,
+        ok: bool,
+        improved: Optional[Trace],
+        reply: str,
+        hermes_session_id: Optional[str] = None,
+        errors: Optional[list[str]] = None,
+        attempts: int = 0,
+        duration_s: float = 0.0,
+    ):
+        self.session_id = session_id          # source trace session id
+        self.ok = ok
+        self.improved = improved
+        self.reply = reply
+        self.hermes_session_id = hermes_session_id
+        self.errors = errors or []
+        self.attempts = attempts
+        self.duration_s = duration_s
+
+    def to_dict(self) -> dict:
+        return {
+            "session_id": self.session_id,
+            "ok": self.ok,
+            "errors": self.errors,
+            "hermes_session_id": self.hermes_session_id,
+            "attempts": self.attempts,
+            "duration_s": round(self.duration_s, 1),
+        }
+
+
+def _run_hermes(
+    cfg: Config, prompt_path: str, workdir: str
+) -> tuple[int, str, Optional[str], str]:
+    """Invoke hermes chat one-shot; returns (exit_code, stdout, session_id)."""
+    cmd = [
+        "hermes", "chat",
+        "--query-file", prompt_path,
+        "--in", workdir,
+        "-Q",
+        "--format", "text",
+        "--pass-session-id",
+    ]
+    if cfg.model:
+        cmd += ["-m", cfg.model]
+    if cfg.provider:
+        cmd += ["--provider", cfg.provider]
+    if cfg.max_turns:
+        cmd += ["--max-turns", str(cfg.max_turns)]
+    if cfg.run_budget:
+        cmd += ["--run-budget", str(cfg.run_budget)]
+    if cfg.skills:
+        cmd += ["-s", ",".join(cfg.skills)]
+    proc = subprocess.run(
+        cmd,
+        capture_output=True,
+        text=True,
+        timeout=cfg.harness_timeout_s,
+        cwd=workdir,
+    )
+    session_id = None
+    for line in (proc.stdout or "").splitlines():
+        if line.strip().startswith("session_id:"):
+            session_id = line.split(":", 1)[1].strip() or None
+    return proc.returncode, proc.stdout, session_id, proc.stderr
+
+
+def _parse_improved(path: str) -> Optional[Trace]:
+    """Parse the improver-written improved.json into a Trace, leniently."""
+    if not os.path.exists(path):
+        return None
+    with open(path, "r", encoding="utf-8") as f:
+        raw = f.read()
+    # tolerate fenced code blocks from chatty models
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = raw.split("\n", 1)[1] if "\n" in raw else ""
+        if raw.rstrip().endswith("```"):
+            raw = raw.rstrip()[: -3]
+    data = json_loads_lenient(raw)
+    if not isinstance(data, dict):
+        return None
+    msgs = data.get("messages")
+    if not isinstance(msgs, list):
+        return None
+    from traceforge.trace import Message, ToolCall
+
+    out = []
+    for d in msgs:
+        if not isinstance(d, dict) or "role" not in d:
+            return None
+        m = {
+            "role": d["role"],
+            "content": d.get("content"),
+            "reasoning": d.get("reasoning"),
+            "tool_call_id": d.get("tool_call_id"),
+            "tool_name": d.get("tool_name"),
+        }
+        m = {k: v for k, v in m.items() if v is not None}
+        if "tool_calls" in d and isinstance(d["tool_calls"], list):
+            m["tool_calls"] = [
+                {
+                    "id": str(tc.get("id", "")),
+                    "name": str(tc.get("name", "")),
+                    "arguments": tc.get("arguments") or "{}",
+                }
+                for tc in d["tool_calls"]
+                if isinstance(tc, dict)
+            ]
+        out.append(Message.from_dict(m))
+    return Trace(
+        session_id="improved-placeholder",  # set by caller
+        messages=out,
+        improved=True,
+    )
+
+
+def _append_status(cfg: Config, sid: str, status: dict) -> None:
+    """Append a stage-level status row to the manifest."""
+    import json as _json
+
+    os.makedirs(os.path.dirname(cfg.manifest_path), exist_ok=True)
+    row = {"session_id": sid, **status}
+    with open(cfg.manifest_path, "a", encoding="utf-8") as f:
+        f.write(_json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def _load_gate_cache(workdir: str) -> dict:
+    """Read the workdir's gate-pass cache (digest -> list of gate rows).
+
+    Batches die at arbitrary points (turn interruptions, terminal close,
+    SIGTERM to the worker pool); without this, every verify/review round
+    already PAID FOR on an unchanged artifact is re-run from scratch on
+    the next resume. The cache is keyed by the artifact's content digest
+    so a rewritten (different) artifact never inherits stale passes.
+    Cached passes from an OLDER standards version are discarded — judge
+    rules evolve, and a verdict recorded under old rules must not
+    replay against new ones.
+    """
+    path = os.path.join(workdir, GATE_CACHE_FILE)
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            data = json.load(f)
+        if not isinstance(data, dict):
+            return {}
+        if data.get("standards") != GATE_CACHE_STANDARDS:
+            return {}
+        return data
+    except (OSError, json.JSONDecodeError):
+        return {}
+
+
+def _save_gate_cache(workdir: str, cache: dict) -> None:
+    """Persist the gate-pass cache atomically (workdir, per trace)."""
+    path = os.path.join(workdir, GATE_CACHE_FILE)
+    tmp = path + ".tmp"
+    cache = dict(cache)
+    cache["standards"] = GATE_CACHE_STANDARDS
+    with open(tmp, "w", encoding="utf-8") as f:
+        json.dump(cache, f, ensure_ascii=False)
+    os.replace(tmp, path)
+
+
+def _cache_gate_pass(cache: dict, digest: str, stage: str, round_no: int) -> None:
+    """Record a gate PASS for one artifact digest."""
+    rows = cache.setdefault(digest, [])
+    key = f"{stage}:{round_no}"
+    if key not in rows:
+        rows.append(key)
+
+
+def _acquire_workdir_lock(workdir: str) -> bool:
+    """O_EXCL pid lockfile; locks left by dead processes are stolen.
+
+    Two concurrent improve batches working the same sid corrupt each
+    other (shared improved.json / verdict.json — the verifier unlinks its
+    verdict file before each run) and burn double model time. The lock
+    makes one-improver-per-workdir structural, not a user discipline.
+    """
+    lock_path = os.path.join(workdir, ".lock")
+    my_pid = str(os.getpid())
+    for _ in range(3):
+        try:
+            fd = os.open(lock_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)
+            try:
+                os.write(fd, my_pid.encode())
+            finally:
+                os.close(fd)
+            return True
+        except FileExistsError:
+            try:
+                other = open(lock_path, encoding="utf-8").read().strip()
+            except OSError:
+                continue
+            pid = int(other) if other.isdigit() else -1
+            alive = False
+            if pid > 0:
+                try:
+                    os.kill(pid, 0)
+                    alive = True
+                except ProcessLookupError:
+                    alive = False
+                except PermissionError:
+                    alive = True  # exists, not ours: assume alive
+                except OverflowError:
+                    alive = False
+            if alive:
+                return False
+            try:
+                os.unlink(lock_path)  # stale lock from a dead run: steal
+            except OSError:
+                pass
+    return False
+
+
+def _release_workdir_lock(workdir: str) -> None:
+    """Drop our own lock (never another live run's)."""
+    lock_path = os.path.join(workdir, ".lock")
+    try:
+        with open(lock_path, encoding="utf-8") as f:
+            if f.read().strip() == str(os.getpid()):
+                os.unlink(lock_path)
+    except OSError:
+        pass
+
+
+def _write_outputs_index(
+    cfg: Config, trace: Trace, workdir: str, max_chars: int = 250_000
+) -> str:
+    """Full real tool outputs as outputs.jsonl, keyed by rendering index.
+
+    The compact rendering clips each tool result to a short head; P4
+    demands verbatim reuse, so the improver needs the unclipped text.
+    One JSON object per line: {"i": N, "tool", "args", "output"} where N
+    is the same [N] message index the rendering shows — the improver can
+    correlate a clipped result in its prompt with the full text here.
+
+    max_chars is a byte-blow safety valve (measured corpus max real
+    output: ~111KB), NOT a compression budget: the 20k cap this used to
+    apply hid exactly the mid-tail evidence (p95 = 20k) that P4 demands
+    the improver quote — on exactly the big traces where review kills
+    artifacts — forcing composed pseudo-outputs. Outputs beyond the
+    safety valve carry an honest total-marker, never a fake count.
+    """
+    path = os.path.join(workdir, "outputs.jsonl")
+    arg_of: dict[str, str] = {}
+    for m in trace.messages:
+        for tc in m.tool_calls:
+            arg_of[tc.id] = tc.arguments or "{}"
+    with open(path, "w", encoding="utf-8") as f:
+        for i, m in enumerate(trace.messages, 1):
+            if m.role != "tool" or not m.content:
+                continue
+            out = m.content
+            if len(out) > max_chars:
+                out = out[:max_chars] + f" …[+{len(out) - max_chars} chars total]"
+            f.write(json.dumps({
+                "i": i,
+                "tool": m.tool_name or "?",
+                "args": arg_of.get(m.tool_call_id or "", "")[:400],
+                "output": out,
+            }, ensure_ascii=False) + "\n")
+    return path
+
+
+def _generate_single(cfg, trace, rendered, workdir, output_path, errors, prev_path, sid):
+    """One single-shot improver invocation. Returns (candidate, reply, hsid, code, stderr) or raises Timeout."""
+    from traceforge.classify import detect_language
+    prompt = build_prompt(
+        rendered_trace=rendered,
+        guidelines_path=cfg.guidelines_path,
+        output_path=output_path,
+        outputs_path=os.path.join(workdir, "outputs.jsonl"),
+        errors=errors,
+        prev_path=prev_path,
+        language=detect_language(trace.first_task_message() or ""),
+    )
+    prompt_path = write_prompt_file(cfg.prompts_dir, sid, prompt)
+    try:
+        code, stdout, hsid, stderr = _run_hermes(cfg, prompt_path, workdir)
+    except subprocess.TimeoutExpired as e:
+        raise e
+    reply = (stdout or "").strip()
+    return _parse_improved(output_path), reply, hsid, code, stderr
+
+
+def improve_trace(
+    cfg: Config,
+    trace: Trace,
+    *,
+    max_attempts: int = 2,
+    validator=None,
+    status_cb=None,
+) -> ImproverResult:
+    """Run the agentic improver on one trace, with a validation repair loop.
+
+    Long traces (rendered size > cfg.chunk_max_chars) go through the staged
+    map-digest-write pipeline instead of the single shot. Every admitted
+    trace passes: N independent verification rounds (web fact-check) and
+    M independent review rounds (quality-pillar review) — all must PASS.
+
+    ``status_cb`` (optional) receives short human stage strings
+    ("improving", "digest", "verify 1/2", ...) for live progress display.
+    """
+    from traceforge.trace import Trace as _T
+    from traceforge.validate import validate_structural
+    from traceforge.grounding import validate_grounding
+    from traceforge.improver.verify import run_verifier
+    from traceforge.improver.review import run_review
+
+    def _status(s: str) -> None:
+        if status_cb is not None:
+            status_cb(s)
+
+    started = time.time()
+    sid = trace.session_id
+    workdir = os.path.join(cfg.improver_workdir, sid)
+    os.makedirs(workdir, exist_ok=True)
+    # One improver per workdir, structurally: concurrent batches on the
+    # same sid race on improved.json/verdict.json and pay the model twice.
+    if not _acquire_workdir_lock(workdir):
+        return ImproverResult(
+            session_id=sid, ok=False, improved=None, reply="",
+            hermes_session_id=None,
+            errors=["workdir locked by a live improver run — skipping"],
+            attempts=0, duration_s=0.0,
+        )
+    try:
+        return _improve_locked(cfg, trace, max_attempts, status_cb)
+    finally:
+        _release_workdir_lock(workdir)
+
+
+def _improve_locked(
+    cfg: Config, trace: Trace, max_attempts: int, status_cb
+) -> ImproverResult:
+    from traceforge.trace import Trace as _T
+    from traceforge.validate import validate_structural
+    from traceforge.grounding import validate_grounding
+    from traceforge.improver.verify import run_verifier
+    from traceforge.improver.review import run_review
+
+    def _status(s: str) -> None:
+        if status_cb is not None:
+            status_cb(s)
+
+    started = time.time()
+    sid = trace.session_id
+    workdir = os.path.join(cfg.improver_workdir, sid)
+    rendered = trace.render_compact(
+        user_chars=cfg.user_chars,
+        assistant_chars=cfg.assistant_chars,
+        tool_result_chars=cfg.tool_result_chars,
+        tool_args_chars=cfg.tool_args_chars,
+        include_reasoning=cfg.include_reasoning,
+    )
+    output_path = os.path.join(workdir, "improved.json")
+    # Full unclipped tool outputs on disk: P4's verbatim-reuse rule is
+    # only satisfiable when the improver can actually see the full text.
+    outputs_path = _write_outputs_index(cfg, trace, workdir)
+
+    errors: list[str] = []
+    prev_path: Optional[str] = None
+    attempts = 0
+    improved: Optional[Trace] = None
+    reply = ""
+    hermes_session_id: Optional[str] = None
+    # long-trace attempts don't run hermes chat directly (stages do); the
+    # loop below must not touch single-shot navigation vars on that path
+    code = 0
+    stderr = ""
+
+    for attempt in range(1, max_attempts + 1):
+        attempts = attempt
+        candidate = None
+        # Fast-path: when a previous run already left an artifact in this
+        # workdir, re-adjudicate it first (regate). Only pay the model for
+        # a fresh write when nothing usable exists on disk. This makes
+        # "rerun after a gate change" nearly free instead of a full
+        # re-write of every trace.
+        if attempt == 1 and os.path.exists(output_path):
+            _status("regate")
+            candidate = _parse_improved(output_path)
+            # Static pre-flight: an artifact that still carries rendering
+            # clip markers is dead on arrival at review (reviewers kill
+            # invented counts; grounding silently strips them). Fail it
+            # here — one static check instead of 4 deep-thinker
+            # sessions re-discovering it.
+            if candidate is not None and _clip_markers_present(candidate):
+                candidate = None
+                errors = [CLIP_MARKER_ERROR]
+                # keep the failed artifact as the repair reference: the
+                # next attempt must FIX it (REPAIR_HEADER + errors), not
+                # blind-rewrite from a fresh TASK_HEADER that would
+                # silently drop the specific clip-marker error.
+                prev_path = output_path
+        if candidate is not None:
+            reply = "REGATE (existing artifact)"
+            code = 0
+            stderr = ""
+        elif len(rendered) <= cfg.chunk_max_chars:
+            _status("improving" if attempt == 1 else f"repair {attempt}")
+            try:
+                candidate, reply_one, hsid, code, stderr = _generate_single(
+                    cfg, trace, rendered, workdir, output_path, errors,
+                    prev_path, sid,
+                )
+            except subprocess.TimeoutExpired:
+                errors = [f"harness timeout after {cfg.harness_timeout_s}s"]
+                continue
+            reply = reply_one
+            hermes_session_id = hsid
+            if reply.startswith("REJECTED"):
+                errors = [f"rejected by improver: {reply[:300]}"]
+                break
+        else:
+            # long-trace staged pipeline: digest -> map -> writer
+            from traceforge.improver import longtrace
+
+            chunk_dir = os.path.join(workdir, "chunks")
+            if prev_path is None:
+                _status("digest+map")
+                chunks = chunker.chunk_trace(
+                    trace,
+                    max_chars=cfg.chunk_max_chars,
+                    overlap_messages=cfg.chunk_overlap_messages,
+                    render_kwargs={
+                        "user_chars": cfg.user_chars,
+                        "assistant_chars": cfg.assistant_chars,
+                        "tool_result_chars": cfg.tool_result_chars,
+                        "tool_args_chars": cfg.tool_args_chars,
+                        "include_reasoning": cfg.include_reasoning,
+                    },
+                )
+                if not chunks:
+                    errors = ["chunking produced no chunks"]
+                    continue
+                longtrace.write_chunks(chunk_dir, chunks, trace)
+                _status(f"digest 0/{len(chunks)}")
+                digests, derrs = longtrace.digest_all(
+                    cfg, trace, chunks, workdir,
+                    progress_cb=lambda done, total: _status(
+                        f"digest {done}/{total}"
+                    ),
+                )
+                if derrs:
+                    errors = [f"digest failures: {derrs}"]
+                    continue
+                _status(f"map ({len(chunks)} chunks)")
+                map_d, merr = longtrace.build_map(cfg, trace, digests, workdir)
+                if map_d is None:
+                    errors = [f"map merge failed: {merr}"]
+                    continue
+                _append_status(cfg, sid, {
+                    "stage": "map", "chunks": len(chunks),
+                    "approx_tokens": sum(c.approx_chars for c in chunks) // 4,
+                })
+            # writer (also used for repair attempts on long traces)
+            map_d = longtrace._json_file(os.path.join(workdir, "session_map.json"))
+            if map_d is None:
+                errors = ["session map missing after long-trace stage"]
+                continue
+            _status("writer")
+            candidate, werr = longtrace.run_writer(
+                cfg, trace, map_d, chunk_dir, workdir, output_path,
+                repair_errors=errors if prev_path else None,
+            )
+            if candidate is None:
+                errors = [f"long-trace writer failed: {werr}"]
+                continue
+            reply = "WRITER DONE"
+        # The artifact on disk is the source of truth — NOT the exit code.
+        # The improver may hit its turn cap after writing a perfectly valid
+        # file (e.g. right after a repair); validate what exists regardless.
+        candidate = _parse_improved(output_path)
+        if candidate is None:
+            if code != 0:
+                stderr_tail = f"; stderr: {stderr.strip()[-500:]}" if stderr else ""
+                errors = [
+                    f"hermes chat exited with code {code} and "
+                    f"improved.json missing/unparseable{stderr_tail}"
+                ]
+            else:
+                errors = [
+                    "improved.json missing, unparseable, or missing 'messages'"
+                ]
+            continue
+        # Static clip-marker gate on EVERY path, not just regate: writers
+        # and repairs can still copy markers out of the clipped rendering.
+        # A marker artifact is dead on arrival at review (and grounding
+        # silently forgives it) — fail fast here instead of burning
+        # verifier + reviewer sessions on it.
+        if _clip_markers_present(candidate):
+            errors = [f"clip-marker gate: {CLIP_MARKER_ERROR}"]
+            # repair must fix THIS artifact (REPAIR_HEADER + errors), not
+            # blind-rewrite from a TASK_HEADER that drops the specifics
+            prev_path = output_path
+            continue
+        prev_path = output_path
+        improved = candidate
+        # Pipeline framing markers ("[trace segment k of n]", "[subagent
+        # brief — role: ...]") belong to the improver's INPUT; strip them
+        # before the gates so the echo check compares against the real task
+        # and the promoted artifact carries no scaffolding.
+        from traceforge.validate import strip_scaffold_markers
+        if strip_scaffold_markers(improved):
+            improved.save(output_path)
+        errs = validate_structural(improved)
+        if errs and any(_EMPTY_ROW_RE in e for e in errs):
+            # blank assistant rows are pure noise — drop them and re-check
+            # rather than paying a full rewrite to remove one empty row.
+            # Runs even when other errors are present: a trace with a blank
+            # row AND a tool tail must get both fixes, not neither.
+            if _drop_empty_assistant_rows(improved):
+                improved.save(output_path)  # repair must start from the fix
+                errs = validate_structural(improved)
+                if not errs:
+                    _status("dropped empty assistant rows")
+        if _only_tool_tail(errs):
+            # writer stopped mid-call: one row short of valid. Synthesize
+            # the closing answer instead of paying a full repair round
+            # that reproduces the same cut (measured: 49/1535 traces).
+            fixed = _append_synth_final(cfg, improved, trace)
+            if fixed is not None:
+                improved = fixed
+                # persist BEFORE re-validating: a later repair round reads
+                # prev_path from disk, so an in-memory-only fix is thrown
+                # away and the writer reproduces the same tool tail.
+                improved.save(output_path)
+                errs = validate_structural(improved)
+                if not errs:
+                    _status("final answer (tool-tail synth)")
+        if not errs:
+            errs = validate_grounding(improved, trace)
+        if not errs and (cfg.verify or cfg.review):
+            # Gate-pass cache: rounds that already PASSED for THIS exact
+            # artifact content are replayed instead of re-paid. Survives
+            # batch interruptions (turn cap, SIGTERM, terminal close); a
+            # rewritten artifact has a different digest and re-runs all
+            # rounds fresh.
+            digest = _artifact_digest(improved)
+            gate_cache = _load_gate_cache(workdir)
+            gate_pass_keys = set(gate_cache.get(digest, []))
+
+            def _gate_pass(stage: str, round_no: int) -> None:
+                _cache_gate_pass(gate_cache, digest, stage, round_no)
+                _save_gate_cache(workdir, gate_cache)
+
+            verify_failures: list[str] = []
+            # ---- N independent verification rounds (web fact-check)
+            # Strict admission: every round must end PASS. A transient ERROR
+            # (timeout, unparseable verdict) gets ONE in-place retry; a
+            # second ERROR surfaces as an error row — never silent
+            # under-verification, never a model repair for infra issues.
+            if cfg.verify:
+                passes = 0
+                vr = 1
+                while vr <= cfg.verify_rounds:
+                    if f"verify:{vr}" in gate_pass_keys:
+                        passes += 1  # already adjudicated on identical bytes
+                        _status(f"verify {vr}/{cfg.verify_rounds} (cached)")
+                        _append_status(cfg, sid, {
+                            "stage": "verify", "round": vr,
+                            "verdict": "PASS", "cached": True,
+                        })
+                        vr += 1
+                        continue
+                    _status(f"verify {vr}/{cfg.verify_rounds}")
+                    vres = run_verifier(cfg, output_path, workdir)
+                    if vres.verdict == "PASS":
+                        passes += 1
+                        _gate_pass("verify", vr)
+                        _append_status(cfg, sid, {
+                            "stage": "verify", "round": vr,
+                            "verdict": "PASS",
+                            "claims_checked": vres.claims_checked,
+                        })
+                        vr += 1
+                        continue
+                    if vres.verdict == "FAIL":
+                        claims = "\n".join(
+                            f'- "{u.get("claim", "?")}" — {u.get("why", "?")}'
+                            for u in vres.unsupported[:10]
+                        )
+                        verify_failures.append(
+                            f"[verify r{vr}] external claims unsupported:\n{claims}"
+                        )
+                        _append_status(cfg, sid, {
+                            "stage": "verify", "round": vr, "verdict": "FAIL",
+                            "claims_checked": vres.claims_checked,
+                        })
+                        break  # concrete FAIL: repair next attempt
+                    # transient ERROR: one in-place retry of the same round
+                    retry = run_verifier(cfg, output_path, workdir)
+                    if retry.verdict == "PASS":
+                        passes += 1
+                        _gate_pass("verify", vr)
+                        _append_status(cfg, sid, {
+                            "stage": "verify", "round": vr,
+                            "verdict": "PASS", "retried": True,
+                            "claims_checked": retry.claims_checked,
+                        })
+                        vr += 1
+                        continue
+                    if retry.verdict == "FAIL":
+                        claims = "\n".join(
+                            f'- "{u.get("claim", "?")}" — {u.get("why", "?")}'
+                            for u in retry.unsupported[:10]
+                        )
+                        verify_failures.append(
+                            f"[verify r{vr}] external claims unsupported:\n{claims}"
+                        )
+                        _append_status(cfg, sid, {
+                            "stage": "verify", "round": vr, "verdict": "FAIL",
+                            "retried": True,
+                            "claims_checked": retry.claims_checked,
+                        })
+                        break
+                    verify_failures.append(
+                        f"[verify r{vr}] infra error x2: "
+                        f"{vres.error or 'verifier error'} / "
+                        f"{retry.error or 'verifier error'}"
+                    )
+                    _append_status(cfg, sid, {
+                        "stage": "verify", "round": vr, "verdict": "ERROR",
+                        "error": (
+                            f"{vres.error or 'verifier error'} / "
+                            f"{retry.error or 'verifier error'}"
+                        )[:300],
+                    })
+                    break
+            # ---- M independent review rounds (quality pillars)
+            review_failures: list[str] = []
+            task_snip = trace.first_task_message() or "(no user message)"
+            if not verify_failures and cfg.review:
+                for rr in range(1, cfg.review_rounds + 1):
+                    if f"review:{rr}" in gate_pass_keys:
+                        _status(f"review {rr}/{cfg.review_rounds} (cached)")
+                        _append_status(cfg, sid, {
+                            "stage": "review", "round": rr, "verdict": "PASS",
+                            "cached": True,
+                        })
+                        continue
+                    _status(f"review {rr}/{cfg.review_rounds}")
+                    rres = run_review(
+                        cfg, output_path, task_snip, workdir, rr
+                    )
+                    if not rres.ok() and rres.verdict == "ERROR":
+                        rres = run_review(
+                            cfg, output_path, task_snip, workdir, rr
+                        )  # one in-place retry on transient infra error
+                    if rres.ok():
+                        _gate_pass("review", rr)
+                        _append_status(cfg, sid, {
+                            "stage": "review", "round": rr, "verdict": "PASS",
+                        })
+                        continue
+                    if rres.verdict == "FAIL":
+                        review_failures.append(
+                            "[review r%d] pillar violations:\n%s"
+                            % (rr, "\n".join(rres.violations[:10]))
+                        )
+                        _append_status(cfg, sid, {
+                            "stage": "review", "round": rr, "verdict": "FAIL",
+                        })
+                        break  # concrete FAIL: repair next attempt
+                    review_failures.append(
+                        f"[review r{rr}] infra error x2: {rres.error or 'review error'}"
+                    )
+                    _append_status(cfg, sid, {
+                        "stage": "review", "round": rr, "verdict": "ERROR",
+                        "error": (rres.error or "review error")[:300],
+                    })
+                    break
+            errs = verify_failures + review_failures
+        if errs:
+            # Infra failures (verifier/reviewer down) are NOT content errors:
+            # a model repair cannot fix them. Fail fast without re-running
+            # the improver so no model time is wasted on infrastructure.
+            if any("infra error x2" in e for e in errs):
+                return ImproverResult(
+                    session_id=sid, ok=False, improved=None, reply=reply,
+                    hermes_session_id=hermes_session_id,
+                    errors=errs, attempts=attempts,
+                    duration_s=time.time() - started,
+                )
+            errors = errs + (
+                [f"note: hermes exit code {code}"] if code != 0 else []
+            )
+            continue
+        improved.session_id = f"ideal-{sid}"
+        improved.parent_session_id = sid
+        # cfg.model None = default profile; the exact model is recoverable
+        # from hermes_session_id (recorded in the manifest).
+        improved.model = cfg.model or "hermes-default"
+        improved.source = "hermes-one-shot"
+        return ImproverResult(
+            session_id=sid, ok=True, improved=improved, reply=reply,
+            hermes_session_id=hermes_session_id, errors=[],
+            attempts=attempts, duration_s=time.time() - started,
+        )
+
+    return ImproverResult(
+        session_id=sid, ok=False, improved=None, reply=reply,
+        hermes_session_id=hermes_session_id, errors=errors,
+        attempts=attempts, duration_s=time.time() - started,
+    )
+
+
+def cleanup_workdir(cfg: Config, sid: str, keep: bool = False) -> None:
+    wd = os.path.join(cfg.improver_workdir, sid)
+    if not keep and os.path.isdir(wd):
+        shutil.rmtree(wd, ignore_errors=True)

traceforge/improver/longtrace.py

@@ -0,0 +1,182 @@
+"""Long-trace pipeline: chunk -> parallel digests -> map -> writer.
+
+For sessions whose compact rendering exceeds cfg.chunk_max_chars, the
+single-shot improver is replaced by this staged pipeline. The trace's raw
+chunks are ALSO written to disk (chunk_dir/chunk_NNN.txt) so the writer
+can look up exact tool outputs by reference, keeping each agent context
+small even for ~10M-token sessions.
+"""
+from __future__ import annotations
+
+import concurrent.futures as cf
+import json
+import os
+import re
+import subprocess
+import threading
+import time
+from typing import Optional
+
+from traceforge import chunker
+from traceforge.config import Config
+from traceforge.trace import Message, Trace, json_loads_lenient
+from traceforge.improver.digest_prompts import (
+    CHUNK_DIGEST_HEADER,
+    MAP_MERGE_HEADER,
+    WRITER_LONG_HEADER,
+)
+
+
+def _hermes_run(
+    prompt_path: str, workdir: str, max_turns: int, timeout: int
+) -> tuple[int, str, str]:
+    cmd = [
+        "hermes", "chat",
+        "--query-file", prompt_path,
+        "--in", workdir,
+        "-Q",
+        "--max-turns", str(max_turns),
+    ]
+    proc = subprocess.run(
+        cmd, capture_output=True, text=True, timeout=timeout, cwd=workdir
+    )
+    return proc.returncode, proc.stdout or "", proc.stderr or ""
+
+
+def _json_file(path: str) -> Optional[dict]:
+    if not os.path.exists(path):
+        return None
+    return json_loads_lenient(open(path, encoding="utf-8").read())
+
+
+def _put_prompt(cfg: Config, name: str, text: str) -> str:
+    os.makedirs(cfg.prompts_dir, exist_ok=True)
+    p = os.path.join(cfg.prompts_dir, name)
+    with open(p, "w", encoding="utf-8") as f:
+        f.write(text)
+    return p
+
+
+def write_chunks(
+    chunk_dir: str, chunks: "list[chunker.Chunk]", trace: Trace
+) -> None:
+    """Persist raw chunks + a ref->output index for verbatim lookup."""
+    os.makedirs(chunk_dir, exist_ok=True)
+    for c in chunks:
+        with open(
+            os.path.join(chunk_dir, f"chunk_{c.index:03d}.txt"), "w",
+            encoding="utf-8",
+        ) as f:
+            f.write(c.rendered)
+
+
+def _digest_one(
+    cfg: Config, trace: Trace, c: chunker.Chunk, workdir: str
+) -> tuple[Optional[dict], str]:
+    """Run one digester session over one chunk. Returns (digest, error)."""
+    digest_path = os.path.join(workdir, f"digest_{c.index:03d}.json")
+    prompt = CHUNK_DIGEST_HEADER.format(
+        digest_path=digest_path, n=c.index
+    ) + c.rendered
+    pp = _put_prompt(cfg, f"digest_{trace.session_id}_c{c.index:03d}.txt", prompt)
+    try:
+        code, _out, _err = _hermes_run(
+            pp, workdir, cfg.digest_max_turns, cfg.digest_timeout_s
+        )
+    except subprocess.TimeoutExpired:
+        return None, f"chunk {c.index} digest timeout"
+    d = _json_file(digest_path)
+    if d is None:
+        return None, f"chunk {c.index} digest unparseable (exit {code})"
+    return d, ""
+
+
+def digest_all(
+    cfg: Config, trace: Trace, chunks: list, workdir: str,
+    progress_cb=None,
+) -> tuple[list[dict], list[str]]:
+    """Digest all chunks, cfg.digest_workers in parallel.
+
+    progress_cb(done, total) fires after each chunk completes (from the
+    worker threads) — used for live progress display.
+    """
+    errors: list[str] = []
+    digests: list[dict] = []
+    done = 0
+    lock = threading.Lock()
+    with cf.ThreadPoolExecutor(max_workers=cfg.digest_workers) as ex:
+        futs = {
+            ex.submit(_digest_one, cfg, trace, c, workdir): c.index
+            for c in chunks
+        }
+        for fut in cf.as_completed(futs):
+            idx = futs[fut]
+            d, err = fut.result()
+            with lock:
+                done += 1
+                if progress_cb is not None:
+                    try:
+                        progress_cb(done, len(chunks))
+                    except Exception:
+                        pass  # display callback must never break the stage
+                if err:
+                    errors.append(err)
+                elif d is not None:
+                    digests.append(d)
+    digests.sort(key=lambda d: int(d.get("chunk", 0)))
+    return digests, errors
+
+
+def build_map(
+    cfg: Config, trace: Trace, digests: list[dict], workdir: str
+) -> tuple[Optional[dict], str]:
+    """Merge digests into the unified session map."""
+    map_path = os.path.join(workdir, "session_map.json")
+    blob = "\n\n".join(json.dumps(d, ensure_ascii=False, indent=1) for d in digests)
+    prompt = MAP_MERGE_HEADER.format(map_path=map_path) + "\nDIGESTS:\n" + blob
+    pp = _put_prompt(cfg, f"map_{trace.session_id}.txt", prompt)
+    try:
+        code, _out, _err = _hermes_run(
+            pp, workdir, cfg.digest_max_turns, cfg.digest_timeout_s
+        )
+    except subprocess.TimeoutExpired:
+        return None, "map merge timeout"
+    m = _json_file(map_path)
+    if m is None:
+        return None, f"session map unparseable (exit {code})"
+    return m, ""
+
+
+def run_writer(
+    cfg: Config, trace: Trace, map_d: dict, chunk_dir: str,
+    workdir: str, output_path: str,
+    repair_errors: Optional[list[str]] = None,
+) -> tuple[Optional[Trace], str]:
+    """Run the ideal-trace writer over the session map."""
+    map_path = os.path.join(workdir, "session_map.json")
+    head = WRITER_LONG_HEADER.format(
+        output_path=output_path,
+        guidelines_path=cfg.guidelines_path,
+        map_path=map_path,
+        chunk_dir=chunk_dir,
+    )
+    if repair_errors:
+        head = (
+            "Your previous ideal trace FAILED validation. Fix these exact "
+            f"errors:\n- " + "\n- ".join(repair_errors[:20]) +
+            "\n\nThe previous attempt is at " + output_path +
+            " — read it, fix it in place.\n\n" + head
+        )
+    pp = _put_prompt(cfg, f"writer_{trace.session_id}.txt", head)
+    try:
+        code, _out, _err = _hermes_run(
+            pp, workdir, cfg.max_turns or 16, cfg.harness_timeout_s
+        )
+    except subprocess.TimeoutExpired:
+        return None, "writer timeout"
+    from traceforge.improver.harness import _parse_improved
+
+    t = _parse_improved(output_path)
+    if t is None:
+        return None, f"writer output unparseable (exit {code})"
+    return t, ""

traceforge/improver/prompt.py

@@ -0,0 +1,241 @@
+"""Builds the improver prompt: task spec + guidelines + compact trace.
+
+The improver is a Hermes one-shot session with tools. It is told to
+1) read the guidelines file itself (agentic grounding),
+2) study the rendered real trace,
+3) write the ideal-world rewrite to <workdir>/improved.json,
+4) self-check JSON validity before finishing.
+"""
+from __future__ import annotations
+
+import os
+
+TASK_HEADER = """You are a reference improver agent. Your job: rewrite the
+REAL agent trace below into the IDEAL-WORLD version of the same task — what a
+perfect execution would have looked like, following the guidelines exactly.
+The rewrite is the REFERENCE a benchmark scores the source model against: the
+less you have to change, the better that model is. Change only what the
+guidelines require.
+
+SCOPE — this is a REWRITE, not a re-execution. Hard limits:
+- The rendered trace below is your ONLY source of truth. Do NOT investigate
+  the original environment: no reading the actual project files, no git
+  history, no running commands from the trace, no browsing. You cannot and
+  must not verify against the real world.
+- Work alone. Do not spawn subagents or delegate.
+- Tool budget (this is all you need): (1) read the guidelines file,
+  (2) write improved.json, (3) re-read it to validate, (4) reply.
+- Go straight to writing. Analysis of the trace happens in your reply
+  reasoning, not in exploratory tool calls.
+
+QUALITY PILLARS (binding — the guidelines file is the full spec):
+- P1 Efficient but deep reasoning: high-signal decision logic per turn
+  (why this call, what the result implies, what to verify next); no filler.
+  FOCUS: answer exactly what was asked — for a follow-up question,
+  deliver the new piece only; never re-present facts the reader already
+  got in an earlier turn of the same session.
+  SUBSTANCE: the final answer presents the concrete result the task asked
+  for — the actual numbers, names, paths, values or code from the tool
+  evidence — never a description of what was retrieved. "I pulled the
+  nutrition data and the recipe" is poor; the data and the recipe ARE the
+  answer. Naming the work instead of showing its result fails the task
+  even when every tool call was correct.
+- P2 No committed mistakes: FIRST list the real trace's mistakes (failed
+  tool calls, retry loops, wrong turns, user feedback/corrections), then
+  route the ideal path around every single one — correct action the first
+  time; never show the failure or the repair. WHY: the assistant turns are
+  the loss-bearing target; a mistake in the target is imitated, a mistake in
+  the context is not. ONE exception (guidelines §4.1): an environment-caused
+  failure the ideal agent could not foresee may stay as a byte-verbatim tool
+  result, with exactly one corrective step in the next assistant turn and
+  recorded as a retained recovery by the harness. Model-caused errors (wrong
+  tool, wrong args, wrong path, bad reasoning) are never eligible.
+- P3 Perfect tool calls: every call necessary and correctly parameterized;
+  batch independent calls in parallel; no exploratory flailing; no call
+  whose result the narrative doesn't use. BATCHING IS STRUCTURAL: when the
+  real trace issues two calls in separate assistant turns but the second
+  does not consume the first's output, put BOTH tool_calls in ONE assistant
+  turn and let the two results follow in order. Do not preserve the source's
+  one-call-per-turn rhythm — that rhythm is the defect the judge rejects.
+  A duplicate call (same tool, same arguments) is dropped, never batched.
+- P4 Fact-driven: reuse real tool outputs verbatim where the ideal path
+  makes the same call; never invent files, commands, or results; no fake
+  success — the final answer is grounded in the trace's real evidence.
+  The FULL unclipped text of every real tool output is in {outputs_path}
+  (one JSON object per line; "i" = the [N] message index in the rendered
+  trace). When the ideal path makes the same call as the real trace, take
+  the result from there (a contiguous portion is fine). NEVER copy the
+  rendering's clip markers ("…[+N chars clipped]") into the ideal trace —
+  they are display artifacts of THIS prompt, not content. If you compress
+  a tool result, every substantive line must still be quoted from the
+  real output.
+  CONSTRUCTION ORDER (do this in this exact order): (1) decide the ideal
+  call path; (2) for every tool call on it, COPY the result content from
+  the matching outputs.jsonl row FIRST — byte-verbatim or a contiguous
+  slice; (3) only then write the reasoning and final answer AROUND those
+  real results. A tool result you cannot quote from outputs.jsonl is NOT
+  writable: either reroute the ideal path so that call is unnecessary, or
+  drop the claim. NEVER assemble a tool result from fragments you believe
+  are true — stitching individually-plausible lines is fabrication, the
+  single most common rejection reason.
+  External-world claims (versions, API behavior, dates, technical facts)
+  only when the trace itself proves them — otherwise leave them out.
+  A web-searching fact-checker reviews every external claim AFTER you
+  finish; unsupported claims send the trace back for repair.
+- P5 Language: one language per trace (the first user message's language);
+  perfect grammar in assistant prose; keep the user message verbatim.
+  NEVER open by restating the brief. The first assistant turn must do
+  work — a tool call, a decision, a finding — not re-present the user's
+  own text back to them. A message that is a near-copy of the opening
+  brief (>=85% similar) is rejected mechanically before the judge sees it.
+- P6 No refusals: complete the task. If it is genuinely un-completable,
+  reply starting "REJECTED: <one-line reason>" and write no output file.
+
+TRAINING-SIGNAL CONTRACT (why the pillars are shaped this way): this corpus
+is SFT — loss is computed on assistant tokens only; tool results, user turns
+and system prompts are masked context. Never put a failure, a fabricated
+error, or a "try again" narrative in an assistant turn. Never fabricate a
+tool output to make a recovery work. This corpus is not preference or RL
+data: do not invent rejected samples, preference pairs, or reward labels.
+
+PROCEDURE: (1) read the guidelines, (2) mine the mistakes, (3) write
+{output_path} in one go, (4) re-read it and validate (JSON, pairing,
+pillars), (5) summarize the changes.
+
+Ground rules:
+- Read the guidelines file first: {guidelines_path}
+  Treat it as the authoritative spec. If it has no rule for a case you hit,
+  follow the six pillars and the training-signal contract, and use your best
+  judgment — never invent a rule that contradicts them.
+- CRITICAL — intent fidelity: the ideal trace's FIRST user message must be
+  the real trace's first USER TASK message, verbatim, byte-exact (skip
+  handoff artifacts — `[CONTEXT COMPACTION]` Hermes rows and "This
+  session is being continued…" Claude Code rows; they carry no task).
+  If the real trace contains no user task message, do not invent
+  one: reply REJECTED: no user task in source trace and write no file.
+  You may merge subsequent real user messages into a shorter, cleaner
+  dialogue, but the OPENING task must stay the same task.
+- Preserve the original task intent (first user message) and any real context
+  (file paths, commands, tool results) that a correct execution would also
+  have produced. You MAY compress and clean the trace's own tool results,
+  but you must NOT invent facts the trace does not support.
+- Keep realistic tool_io. The improved trace must still read like a genuine
+  agent session — just a flawless one: efficient tool use, clear reasoning,
+  no flailing, no exploration dead-ends, no policy violations, precise
+  final answer.
+
+Output contract (CRITICAL):
+- Write the improved trace as JSON to exactly this path: {output_path}
+- Schema:
+  {{
+    "messages": [
+      {{"role": "user"|"assistant"|"tool",
+        "content": "...",
+        "tool_calls": [{{"id": "...", "name": "...",
+                          "arguments": "<json string>"}}]  // assistant only
+        "tool_call_id": "...", "tool_name": "..."          // tool rows only
+      }}
+    ]
+  }}
+  Optional per-message key: "reasoning" (string) for thinking text.
+- tool_calls[].arguments must be a JSON-encoded STRING (OpenAI style).
+- Every assistant tool_call id must have exactly one matching tool row
+  (tool_call_id), and vice versa.
+- Validate your output before finishing: re-read the file you wrote with a
+  tool, confirm it parses as JSON and the schema + pairing rules hold.
+- Then reply with a one-paragraph summary of what you changed and why.
+"""
+
+REPAIR_HEADER = """Your previous attempt at improving this trace FAILED
+validation. Fix it.
+
+Validation errors:
+{errors}
+
+Previous attempt: {prev_path}
+(Rendered real trace and guidelines still apply, restated below.)
+
+The FULL unclipped text of every real tool output is in {outputs_path}
+(one JSON object per line; "i" = the [N] message index in the rendered
+trace). Grounding failures mean: take the ideal tool results from there
+verbatim (contiguous portion fine) — do not paraphrase real outputs.
+NEVER replace a bad result by assembling plausible lines: copy byte-
+verbatim from outputs.jsonl, or drop the call/claim. Clip markers
+("…[+N chars clipped]") are display artifacts — never reproduce them,
+and remove any that the previous attempt copied in.
+
+SCOPE (same hard limits as before): this is a REWRITE, not an
+investigation. Do NOT read project files, repo source, gate code, or git
+history. Do NOT spawn subagents. Budget: (1) read the previous
+improved.json once, (2) read the specific outputs.jsonl rows cited in
+the errors above, (3) write the corrected file, (4) re-read it once to
+validate, (5) reply. NO other tool calls — every forensic check the
+reviewer already did for you is spelled out in the errors.
+
+WRITE MECHANICS: to overwrite an existing file, read the ENTIRE current
+file first (not a page of it), or write via a Python script
+(execute_code / terminal) that builds the new JSON with json.dumps —
+never hand-type large JSON. If a write is refused, re-read the full file
+and retry the write immediately; do not spend turns analyzing the
+refusal.
+
+Write the corrected improved trace JSON to: {output_path}
+Validate the file yourself before finishing, then summarize the fixes.
+"""
+
+
+def build_prompt(
+    *,
+    rendered_trace: str,
+    guidelines_path: str,
+    output_path: str,
+    outputs_path: "str | None" = None,
+    repair: "os._Environ | None" = None,
+    errors: list[str] | None = None,
+    prev_path: str | None = None,
+    language: str = "en",
+) -> str:
+    """Compose the full improver prompt for one trace."""
+    if errors and prev_path:
+        head = REPAIR_HEADER.format(
+            errors="\n".join(f"- {e}" for e in errors),
+            prev_path=prev_path,
+            output_path=output_path,
+            outputs_path=outputs_path or "(not provided)",
+        )
+    else:
+        head = TASK_HEADER.format(
+            guidelines_path=guidelines_path,
+            output_path=output_path,
+            outputs_path=outputs_path or "(not provided)",
+        )
+    # P5 is stated in the guidelines, but the writer still mirrors the
+    # language of the source material (German briefs, German tool output)
+    # instead of the task's. Naming the language explicitly, next to the
+    # trace, is what actually holds it.
+    from traceforge.classify import LANGUAGE_NAMES
+    lang = LANGUAGE_NAMES.get(language, "English")
+    lang_line = (
+        f"\nLANGUAGE (hard requirement): this trace's task is in {lang}. "
+        f"Write EVERY assistant message — prose, plan steps, final answer — "
+        f"in {lang}. The first user message stays byte-verbatim in its "
+        f"original language. Source material in another language (briefs, "
+        f"tool output, file contents) is quoted as-is, never mirrored: "
+        f"quoting German evidence inside an English trace is correct, "
+        f"answering in German is not.\n"
+    )
+    return (
+        f"{head}\n"
+        f"{lang_line}"
+        f"{'=' * 70}\n"
+        f"REAL TRACE TO IMPROVE (compact rendering, clipped tool output):\n\n"
+        f"{rendered_trace}\n"
+    )
+
+
+def write_prompt_file(prompts_dir: str, session_id: str, prompt: str) -> str:
+    os.makedirs(prompts_dir, exist_ok=True)
+    path = os.path.join(prompts_dir, f"improve_{session_id}.txt")
+    with open(path, "w", encoding="utf-8") as f:
+        f.write(prompt)
+    return path

traceforge/improver/review.py

@@ -0,0 +1,231 @@
+"""Quality-pillar reviewer: independent adversarial review rounds.
+
+The improver and verifier already self-check and web-check; this stage is
+the final human-dream-quality gate: independent reviewer sessions read
+the ideal trace against the six pillars and the ORIGINAL user intent,
+with NO knowledge of the improver's reasoning, and must unanimously PASS.
+cfg.review_rounds runs independently (default 2); ALL must PASS.
+"""
+from __future__ import annotations
+
+import dataclasses
+import json
+import os
+import subprocess
+import time
+from typing import Any, Optional
+
+from traceforge.config import Config
+from traceforge.trace import json_loads_lenient
+
+REVIEW_HEADER = """You are an independent, adversarial quality reviewer for
+a synthetic agent trace. You had NO part in creating it. Judge ONLY what
+you see, against the quality pillars below.
+
+The ideal trace under review: {improved_path}
+The ORIGINAL task (for intent comparison): {task_snippet}
+Quality pillars (the spec): {guidelines_path} — read it first.
+
+Judge the trace against every pillar. Typical failure shapes:
+- P1 violated: bloated reasoning, filler, restating the obvious
+- P2 violated: a mistake survives (failed call, retry loop, wrong turn,
+  user-correction absorbed nowhere)
+- P3 violated: unnecessary/exploratory tool calls, missing parallel
+  batching, wrong parameters
+- P4 violated: statements without basis in the trace's tool evidence;
+  fake success; invented paths/commands/results
+- P5 violated: mixed languages, grammar errors, sloppy prose
+- P6 violated: refusal or task abandonment
+- INTENT violated: the ideal trace solves a different task than the
+  original user asked
+
+Verdict JSON to {verdict_path}:
+{{
+  "verdict": "PASS" | "FAIL",
+  "pillar_findings": [
+    {{"pillar": "P1", "status": "ok|violated", "detail": "<if violated>"}}
+  ],
+  "violations": ["<every violation, one line each, concrete>"],
+  "notes": "<one paragraph overall judgment>"
+}}
+All pillars must be ok for PASS. Then reply "REVIEW: PASS" or
+"REVIEW: FAIL".
+
+DISCIPLINE — you have a LIMITED tool-call budget. Spend it wisely:
+- Read the guidelines once, then the improved.json once.
+- Do NOT re-read the same file repeatedly or re-verify what you already
+  checked. Batch related checks into one execute_code/terminal call.
+- From ~turn 10 onward (of 16), STOP forensic tool calls entirely and
+  write the verdict file with what you have. A verdict you can defend
+  from the checks already done is worth infinitely more than one more
+  probe: a missing verdict file FAILS the whole review round as an
+  infra error.
+- The verdict file MUST exist on disk BEFORE your final reply. If you
+  are running out of budget, write it NOW with your best judgment.
+"""
+
+
+@dataclasses.dataclass
+class ReviewResult:
+    verdict: str  # PASS | FAIL | ERROR
+    violations: list[str] = dataclasses.field(default_factory=list)
+    pillar_findings: list[dict] = dataclasses.field(default_factory=list)
+    notes: str = ""
+    duration_s: float = 0.0
+    error: Optional[str] = None
+
+    def ok(self) -> bool:
+        return self.verdict == "PASS"
+
+
+def _run_review_hermes(
+    prompt_path: str, workdir: str, max_turns: int, timeout: int
+) -> tuple[int, str]:
+    cmd = [
+        "hermes", "chat",
+        "--query-file", prompt_path,
+        "--in", workdir,
+        "-Q",
+        "--max-turns", str(max_turns),
+    ]
+    proc = subprocess.run(
+        cmd, capture_output=True, text=True, timeout=timeout, cwd=workdir
+    )
+    return proc.returncode, proc.stdout or ""
+
+
+def parse_review(raw: Optional[str]) -> Optional[dict[str, Any]]:
+    if not raw:
+        return None
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = raw.split("\n", 1)[1] if "\n" in raw else ""
+        if raw.rstrip().endswith("```"):
+            raw = raw.rstrip()[:-3]
+    data = json_loads_lenient(raw)
+    if not isinstance(data, dict) or "verdict" not in data:
+        return None
+    return data
+
+
+def _rescue_verdict_from_stdout(raw: Optional[str]) -> Optional[dict[str, Any]]:
+    """A reviewer that hit its turn cap mid-forensics never writes the
+    verdict file; its final chat message still contains the judgment.
+    REVIEW: PASS/FAIL line is authoritative; lift the richest JSON verdict
+    object the reviewer emitted, if any."""
+    if not raw:
+        return None
+    marker = None
+    for m in ("REVIEW: PASS", "REVIEW: FAIL"):
+        if m in raw:
+            marker = m.split(":")[1].strip()
+            break
+    if marker is None:
+        return None
+    data: dict[str, Any] = {"verdict": marker}
+    for candidate in _json_object_spans(raw):
+        try:
+            obj = json_loads_lenient(candidate)
+        except Exception:
+            continue
+        if isinstance(obj, dict) and "verdict" in obj:
+            data = obj
+            data["verdict"] = marker
+            break
+    return data
+
+
+def _json_object_spans(text: str) -> list[str]:
+    """Balanced top-level {...} spans from text, largest first."""
+    spans: list[tuple[int, int]] = []
+    depth = 0
+    start = -1
+    in_str = False
+    esc = False
+    for i, ch in enumerate(text):
+        if in_str:
+            if esc:
+                esc = False
+            elif ch == "\\":
+                esc = True
+            elif ch == '"':
+                in_str = False
+            continue
+        if ch == '"':
+            in_str = True
+        elif ch == "{":
+            if depth == 0:
+                start = i
+            depth += 1
+        elif ch == "}" and depth:
+            depth -= 1
+            if depth == 0 and start >= 0:
+                spans.append((start, i + 1))
+                start = -1
+    spans.sort(key=lambda s: s[1] - s[0], reverse=True)
+    return [text[a:b] for a, b in spans[:20]]
+
+
+def run_review(
+    cfg: Config,
+    improved_path: str,
+    task_snippet: str,
+    workdir: str,
+    round_no: int,
+) -> ReviewResult:
+    """One independent review round."""
+    started = time.time()
+    verdict_path = os.path.join(
+        workdir, f"review_{round_no:02d}.json"
+    )
+    if os.path.exists(verdict_path):
+        os.unlink(verdict_path)
+    prompt = REVIEW_HEADER.format(
+        improved_path=improved_path,
+        task_snippet=task_snippet[:2000],
+        guidelines_path=cfg.guidelines_path,
+        verdict_path=verdict_path,
+    )
+    os.makedirs(cfg.prompts_dir, exist_ok=True)
+    prompt_path = os.path.join(
+        cfg.prompts_dir,
+        f"review_{os.path.basename(os.path.dirname(improved_path))}"
+        f"_r{round_no}.txt",
+    )
+    with open(prompt_path, "w", encoding="utf-8") as f:
+        f.write(prompt)
+    try:
+        _code, _out = _run_review_hermes(
+            prompt_path, workdir, cfg.review_max_turns, cfg.review_timeout_s
+        )
+    except subprocess.TimeoutExpired:
+        return ReviewResult(
+            verdict="ERROR",
+            error=f"review round {round_no} timeout",
+            duration_s=time.time() - started,
+        )
+    data = parse_review(
+        open(verdict_path, encoding="utf-8").read()
+        if os.path.exists(verdict_path) else None
+    )
+    if data is None:
+        rescue = _rescue_verdict_from_stdout(_out)
+        if rescue is not None:
+            data = rescue
+    if data is None:
+        return ReviewResult(
+            verdict="ERROR",
+            error=f"review round {round_no} produced no parseable verdict",
+            duration_s=time.time() - started,
+        )
+    return ReviewResult(
+        verdict=str(data.get("verdict", "FAIL")).upper(),
+        violations=[
+            str(v) for v in (data.get("violations", []) or [])
+        ],
+        pillar_findings=[
+            p for p in (data.get("pillar_findings", []) or []) if isinstance(p, dict)
+        ],
+        notes=str(data.get("notes", "")),
+        duration_s=time.time() - started,
+    )

traceforge/improver/verify.py

@@ -0,0 +1,210 @@
+"""External-claim verification stage: a web-searching verifier agent.
+
+The grounding gate covers environment-internal facts (paths, outputs from
+the source trace). This stage covers the OTHER class: claims about the
+external world (library versions, API behavior, dates, technical facts)
+that an agent can state without the trace proving them — the classic
+hallucination vector.
+
+The verifier is a Hermes one-shot session with the default toolset (web
+search + extract enabled). It reads the improved trace, extracts
+external-world claims, web-searches each, and writes a verdict JSON.
+It VERDICTS, never rewrites — on FAIL the unsupported claims flow back
+into the improver repair loop.
+"""
+from __future__ import annotations
+
+import dataclasses
+import json
+import os
+import subprocess
+import time
+from typing import Any, Optional
+
+from traceforge.config import Config
+from traceforge.trace import json_loads_lenient
+
+VERIFIER_HEADER = """You are an adversarial fact-checker for a synthetic agent
+trace. The file below is an "ideal-world" agent trace under review for
+corpus admission. Your ONLY job: find incorrect or unverifiable statements
+about the EXTERNAL WORLD and fail the trace if any exist.
+
+What counts as an external claim (CHECK IT):
+- Technical facts: library/framework versions, API behavior, default
+  settings, command flags, file formats, error semantics
+- Factual assertions: dates, quantities, named results, "X supports Y",
+  "Z is deprecated since W"
+- Citations: paper names, spec sections, RFC numbers
+
+What does NOT count (SKIP — already verified against the source trace):
+- Anything about the project's own files/paths/commands/outputs
+- The trace's internal narrative (plans, reasoning steps)
+- Opinions, hedged statements, future intentions
+
+Procedure (use web_search / web_extract as needed — that is the point):
+1. Read the improved trace at: {improved_path}
+2. List every external claim. If there are none, verdict PASS with
+   claims_checked: 0 — do not search.
+3. For each claim: search the web. Primary sources beat blogs; note the
+   source URL per claim. A claim passes when a real source confirms it.
+   A claim fails when sources contradict it — or passes_with_caveat when
+   partially right. Unverifiable-but-asserted-with-confidence = fail.
+4. Write your verdict as JSON to exactly: {verdict_path}
+   Schema:
+   {{
+     "verdict": "PASS" | "FAIL",
+     "claims_checked": <int>,
+     "unsupported": [
+       {{"claim": "...", "why": "...", "searched": "..."}}
+     ],
+     "notes": "<one paragraph>"
+   }}
+   Include ALL checked claims (pass and fail) as a "checked" array with
+   per-claim {{"claim", "verdict": "pass"|"fail"|"pass_with_caveat",
+   "source"}} entries.
+5. Reply with just: VERDICT: PASS or VERDICT: FAIL plus claim count.
+
+Be adversarial: your value is in finding the wrong statement everyone
+else let slide. But do not fail a trace for hedged or internal statements.
+
+DISCIPLINE — you have a LIMITED tool-call budget:
+- Read the trace once; list the claims in your reasoning, then check
+  them with batched searches (multiple queries in parallel tool calls).
+- From ~turn 15 onward (of 20), STOP searching and write verdict.json
+  with what you have. The verdict file MUST exist on disk BEFORE your
+  final reply — a missing verdict file fails the whole verification
+  round as an infra error. If unsure, write your best verdict NOW.
+"""
+
+
+@dataclasses.dataclass
+class VerifierResult:
+    verdict: str  # PASS | FAIL | ERROR
+    claims_checked: int = 0
+    unsupported: list[dict] = dataclasses.field(default_factory=list)
+    checked: list[dict] = dataclasses.field(default_factory=list)
+    notes: str = ""
+    hermes_session_id: Optional[str] = None
+    duration_s: float = 0.0
+    error: Optional[str] = None
+
+    def ok(self) -> bool:
+        return self.verdict == "PASS"
+
+
+def _run_verifier_hermes(
+    cfg: Config, prompt_path: str, workdir: str
+) -> tuple[int, str, str]:
+    cmd = [
+        "hermes", "chat",
+        "--query-file", prompt_path,
+        "--in", workdir,
+        "-Q",
+        "--max-turns", str(cfg.verifier_max_turns),
+    ]
+    proc = subprocess.run(
+        cmd,
+        capture_output=True,
+        text=True,
+        timeout=cfg.verifier_timeout_s,
+        cwd=workdir,
+    )
+    return proc.returncode, proc.stdout or "", proc.stderr or ""
+
+
+def parse_verdict(raw: Optional[str]) -> Optional[dict[str, Any]]:
+    if not raw:
+        return None
+    raw = raw.strip()
+    if raw.startswith("```"):
+        raw = raw.split("\n", 1)[1] if "\n" in raw else ""
+        if raw.rstrip().endswith("```"):
+            raw = raw.rstrip()[:-3]
+    data = json_loads_lenient(raw)
+    if not isinstance(data, dict) or "verdict" not in data:
+        return None
+    return data
+
+
+def _rescue_verdict_from_stdout(raw: Optional[str]) -> Optional[dict[str, Any]]:
+    """A verifier that hit its turn cap mid-research never writes
+    verdict.json; its final chat message still contains the judgment.
+    VERDICT: PASS/FAIL line is authoritative; lift the richest JSON
+    verdict object the verifier emitted, if any."""
+    from traceforge.improver.review import (
+        _json_object_spans,
+    )
+
+    if not raw:
+        return None
+    marker = None
+    for m in ("VERDICT: PASS", "VERDICT: FAIL"):
+        if m in raw:
+            marker = m.split(":")[1].strip()
+            break
+    if marker is None:
+        return None
+    data: dict[str, Any] = {"verdict": marker}
+    for candidate in _json_object_spans(raw):
+        try:
+            obj = json_loads_lenient(candidate)
+        except Exception:
+            continue
+        if isinstance(obj, dict) and "verdict" in obj:
+            data = obj
+            data["verdict"] = marker
+            break
+    return data
+
+
+def run_verifier(
+    cfg: Config, improved_path: str, workdir: str
+) -> VerifierResult:
+    """Fact-check the improved trace's external claims via web search."""
+    started = time.time()
+    verdict_path = os.path.join(workdir, "verdict.json")
+    if os.path.exists(verdict_path):
+        os.unlink(verdict_path)
+    prompt = VERIFIER_HEADER.format(
+        improved_path=improved_path, verdict_path=verdict_path
+    )
+    os.makedirs(cfg.prompts_dir, exist_ok=True)
+    prompt_path = os.path.join(
+        cfg.prompts_dir,
+        f"verify_{os.path.basename(os.path.dirname(improved_path))}.txt",
+    )
+    with open(prompt_path, "w", encoding="utf-8") as f:
+        f.write(prompt)
+    try:
+        code, stdout, stderr = _run_verifier_hermes(cfg, prompt_path, workdir)
+    except subprocess.TimeoutExpired:
+        return VerifierResult(
+            verdict="ERROR", error=f"verifier timeout {cfg.verifier_timeout_s}s",
+            duration_s=time.time() - started,
+        )
+    data = parse_verdict(
+        open(verdict_path, encoding="utf-8").read()
+        if os.path.exists(verdict_path) else None
+    )
+    if data is None:
+        rescue = _rescue_verdict_from_stdout(stdout)
+        if rescue is not None:
+            data = rescue
+    if data is None:
+        tail = stderr.strip()[-400:] if stderr else ""
+        return VerifierResult(
+            verdict="ERROR",
+            error=(
+                f"verifier produced no parseable verdict.json "
+                f"(exit {code}{'; ' + tail if tail else ''})"
+            ),
+            duration_s=time.time() - started,
+        )
+    return VerifierResult(
+        verdict=str(data.get("verdict", "FAIL")).upper(),
+        claims_checked=int(data.get("claims_checked", 0) or 0),
+        unsupported=list(data.get("unsupported", []) or []),
+        checked=list(data.get("checked", []) or []),
+        notes=str(data.get("notes", "")),
+        duration_s=time.time() - started,
+    )

traceforge/live.py

@@ -0,0 +1,508 @@
+"""Live terminal telemetry for long-running batches.
+
+Every stage traceforge spawns (improver, digester, map, writer, verifier,
+reviewer) is a Hermes one-shot running with cwd = the per-trace workdir,
+and Hermes updates live counters on the session row in state.db as the
+model streams. LiveStats polls those rows on a background thread and
+renders:
+
+  TTY    detail rows under the batch bar covering EVERY in-flight
+         trace (stage, age, tokens in/out, instantaneous tok/s,
+         child-session count), plus a batch postfix (total out ·
+         tok/s). Layout adapts to --workers and terminal size: full
+         detail rows while they fit; then packed chips
+         (`<sid-tail> <stage> <age>`, several per row); a stage
+         histogram only when even chips overflow the screen. An
+         in-flight trace is never silently hidden.
+  pipes  a plain heartbeat line every live_heartbeat_s with the same
+         aggregate — every in-flight trace listed up to 10, then
+         per-stage counts — so backgrounded runs show real progress.
+  end    token totals on IMPROVED/FAILED lines and a batch summary line.
+
+Read-only against state.db (mode=ro); poll failures are swallowed and
+keep the last snapshot. Contract: poll() runs on one thread (the built-in
+loop); render helpers are safe from any thread.
+"""
+from __future__ import annotations
+
+import os
+import shutil
+import sqlite3
+import threading
+import time
+from typing import Optional
+
+from traceforge.progress import _fmt_dur
+
+
+def fmt_tok(n) -> str:
+    """Compact token count: 1234 -> '1.2k', 1234567 -> '1.23M'."""
+    n = int(n or 0)
+    if n >= 1_000_000:
+        return f"{n / 1_000_000:.2f}M"
+    if n >= 10_000:
+        return f"{n / 1000:.0f}k"
+    if n >= 1000:
+        return f"{n / 1000:.1f}k"
+    return str(n)
+
+
+def _stage_counts(acts) -> dict:
+    """{stage: n} over [(sid, info), ...] pairs."""
+    counts: dict = {}
+    for _sid, info in acts:
+        st = info.get("stage", "?")
+        counts[st] = counts.get(st, 0) + 1
+    return counts
+
+
+def sid_short(sid: str) -> str:
+    """Distinguishing short form of a session id for detail rows.
+
+    Repo sids are date-prefixed (`20260926_160647_ceb268`), so the
+    first 12 chars are the SAME for every trace started the same
+    minute — head truncation makes rows indistinguishable. Show the
+    tail instead: last `_` segment (hermes) or last 8 chars (uuid).
+    """
+    seg = sid.rsplit("_", 1)[-1] if len(sid) > 10 else sid
+    if len(seg) > 8:
+        seg = seg[-8:]
+    return seg if seg == sid else f"…{seg}"
+
+
+class LiveRegistry:
+    """Thread-safe map of in-flight traces -> current stage + timestamps."""
+
+    def __init__(self):
+        self._lock = threading.Lock()
+        self._traces: dict = {}
+
+    def begin(self, sid: str, stage: str = "starting") -> None:
+        with self._lock:
+            now = time.monotonic()
+            self._traces[sid] = {
+                "stage": stage, "t0": now, "stage_t": now,
+            }
+
+    def set_stage(self, sid: str, stage: str) -> None:
+        with self._lock:
+            d = self._traces.get(sid)
+            if d is not None:
+                d["stage"] = stage
+                d["stage_t"] = time.monotonic()
+
+    def end(self, sid: str) -> None:
+        with self._lock:
+            self._traces.pop(sid, None)
+
+    def snapshot(self):
+        """[(sid, info), ...] oldest first — stable display order."""
+        with self._lock:
+            return [
+                (sid, dict(d))
+                for sid, d in sorted(
+                    self._traces.items(), key=lambda kv: kv[1]["t0"]
+                )
+            ]
+
+
+class LiveStats:
+    """Polls state.db child-session counters; renders live progress."""
+
+    def __init__(self, cfg, registry: LiveRegistry, prog=None,
+                 *, max_sublines: int = 0):
+        self.registry = registry
+        self.prog = prog
+        # 0 = adaptive: show every in-flight trace, switching layout
+        # (full rows -> packed chips -> stage histogram) by screen fit.
+        # >0 (tests, explicit caps): hard cap on full detail rows.
+        self.max_sublines = max(0, int(max_sublines))
+        self._forced_cap = self.max_sublines > 0
+        self.state_db = cfg.state_db
+        self.poll_s = max(0.5, float(getattr(cfg, "live_poll_s", 2.0)))
+        self.heartbeat_s = max(
+            5.0, float(getattr(cfg, "live_heartbeat_s", 30.0))
+        )
+        # ours = one-shot sessions started in this batch window AND inside
+        # this batch's workdir root (cwd leaf = trace sid)
+        self._like = os.path.join(cfg.improver_workdir, "")
+        self.enabled = os.path.exists(self.state_db)
+        self._t0_mono = time.monotonic()
+        self._since = time.time() - 1.0
+        self._per: dict = {}
+        self._prev_out: dict = {}
+        self._sess_models: dict = {}  # child session id -> model name
+        self._sessions = 0
+        self._tot_in = 0
+        self._tot_out = 0
+        self._rate = 0.0
+        self._last_poll = None  # monotonic ts of previous poll
+        self._lock = threading.Lock()
+        self._stop = threading.Event()
+        self._thread: Optional[threading.Thread] = None
+
+    # ----------------------------------------------------------- lifecycle
+    @classmethod
+    def setup(cls, cfg, prog=None, registry=None, **kw):
+        """Factory honoring the live_poll_s=0 kill-switch."""
+        if float(getattr(cfg, "live_poll_s", 0) or 0) <= 0:
+            return None
+        reg = registry or LiveRegistry()
+        return cls(cfg, reg, prog, **kw)
+
+    def start(self) -> None:
+        if not self.enabled or self._thread is not None:
+            return
+        self._thread = threading.Thread(
+            target=self._loop, name="traceforge-live", daemon=True
+        )
+        self._thread.start()
+
+    def stop(self) -> None:
+        self._stop.set()
+        t = self._thread
+        if (
+            t is not None and t.is_alive()
+            and threading.current_thread() is not t
+        ):
+            t.join(timeout=self.poll_s + 1.0)
+        self._thread = None
+
+    # ---------------------------------------------------------------- poll
+    def poll(self) -> None:
+        """One state.db read; recomputes aggregates. Single-thread contract."""
+        try:
+            conn = sqlite3.connect(
+                f"file:{self.state_db}?mode=ro", uri=True, timeout=2.0
+            )
+            try:
+                rows = conn.execute(
+                    "SELECT id, cwd, input_tokens, output_tokens,"
+                    " message_count, api_call_count, ended_at IS NULL, model"
+                    " FROM sessions WHERE source='oneshot'"
+                    " AND started_at >= ?",
+                    (self._since,),
+                ).fetchall()
+            finally:
+                conn.close()
+        except sqlite3.Error:
+            return  # transient lock/IO hiccup: keep last snapshot
+        rows = [r for r in rows if (r[1] or "").startswith(self._like)]
+        now = time.monotonic()
+        with self._lock:
+            prev_out = dict(self._prev_out)
+            last_poll = self._last_poll
+        dt = None
+        if last_poll is not None:
+            dt = max(now - last_poll, 1e-6)
+        per: dict = {}
+        delta_out = 0
+        models: dict = {}
+        for (sess, cwd, tin, tout, msgs, api, running, model) in rows:
+            key = (cwd or "").rstrip("/").rsplit("/", 1)[-1]
+            d = per.setdefault(key, {
+                "in": 0, "out": 0, "msgs": 0, "api": 0,
+                "runs": 0, "running": 0, "rate": 0.0, "models": set(),
+            })
+            tin, tout = tin or 0, tout or 0
+            d["in"] += tin
+            d["out"] += tout
+            d["msgs"] += msgs or 0
+            d["api"] += api or 0
+            d["runs"] += 1
+            d["running"] += 1 if running else 0
+            if model:
+                d["models"].add(str(model))
+                models[sess] = str(model)
+            po = prev_out.get(sess)
+            if po is not None and tout > po:
+                grew = tout - po
+                delta_out += grew
+                d["rate"] += grew / dt if dt else 0.0
+        # splice path: merge direct-HTTP usage (no state.db rows exist
+        # for _post_chat calls; SPLICE_USAGE is the only telemetry source)
+        splice_delta = 0
+        sper: dict = {}
+        try:
+            from traceforge.splice import SPLICE_USAGE
+            sper, splice_delta = SPLICE_USAGE.drain(dt)
+            for sid, b in sper.items():
+                d = per.setdefault(sid, {
+                    "in": 0, "out": 0, "msgs": 0, "api": 0,
+                    "runs": 0, "running": 0, "rate": 0.0, "models": set(),
+                })
+                d["in"] += b["in"]
+                d["out"] += b["out"]
+                d["api"] += b["calls"]
+                d["runs"] += b["calls"]
+                d["rate"] += b["rate"]
+                for m in b["models"]:
+                    d["models"].add(m)
+                    models[f"splice:{sid}:{m}"] = m
+        except Exception:
+            pass  # telemetry must never break the batch
+        for d in per.values():
+            d["rate"] = round(d["rate"], 2)
+            d["models"] = sorted(d["models"])
+        with self._lock:
+            self._per = per
+            self._prev_out = {r[0]: r[3] or 0 for r in rows}
+            self._sessions = len(rows) + len(
+                [s for s in sper if s != "-"]
+            )
+            self._sess_models.update(models)
+            self._tot_in = sum(d["in"] for d in per.values())
+            self._tot_out = sum(d["out"] for d in per.values())
+            if dt:
+                self._rate = round(
+                    0.5 * self._rate
+                    + 0.5 * ((delta_out + splice_delta) / dt), 2
+                )
+            self._last_poll = now
+
+    def _loop(self) -> None:
+        next_hb = time.monotonic() + self.heartbeat_s
+        while not self._stop.is_set():
+            self.poll()
+            if self.prog is not None:
+                if self.prog.enabled:
+                    pf = self._postfix()
+                    if pf:
+                        self.prog.set_postfix(pf)
+                    # always refresh: an empty list clears stale sublines
+                    self.prog.set_sublines(self.sublines())
+                elif time.monotonic() >= next_hb:
+                    self.prog.write(self.heartbeat())
+                    next_hb = time.monotonic() + self.heartbeat_s
+            self._stop.wait(self.poll_s)
+
+    # ------------------------------------------------------------ rendering
+    def _snapshot(self):
+        with self._lock:
+            return dict(self._per), self._tot_in, self._tot_out, self._rate
+
+    def per_trace(self, sid: str):
+        """Aggregate token usage for one trace (all its child sessions)."""
+        with self._lock:
+            d = self._per.get(sid)
+            return dict(d) if d else None
+
+    def sublines(self) -> list:
+        """Detail lines for the Progress bar — covering EVERY in-flight
+        trace, never a "+N more" cutoff.
+
+        Adaptive layout (no Rich TUI needed — the bar is the dashboard):
+          1. full detail rows, one per trace, while they all fit on
+             screen — the default at any worker count with headroom;
+          2. packed chips — several traces per row
+             (`├ <sid8> improving 1:23  <sid8> verify 1/2 0:45`) once
+             full rows would overflow the terminal height;
+          3. stage histogram (`├ stages: improving 9 · verify 2/2 4`) only
+             when rows×width can't fit every chip — traces are never
+             silently hidden at any worker count or terminal size.
+        """
+        acts = self.registry.snapshot()
+        if not acts:
+            return []
+        per, _tin, _tout, _rate = self._snapshot()
+        now = time.monotonic()
+        rows_budget = self._detail_rows_budget()
+        if rows_budget >= len(acts):
+            return self._full_rows(acts, per, now)
+        return self._packed_rows(acts, rows_budget, now)
+
+    # ------------------------------------------------- adaptive subline help
+    def _detail_rows_budget(self) -> int:
+        """How many detail rows the screen has room for.
+
+        Forced caps (>0, tests/explicit limit) win; adaptive mode
+        reserves 4 lines around the frame (bar row, prompt, slack) and
+        keeps at least the historical 8 visible rows.
+        """
+        if self._forced_cap:
+            return self.max_sublines
+        lines = shutil.get_terminal_size((120, 24)).lines
+        return max(8, lines - 4)
+
+    def _full_rows(self, acts, per, now) -> list:
+        cols = shutil.get_terminal_size((120, 24)).columns
+        lines = []
+        last = len(acts) - 1
+        for i, (sid, info) in enumerate(acts):
+            connector = "└" if i == last else "├"
+            head = (
+                f"{connector} {sid_short(sid):<10}"
+                f" {info.get('stage', '?'):<12}"
+                f" {_fmt_dur(now - info.get('t0', now)):>6}"
+            )
+            d = per.get(sid)
+            if not d:
+                lines.append(head + " · telemetry pending")
+                continue
+            # richest first; trim fields until the line fits the terminal
+            m = d.get("models") or []
+            fields = [
+                f"{d['rate']:5.1f} tok/s",
+                f"{fmt_tok(d['out'])} out",
+                f"{fmt_tok(d['in'])} in",
+                "/".join(m) if m else "?",
+                f"{d['runs']} run" + ("s" if d["runs"] != 1 else ""),
+            ]
+            line = head
+            for f in fields:
+                cand = line + f" · {f}"
+                if len(cand) + 1 <= cols:
+                    line = cand
+            lines.append(line)
+        return lines
+
+    @staticmethod
+    def _chip(sid: str, info: dict, now: float) -> str:
+        """One packed trace chip: `<sid8>… <stage≤12> <age>`."""
+        stage = info.get("stage", "?")
+        if len(stage) > 12:
+            stage = stage[:12]
+        return f"{sid_short(sid)} {stage} {_fmt_dur(now - info.get('t0', now))}"
+
+    def _packed_rows(self, acts, rows_budget: int, now) -> list:
+        """Chips packed several-per-row: ALL traces visible, compact.
+
+        Greedy packing by ACTUAL chip width (worst ~28 chars, typically
+        less) honors the terminal columns; the row count stays within
+        rows_budget. If width×rows together cannot show every trace,
+        degrade to the stage histogram — traces are never dropped.
+        """
+        cols = shutil.get_terminal_size((120, 24)).columns
+        chips = [self._chip(sid, info, now) for sid, info in acts]
+        rows: list = []
+        cur: list = []
+        cur_w = 2  # "├ " connector prefix
+        for chip in chips:
+            w = len(chip) + (2 if cur else 0)  # gap before non-first chip
+            if cur and cur_w + w > cols:
+                rows.append(cur)
+                cur, cur_w = [], 2
+                if len(rows) >= rows_budget:
+                    break
+            cur.append(chip)
+            cur_w += w
+        if cur and len(rows) < rows_budget:
+            rows.append(cur)
+        placed = sum(len(r) for r in rows)
+        if placed < len(chips):
+            # cannot fit every trace even packed: histogram, not hiding
+            return self._stage_histogram(acts, rows_budget)
+        return [
+            f"{'├' if i < len(rows) - 1 else '└'} " + "  ".join(r)
+            for i, r in enumerate(rows)
+        ]
+
+    @staticmethod
+    def _stage_histogram(acts, rows_budget: int) -> list:
+        """Last resort (narrow/short terminals): per-stage counts.
+
+        Width-safe: stage groups wrap across rows within the terminal
+        columns; the oldest in-flight trace closes the frame when the
+        row budget allows it.
+        """
+        cols = shutil.get_terminal_size((120, 24)).columns
+        groups = [
+            f"{st} {n}"
+            for st, n in sorted(
+                _stage_counts(acts).items(), key=lambda kv: (-kv[1], kv[0])
+            )
+        ]
+        rows: list = []
+        cur: list = []
+        cur_w = len("├ stages: ")
+        for g in groups:
+            w = len(g) + (3 if cur else 0)  # " · " separator
+            if cur and cur_w + w > cols:
+                rows.append(" · ".join(cur))
+                cur, cur_w = [], len("├ stages: ")
+                if len(rows) >= max(1, rows_budget - 1):
+                    break
+            cur.append(g)
+            cur_w += w
+        if cur:
+            rows.append(" · ".join(cur))
+            if len(rows) > max(1, rows_budget - 1):
+                rows = rows[:max(1, rows_budget - 1)]
+        sid, info = acts[0]  # snapshot() is oldest-first
+        now = time.monotonic()
+        oldest = (
+            f"oldest: {sid_short(sid)} {info.get('stage', '?')}"
+            f" {_fmt_dur(now - info.get('t0', now))}"
+        )
+        if rows_budget == 1:
+            # single-row budget: merge the oldest into the histogram row
+            head = "stages: " + (rows[0] if rows else "")
+            return [f"└ {head} · {oldest}"[:cols]]
+        lines = [f"stages: {r}" if i == 0 else f"        {r}"
+                 for i, r in enumerate(rows)]
+        lines.append(oldest)
+        return [f"{'└' if i == len(lines) - 1 else '├'} {ln}"
+                for i, ln in enumerate(lines[:rows_budget])]
+
+    def _postfix(self) -> str:
+        _per, tin, tout, rate = self._snapshot()
+        if tout or tin:
+            return f"{fmt_tok(tout)} out · {rate:.1f} tok/s"
+        return ""
+
+    def heartbeat(self) -> str:
+        _per, tin, tout, rate = self._snapshot()
+        wall = _fmt_dur(time.monotonic() - self._t0_mono)
+        now = time.monotonic()
+        acts = self.registry.snapshot()
+        # every in-flight trace listed for as long as the line stays
+        # readable (~300 chars); beyond that collapse to stage counts
+        shown = acts[:10]
+        inflight = ", ".join(
+            f"{sid_short(sid)} {info.get('stage', '?')}"
+            f" ({_fmt_dur(now - info.get('t0', now))})"
+            for sid, info in shown
+        )
+        if len(acts) > len(shown):
+            rest = acts[len(shown):]
+            counts = " · ".join(
+                f"{st} {n}"
+                for st, n in sorted(
+                    _stage_counts(rest).items(), key=lambda kv: -kv[1]
+                )
+            )
+            inflight = f"{inflight} … +{len(acts) - len(shown)} ({counts})" if inflight else counts
+        inflight = inflight or "none"
+        # Judge cache hit rate: a repair round re-judges every window, so this
+        # is the share of judge calls that cost nothing. Silent until the
+        # first miss, so a run that never judges does not print "0/0".
+        try:
+            from traceforge.splice import judge_cache_stats
+            js = judge_cache_stats()
+            jtot = js["hit"] + js["miss"]
+            jtxt = (f" · judge cache {100 * js['hit'] / jtot:.0f}%"
+                    f" ({js['hit']}/{jtot})") if jtot else ""
+        except Exception:
+            jtxt = ""
+        return (
+            f"-- [{wall}] {fmt_tok(tin)} in · {fmt_tok(tout)} out"
+            f" · {rate:.1f} tok/s · {self._models_str()} · in flight: {inflight}"
+            f"{jtxt}"
+        )
+
+    def _models_str(self) -> str:
+        """Distinct child models seen this batch, e.g. 'glmb300'."""
+        with self._lock:
+            ms = set(self._sess_models.values())
+        return "/".join(sorted(ms)) if ms else "?"
+
+    def summary(self) -> str:
+        with self._lock:
+            tin, tout, ns = self._tot_in, self._tot_out, self._sessions
+        wall = max(time.monotonic() - self._t0_mono, 1e-6)
+        avg = tout / wall if tout else 0.0
+        return (
+            f"-- live: {ns} child sessions · {fmt_tok(tin)} tok in"
+            f" · {fmt_tok(tout)} tok out · avg {avg:.1f} tok/s gen"
+            f" · model {self._models_str()} · wall {_fmt_dur(wall)}"
+        )

traceforge/progress.py

@@ -0,0 +1,194 @@
+"""Minimal dependency-free progress bars (tqdm-style, stdlib only).
+
+Design notes:
+- Bar renders to stderr and only when stderr is a TTY; when output is
+  piped/redirected (CI, background shells, logs) the bar disables itself
+  and callers fall back to plain per-item lines on stdout.
+- Thread-safe: a single lock guards rendering, so worker threads can
+  update counts / postfixes concurrently.
+- write(msg) clears the current frame, prints msg on stderr, and
+  redraws — the tqdm.write pattern — so per-item result lines never
+  visually corrupt the bar.
+- Multi-line frames: set_sublines(lines) installs detail lines drawn
+  under the bar (one per in-flight worker). Each render draws the whole
+  frame atomically and repaints in place with ANSI cursor-up codes.
+  Height changes (workers starting/finishing) repaint from the old
+  frame's top row, so no stale rows survive.
+"""
+from __future__ import annotations
+
+import shutil
+import sys
+import threading
+import time
+from typing import Optional, TextIO
+
+
+def _fmt_dur(sec: float) -> str:
+    s = int(sec)
+    if s >= 3600:
+        return f"{s // 3600}:{s % 3600 // 60:02d}:{s % 60:02d}"
+    return f"{s // 60:}:{s % 60:02d}"
+
+
+class Progress:
+    """Progress bar with optional detail sublines.
+
+    Frame layout (TTY only):
+
+        desc  42%|########----| 4/10 [1:12<24:51, 3.2s/it] postfix
+        ├ abc123def456 improving     1:23  45.2k in · 3.1k out · 12.4 tok/s
+        └ def789abc123 verify 1/2    0:45  12.0k in · 1.2k out · 4.2 tok/s
+
+    update()/set_postfix()/set_sublines() are thread-safe; render is
+    rate-limited by mininterval so fast loops don't flood the terminal.
+    """
+
+    PAD_TARGET = 150  # pad each line to at least this width (wipe leftovers)
+
+    def __init__(
+        self,
+        total: int,
+        desc: str = "",
+        *,
+        stream: Optional[TextIO] = None,
+        width: int = 24,
+        mininterval: float = 0.25,
+        enabled: Optional[bool] = None,
+    ):
+        self.total = max(int(total), 0)
+        self.n = 0
+        self.desc = desc
+        self.width = width
+        self.mininterval = mininterval
+        self.stream = stream or sys.stderr
+        if enabled is None:
+            enabled = bool(getattr(self.stream, "isatty", lambda: False)())
+        self.enabled = enabled and self.total > 0
+        self._lock = threading.Lock()
+        self._postfix = ""
+        self._sublines: list[str] = []
+        self._frame_rows = 0  # rows the on-screen frame currently spans
+        self._start = time.monotonic()
+        self._last_render = 0.0
+        self._closed = False
+        if self.enabled:
+            self._render(force=True)
+
+    # ------------------------------------------------------------ lifecycle
+    def set_postfix(self, s: str) -> None:
+        with self._lock:
+            self._postfix = s
+            self._maybe_render()
+
+    def set_sublines(self, lines) -> None:
+        """Install/refresh the detail lines drawn under the bar."""
+        with self._lock:
+            if not self.enabled:
+                return
+            new = list(lines)
+            if new == self._sublines:
+                return  # byte-identical: skip render
+            self._sublines = new
+            self._maybe_render(force=True)
+
+    def update(self, n: int = 1) -> None:
+        with self._lock:
+            self.n = min(self.n + n, self.total)
+            self._maybe_render()
+
+    def close(self) -> None:
+        with self._lock:
+            if self._closed:
+                return
+            self._closed = True
+            if self.enabled:
+                self._render(force=True)
+                self.stream.write("\n")
+                self.stream.flush()
+
+    # context-manager sugar
+    def __enter__(self) -> "Progress":
+        return self
+
+    def __exit__(self, *exc) -> None:
+        self.close()
+
+    # ------------------------------------------------------------ rendering
+    def _maybe_render(self, force: bool = False) -> None:
+        if not self.enabled or self._closed:
+            return
+        now = time.monotonic()
+        if force or now - self._last_render >= self.mininterval:
+            self._render(force=True)
+
+    def _render(self, force: bool = False) -> None:
+        if not self.enabled:
+            return
+        now = time.monotonic()
+        if not force and now - self._last_render < self.mininterval:
+            return
+        self._last_render = now
+        elapsed = max(now - self._start, 1e-6)
+        frac = self.n / self.total if self.total else 0.0
+        filled = int(self.width * frac)
+        bar = "#" * filled + "-" * (self.width - filled)
+        # rate: per-item time; 0 completed -> unknown
+        if self.n > 0:
+            per = elapsed / self.n
+            eta = _fmt_dur(per * (self.total - self.n))
+            rate = f"{per:.1f}s/it"
+        else:
+            eta, rate = "-:--", "?it/s"
+        line = (
+            f"{self.desc} {int(frac * 100):3d}%|{bar}|"
+            f" {self.n}/{self.total} [{_fmt_dur(elapsed)}<{eta}, {rate}]"
+        )
+        if self._postfix:
+            line += f" {self._postfix}"
+        frame = [line] + list(self._sublines)
+        old_h = self._frame_rows
+        if old_h > 0 and len(frame) < old_h:
+            # frame shrank: overwrite stale rows below with blanks
+            frame += [""] * (old_h - len(frame))
+        # truncate each line to the terminal width — a longer line would
+        # wrap to a second physical row and break the up-cursor math
+        cols = shutil.get_terminal_size((120, 24)).columns
+        padded = []
+        for ln in frame:
+            ln = ln[:cols]
+            padded.append(ln + " " * (cols - len(ln)))
+        rows = len(padded)
+        body = "\r" + "\n".join(padded)
+
+        # Cursor sits ON the frame's last row (lines joined with \n, no
+        # trailing newline). Move up to the frame's top row before the
+        # rewrite; F also returns the cursor to column 0. old_h == 1
+        # needs no prefix: the cursor is already on the only frame row
+        # and the leading \r returns it to column 0.
+        if old_h >= 2:
+            body = f"\x1b[{old_h - 1}F" + body
+
+        self.stream.write(body)
+        self.stream.flush()
+        self._frame_rows = rows
+
+    def write(self, msg: str) -> None:
+        """Print a permanent line above the frame (thread-safe)."""
+        with self._lock:
+            if not self.enabled:
+                print(msg)  # non-TTY: plain stdout log line
+                return
+            if self._frame_rows > 0:
+                # park on the frame's TOP row: cursor sits ON the last
+                # row of the frame, so up = rows-1 lands on the top row
+                up = max(self._frame_rows - 1, 1)
+                self.stream.write(f"\x1b[{up}F")
+                self.stream.write("\x1b[0J")  # clear cursor..end of screen
+                self._frame_rows = 0  # frame gone; next render starts here
+            else:
+                self.stream.write("\r" + " " * 130 + "\r")
+            self.stream.write(msg + "\n")
+            self.stream.flush()
+            if not self._closed:
+                self._render(force=True)

traceforge/readers/__init__.py

@@ -0,0 +1,11 @@
+from traceforge.readers.claude import ClaudeTraceReader
+from traceforge.readers.codex import CodexTraceReader
+from traceforge.readers.hermes import HermesTraceReader
+from traceforge.readers.nemotron import NemotronTraceReader
+
+__all__ = [
+    "HermesTraceReader",
+    "ClaudeTraceReader",
+    "CodexTraceReader",
+    "NemotronTraceReader",
+]

traceforge/readers/_common.py

@@ -0,0 +1,184 @@
+"""Shared reader helpers: self-reference guard + silent-skip accounting.
+
+Both concerns are cross-cutting (every reader has a store it must not
+read its own output back from, and every reader has code paths that
+skip a file or a row without telling anyone), so they live here rather
+than being re-implemented per reader.
+"""
+from __future__ import annotations
+
+import re
+from collections import Counter
+from typing import Optional
+
+
+def _norm(path: Optional[str]) -> str:
+    """Forward-slash, case-folded, drive-less form of a path for comparison.
+
+    Seed archives come from other machines: a macOS rollout carries
+    ``/Users/x/dev/traceforge/data/work/<sid>`` while the live guard is
+    ``C:\\Users\\y\\...\\data\\work``. Comparing raw strings misses that,
+    which is exactly the self-reference leak the guard exists to stop.
+    """
+    if not path:
+        return ""
+    p = str(path).replace("\\", "/")
+    if len(p) > 1 and p[1] == ":":
+        p = p[2:]  # strip the Windows drive letter
+    return p.rstrip("/").lower()
+
+
+def in_workdir(cwd: Optional[str], workdir: Optional[str]) -> bool:
+    """True when ``cwd`` sits inside ``workdir`` (any platform's spelling).
+
+    Two matches count:
+      1. the normalized cwd starts with the normalized workdir;
+      2. the cwd ends with the workdir's last two components
+         (``data/work/<sid>``) — the portable form, which catches a
+         foreign-platform archive whose absolute prefix differs.
+    """
+    c, w = _norm(cwd), _norm(workdir)
+    if not c or not w:
+        return False
+    if c == w or c.startswith(w + "/"):
+        return True
+    tail = "/".join(w.split("/")[-2:])
+    return bool(tail) and (c == tail or c.startswith(tail + "/") or f"/{tail}/" in c)
+
+
+def new_skips() -> Counter:
+    """Counter for rows/files a reader dropped without a candidate row."""
+    return Counter()
+
+
+def bump(skips: Optional[Counter], reason: str, n: int = 1) -> None:
+    """Increment a skip counter when one is attached (never fatal)."""
+    if skips is not None:
+        skips[reason] += n
+
+
+def oversize_row(row: dict, max_messages: int) -> dict:
+    """Flag a candidate that exceeds the (soft) message cap.
+
+    ``max_messages`` is a SOFT cap: a session above it is still a
+    candidate because the pipeline bounds what it actually processes by
+    rendered size (``split_long`` / ``split_max_chars``), not by message
+    count. The flag lets callers report the long-session share instead
+    of silently discarding it.
+
+    Returns a COPY: the codex reader keeps the same dict in its on-disk
+    summary cache, and a persisted ``oversize`` flag would survive a
+    later ``max_messages`` change.
+    """
+    out = dict(row)
+    if max_messages and out.get("message_count", 0) > max_messages:
+        out["oversize"] = True
+    return out
+
+
+def workdir_of(cfg) -> str:
+    """Improver workdir for a Config (empty when unset)."""
+    return getattr(cfg, "improver_workdir", "") or ""
+
+
+# --------------------------------------------------------------- dispatch briefs
+# An orchestrator that dispatches a subagent pastes its own brief into the
+# child's first user row: 6-7KB of Orca plumbing (session/task ids, CLI docs,
+# worker_done rules) followed by a ``=== TASK ===`` block holding the real
+# instruction. Promoted as-is, the trainee learns to emit harness plumbing as
+# the user turn, so ``validate`` rejects the trace outright. Measured
+# 2026-10-24 on the raw pool: 130 claude-code traces open with this brief, 0
+# promoted, 20 gate failures. The TASK block IS the user's own words, so keep
+# exactly that and drop the plumbing.
+_DISPATCH_BRIEF_RE = re.compile(
+    r"You are working inside Orca, a multi-agent IDE"
+    r"|You are a dispatched worker"
+    r"|Your coordinator'?s Orca session ID is"
+    r"|=== CLI COMMANDS ===",
+    re.IGNORECASE,
+)
+_TASK_BLOCK = "=== TASK ==="
+# The brief arrives wrapped in the parent's paste envelope; the closing tag
+# trails the TASK block and is not part of the task.
+_PASTE_CLOSE_RE = re.compile(r"\s*</pasted_content[^>]*>\s*$", re.IGNORECASE)
+# A split segment's task row is the original task with this marker prepended
+# (splitter.save_segments). Unwrapping a brief must not eat it: the marker is
+# what tells the trainee the window is a continuation, not a fresh request.
+_SEGMENT_PREFIX_RE = re.compile(r"^(\[trace segment \d+ of \d+[^\]]*\]\s*)")
+
+# Orchestration notifications: the harness injecting "You have N orchestration
+# messages. Run `orca orchestration check`." into the conversation. Pure
+# plumbing, never user content — and because ``validate`` matches the bare
+# phrase ``orca orchestration``, ONE such row kills an otherwise clean trace
+# (measured 2026-10-24: 4 of 20 dispatch-brief gate failures had a clean task
+# row and a 65-char notice at row 168).
+_ORCH_NOTICE_RE = re.compile(
+    r"^\s*You have \d+ orchestration messages?\."
+    r"\s*Run `orca orchestration check",
+    re.IGNORECASE,
+)
+
+
+def is_orchestration_notice(text: Optional[str]) -> bool:
+    """True for the harness's 'You have N orchestration messages' row."""
+    return bool(_ORCH_NOTICE_RE.match(text or ""))
+
+
+def unwrap_dispatch_brief(text: str) -> str:
+    """The real task inside an orchestrator dispatch brief.
+
+    Returns ``text`` unchanged when it is not a brief, the ``=== TASK ===``
+    block when it is, and ``""`` when the brief carries no task block (pure
+    harness plumbing — the row must be dropped, not kept).
+    """
+    if not _DISPATCH_BRIEF_RE.search(text):
+        return text
+    if _TASK_BLOCK in text:
+        task = text.split(_TASK_BLOCK, 1)[1]
+        task = _PASTE_CLOSE_RE.sub("", task).strip()
+        seg = _SEGMENT_PREFIX_RE.match(text)
+        return (seg.group(1) + task) if seg else task
+    return ""
+
+
+def reframe_task_rows(messages: list, skips: Optional[Counter] = None) -> list:
+    """Drop harness plumbing user rows; unwrap dispatch-brief user rows.
+
+    Two loss points, one pass (both are the same defect: harness plumbing
+    sitting where the user's own words belong):
+
+      1. ``You have N orchestration messages`` notification rows — dropped
+         everywhere they appear, counted ``orchestration_notice_row``.
+      2. A dispatch brief in a user row — replaced by its ``=== TASK ===``
+         block (``dispatch_brief_reframed``), or dropped when the brief
+         carries no task (``dispatch_brief_no_task``). Every user row is
+         checked, not just the first: a codex child can receive a second
+         dispatch mid-thread, and the validate gate rejects the trace for
+         ANY row that still carries the boilerplate.
+
+    Returns a new list; the caller decides what an empty result means (a
+    taskless trace, which the reader's own taskless path then counts).
+    """
+    out = []
+    n_notice = 0
+    for m in messages:
+        if m.role == "user" and is_orchestration_notice(m.content):
+            n_notice += 1
+            continue
+        out.append(m)
+    bump(skips, "orchestration_notice_row", n_notice)
+
+    kept = []
+    for m in out:
+        if m.role == "user" and _DISPATCH_BRIEF_RE.search(m.content or ""):
+            task = unwrap_dispatch_brief(m.content)
+            if task:
+                m.content = task
+                bump(skips, "dispatch_brief_reframed")
+            else:
+                bump(skips, "dispatch_brief_no_task")
+            if not task:
+                continue
+        kept.append(m)
+    return kept
+

traceforge/readers/claude.py

@@ -0,0 +1,531 @@
+"""Claude Code trace reader — session JSONLs under ~/.claude/projects.
+
+Verified schema (Claude Code 2.x, probed 2026-09-26). Layout:
+  ~/.claude/projects/-<slugified-cwd>/<session-uuid>.jsonl
+one JSON object per line, relevant row types:
+  user        message:{role:"user", content: str | [tool_result blocks]}
+  assistant   message:{role:"assistant", content:[text|thinking|tool_use],
+              model, usage}
+  custom-title / ai-title        title sources (customTitle / aiTitle)
+  queue-operation, attachment, system, last-prompt, bridge-session, ... → skip
+
+Probed skip rules (UI/engine artifacts, never agent content):
+  - isSidechain rows INSIDE top-level session files: subagent traffic
+    relayed into the parent transcript (also: only top-level
+    <project>/*.jsonl files are scanned for parent sessions; the
+    <uuid>/ subdirs hold the subagents' OWN transcripts and are read
+    by the sidechain index — see include_sidechains)
+  - isMeta string rows starting with '<local-command-caveat>' or 'Another
+    Claude session sent a message' (subagent hand-back wrapper); other
+    isMeta rows carry real goal-continuation tasks and are KEPT
+  - user string rows starting with '<command-'/'<local-command-'/'<bash-'/
+    '<system-reminder>' — EXCEPT '/goal' command rows, whose <command-args>
+    carry the real task text, which is extracted as a user message
+  - the session-continuation row ('This session is being continued ...') is
+    KEPT as a user message (handoff context, cf. Hermes compaction rows);
+    Trace.first_task_message() skips it when picking the task
+
+Tool results live as tool_result blocks inside USER rows (results arrive in
+the row after the assistant turn, possibly several blocks batched for
+parallel calls). Tool names are resolved via a tool_use_id → name map built
+from the assistant tool_use blocks in the same file.
+"""
+from __future__ import annotations
+
+import json
+import os
+import re
+from datetime import datetime
+from typing import Any, Optional
+
+from traceforge.readers._common import (
+    bump, new_skips, oversize_row, reframe_task_rows,
+)
+from traceforge.trace import Message, ToolCall, Trace
+
+_ARTIFACT_PREFIXES = (
+    "<command-",
+    "<local-command-",
+    "<bash-",
+    "<system-reminder>",
+    "<task-",
+)
+_META_SKIP_PREFIXES = (
+    "<local-command-caveat>",
+    "Another Claude session sent a message",
+)
+CONTINUATION_PREFIX = "This session is being continued"
+_GOAL_ARGS_RE = re.compile(r"<command-args>(.*?)</command-args>", re.DOTALL)
+
+
+def _iso_to_epoch(ts: Optional[str]) -> Optional[float]:
+    if not ts:
+        return None
+    try:
+        return datetime.fromisoformat(str(ts).replace("Z", "+00:00")).timestamp()
+    except ValueError:
+        return None
+
+
+def _block_text(blocks: Any) -> str:
+    """Flatten a content-blocks list to text (tool_result content nests)."""
+    if isinstance(blocks, str):
+        return blocks
+    parts: list[str] = []
+    if not isinstance(blocks, list):
+        return ""
+    for b in blocks:
+        if isinstance(b, str):
+            parts.append(b)
+        elif isinstance(b, dict):
+            t = b.get("type")
+            if t == "text" and b.get("text"):
+                parts.append(str(b["text"]))
+            elif t == "image":
+                parts.append("[image]")
+            elif t == "tool_result" and b.get("content") is not None:
+                parts.append(_block_text(b["content"]))
+    return "\n\n".join(p for p in parts if p)
+
+
+class ClaudeTraceReader:
+    """Reads real traces from the Claude Code session store (read-only).
+
+    include_sidechains=True (default) also indexes the subagents' OWN
+    transcripts (<session-uuid>/subagents/agent-*.jsonl) as separate
+    candidate traces. Verified schema (probed 2026-10-12): the sidechain
+    files use the SAME user/assistant/tool_result row shapes as parent
+    session files, with isSidechain=true on every row. Sidechain
+    threads open with the dispatched lane-brief ("You are lane A of a
+    5-lane agent team…") — that row IS the task; the thread becomes a
+    self-contained trainable (task → work → result) trace, the same
+    doctrine as codex taskless-child recovery. MASS (arXiv 2610.12176):
+    orchestrator+subagent conversations are the densest RSI training
+    signal — skipping them discards the highest-value data in the store.
+    """
+
+    def __init__(
+        self,
+        projects_dir: Optional[list[str] | str] = None,
+        *,
+        include_sidechains: bool = True,
+    ):
+        # Multi-store support: a list of projects dirs is indexed
+        # together (live store + seed archives). Same session uuid in
+        # two stores -> the LARGER file wins (fresher snapshot of the
+        # same session), matching the single-store collision rule.
+        if projects_dir is None:
+            self.projects_dirs = [
+                os.path.expanduser(os.path.join("~", ".claude", "projects"))
+            ]
+        elif isinstance(projects_dir, (list, tuple)):
+            self.projects_dirs = [
+                os.path.expanduser(d) for d in projects_dir
+            ]
+        else:
+            self.projects_dirs = [os.path.expanduser(projects_dir)]
+        self.include_sidechains = include_sidechains
+        # files/rows dropped without a candidate row: reported by stats()
+        self.skips = new_skips()
+        # memoized _index() — see the docstring there (double-counted skips)
+        self._idx_cache: Optional[dict[str, str]] = None
+
+    @property
+    def projects_dir(self) -> str:
+        """First store (backwards compat for callers/tests)."""
+        return self.projects_dirs[0]
+
+    # ---------------------------------------------------------------- index
+
+    def _index(self) -> dict[str, str]:
+        """session uuid -> newest (largest) jsonl path across stores.
+
+        Memoized (2026-10-24): the index walk is what bumps the file-level
+        skip counters (``sidechain_sidecar``, ``non_transcript_file``), so
+        re-walking it — ``stats()`` used to call ``_index()`` a second time
+        after ``list_sessions`` — reported every file skip TWICE (measured:
+        2,684 sidecars where the store holds 1,342). A skip census that
+        doubles itself is worse than no census.
+        """
+        if self._idx_cache is not None:
+            return self._idx_cache
+        idx: dict[str, tuple[int, str]] = {}
+        found = False
+        for projects_dir in self.projects_dirs:
+            if not os.path.isdir(projects_dir):
+                continue  # missing store (seed archive) is not an error
+            found = True
+            self._index_store(projects_dir, idx)
+        if not found:
+            raise FileNotFoundError(
+                f"Claude projects dir not found: {self.projects_dirs}"
+            )
+        self._idx_cache = {sid: path for sid, (_, path) in idx.items()}
+        return self._idx_cache
+
+    def _index_store(self, projects_dir: str, idx: dict) -> None:
+        local_sids: list[str] = []  # sids discovered in THIS store
+        for proj in sorted(os.listdir(projects_dir)):
+            proj_dir = os.path.join(projects_dir, proj)
+            if not os.path.isdir(proj_dir):
+                continue  # files at top level are never traces
+            try:
+                entries = os.listdir(proj_dir)
+            except OSError:
+                continue
+            for fn in entries:
+                if not fn.endswith(".jsonl"):
+                    # A non-transcript FILE (a .json sidecar, a stray .txt)
+                    # is counted, not just .json — the counter used to miss
+                    # every other extension. Directories are the <uuid>/
+                    # sidechain containers, not skips.
+                    if os.path.isfile(os.path.join(proj_dir, fn)):
+                        bump(self.skips, "non_transcript_file")
+                    continue  # <uuid>/ subdirs: parent's sidechains live inside
+                sid = fn[:-6]
+                path = os.path.join(proj_dir, fn)
+                try:
+                    size = os.path.getsize(path)
+                except OSError:
+                    bump(self.skips, "unreadable_file")
+                    continue
+                prev = idx.get(sid)
+                if prev is None or size > prev[0]:
+                    idx[sid] = (size, path)
+                local_sids.append(sid)
+        if self.include_sidechains:
+            # Only THIS store's sids (2026-10-24). Iterating the global idx
+            # re-scanned every earlier store's subagents dir on each store
+            # pass, so a two-store setup counted the live store's sidecars
+            # twice (measured: 1,342 reported where the stores hold 1,169).
+            for sid in local_sids:
+                path = idx[sid][1]
+                sub_dir = os.path.join(
+                    os.path.dirname(path), sid, "subagents"
+                )
+                if not os.path.isdir(sub_dir):
+                    continue
+                try:
+                    subs = sorted(os.listdir(sub_dir))
+                except OSError:
+                    continue
+                for fn in subs:
+                    if not (fn.startswith("agent-") and fn.endswith(".jsonl")):
+                        # agent-*.meta.json sidecars (and the odd
+                        # `workflows/` dir): metadata, not a transcript.
+                        # Counted so the skip is visible.
+                        bump(self.skips, "sidechain_sidecar")
+                        continue
+                    # namespaced id: <parent>#sa-<agentfile-stem> —
+                    # '#' cannot appear in session uuids, so the raw-cache
+                    # namespace stays collision-free (same rule as the
+                    # splitter's <sid>#pN segments)
+                    sub_sid = f"{sid}#sa-{fn[6:-6]}"
+                    sub_path = os.path.join(sub_dir, fn)
+                    try:
+                        sub_size = os.path.getsize(sub_path)
+                    except OSError:
+                        continue
+                    prev = idx.get(sub_sid)
+                    if prev is None or sub_size > prev[0]:
+                        idx[sub_sid] = (sub_size, sub_path)
+        return {sid: path for sid, (_, path) in idx.items()}
+
+    # ---------------------------------------------------------------- parse
+
+    def _parse_file(self, path: str, session_id: Optional[str] = None) -> Trace:
+        apath = os.path.abspath(path).replace("/", os.sep)
+        # a subagent's OWN transcript: every row carries isSidechain=true
+        # (kept here; only RELAYED sidechain rows inside parent files
+        # are skipped) and the sid defaults to the namespaced form
+        is_sidechain = os.sep + "subagents" + os.sep in apath
+        if session_id:
+            sid = session_id
+        elif is_sidechain:
+            stem = os.path.basename(path)[:-6]
+            parent = os.path.basename(os.path.dirname(os.path.dirname(path)))
+            sid = f"{parent}#sa-{stem[len('agent-'):]}"
+        else:
+            sid = os.path.basename(path)[:-6]
+        messages: list[Message] = []
+        tool_names: dict[str, str] = {}  # tool_use_id -> name
+        model: Optional[str] = None
+        title_custom: Optional[str] = None
+        title_ai: Optional[str] = None
+        cwd: Optional[str] = None
+        started: Optional[float] = None
+        last_ts: Optional[float] = None
+
+        with open(path, "r", encoding="utf-8", errors="replace") as f:
+            for line in f:
+                try:
+                    d = json.loads(line)
+                except json.JSONDecodeError:
+                    bump(self.skips, "unparseable_line")
+                    continue  # truncated/interleaved tail lines
+                if not isinstance(d, dict):
+                    continue
+                ts = _iso_to_epoch(d.get("timestamp"))
+                if started is None and ts is not None:
+                    started = ts
+                if ts is not None:
+                    last_ts = ts
+
+                t = d.get("type")
+                if t == "custom-title":
+                    title_custom = d.get("customTitle") or title_custom
+                    continue
+                if t == "ai-title":
+                    title_ai = d.get("aiTitle") or title_ai
+                    continue
+                if t not in ("user", "assistant"):
+                    continue
+                if d.get("isSidechain") and not is_sidechain:
+                    continue  # relayed subagent traffic inside a parent file
+                if cwd is None and d.get("cwd"):
+                    cwd = d["cwd"]
+
+                m = d.get("message")
+                if not isinstance(m, dict):
+                    continue
+                content = m.get("content")
+
+                if t == "assistant":
+                    if model is None and m.get("model"):
+                        model = str(m["model"])
+                    texts: list[str] = []
+                    thinks: list[str] = []
+                    calls: list[ToolCall] = []
+                    if isinstance(content, list):
+                        for blk in content:
+                            if not isinstance(blk, dict):
+                                continue
+                            bt = blk.get("type")
+                            if bt == "text" and blk.get("text"):
+                                texts.append(str(blk["text"]))
+                            elif bt == "thinking" and blk.get("thinking"):
+                                thinks.append(str(blk["thinking"]))
+                            elif bt == "tool_use":
+                                tc_id = str(blk.get("id") or "")
+                                name = str(blk.get("name") or "")
+                                if tc_id:
+                                    tool_names[tc_id] = name
+                                calls.append(
+                                    ToolCall(
+                                        id=tc_id,
+                                        name=name,
+                                        arguments=json.dumps(
+                                            blk.get("input") or {},
+                                            ensure_ascii=False,
+                                        ),
+                                    )
+                                )
+                    if not (texts or thinks or calls):
+                        continue
+                    messages.append(
+                        Message(
+                            role="assistant",
+                            content="\n\n".join(texts) or None,
+                            reasoning="\n\n".join(thinks) or None,
+                            tool_calls=calls,
+                            timestamp=ts,
+                        )
+                    )
+
+                elif t == "user":
+                    if isinstance(content, str):
+                        if d.get("isMeta") and content.startswith(
+                            _META_SKIP_PREFIXES
+                        ):
+                            continue
+                        if content.startswith("<command-name>"):
+                            # /goal rows carry the task in <command-args>
+                            if "/goal" in content[:80]:
+                                mm = _GOAL_ARGS_RE.search(content)
+                                if mm and mm.group(1).strip():
+                                    messages.append(
+                                        Message(
+                                            role="user",
+                                            content=mm.group(1).strip(),
+                                            timestamp=ts,
+                                        )
+                                    )
+                            continue
+                        if content.startswith(_ARTIFACT_PREFIXES):
+                            continue
+                        messages.append(
+                            Message(role="user", content=content, timestamp=ts)
+                        )
+                    elif isinstance(content, list):
+                        for blk in content:
+                            if (
+                                not isinstance(blk, dict)
+                                or blk.get("type") != "tool_result"
+                            ):
+                                continue
+                            tc_id = str(blk.get("tool_use_id") or "")
+                            body = _block_text(blk.get("content"))
+                            if not body and blk.get("is_error"):
+                                body = "(tool error)"
+                            if not body and not tc_id:
+                                continue
+                            messages.append(
+                                Message(
+                                    role="tool",
+                                    content=body or None,
+                                    tool_call_id=tc_id,
+                                    tool_name=tool_names.get(tc_id),
+                                    timestamp=ts,
+                                )
+                            )
+
+        # Harness plumbing in user rows (2026-10-24): an orchestrator's
+        # dispatch brief pasted as the task row, and the harness's "You have
+        # N orchestration messages" notifications. Both are plumbing where
+        # the user's own words belong; the brief's `=== TASK ===` block is
+        # the real request, so keep that and drop the rest. Measured: 130
+        # claude-code raw traces opened with the brief, 0 promoted, 20 gate
+        # failures — and 4 of those had a CLEAN task row killed by a 65-char
+        # notice at row 168.
+        messages = reframe_task_rows(messages, self.skips)
+
+        # Subagent-brief recovery (MASS doctrine, mirrors codex taskless-
+        # child recovery): a sidechain thread opens with the dispatched
+        # lane-brief — already a real first user row in Claude files, so
+        # nothing to synthesize. But a sidechain whose rows are ALL
+        # assistant/tool (handback content only) stays taskless — drop it
+        # here rather than letting it die at the improve gates.
+        if is_sidechain and not any(
+            m.role == "user" for m in messages
+        ):
+            return Trace(session_id=sid, model=model, title=None,
+                         started_at=started or last_ts, cwd=cwd,
+                         messages=[], source="claude-sidechain")
+
+        return Trace(
+            session_id=sid,
+            model=model,
+            title=title_custom or title_ai,
+            started_at=started or last_ts,
+            cwd=cwd,
+            messages=messages,
+            source="claude-sidechain" if is_sidechain else "claude-code",
+        )
+
+    # ------------------------------------------------------------- sessions
+
+    def list_sessions(
+        self,
+        *,
+        min_messages: int = 10,
+        max_messages: int = 120,
+        min_tool_calls: int = 2,
+        limit: Optional[int] = None,
+    ) -> list[dict[str, Any]]:
+        """Candidate traces for improvement, newest first (summaries only).
+
+        ``max_messages`` is a SOFT cap (2026-10-23): a session above it is
+        still listed, flagged ``oversize``. The hard cap it replaced
+        discarded 1,217 real claude sessions (median 245 messages, 107
+        tool calls, ~25% of them over ``split_max_chars`` and therefore
+        split candidates) before they could reach ``split_long``.
+        """
+        rows: list[dict[str, Any]] = []
+        for sid, path in self._index().items():
+            try:
+                trace = self._parse_file(path, sid)
+            except OSError:
+                bump(self.skips, "unreadable_file")
+                continue
+            n_msgs = len(trace.messages)
+            if not n_msgs:
+                bump(self.skips, "taskless_sidechain"
+                     if trace.source == "claude-sidechain" else "empty_trace")
+                continue
+            if n_msgs < min_messages:
+                bump(self.skips, "below_min_messages")
+                continue
+            n_calls = sum(len(m.tool_calls) for m in trace.messages)
+            if n_calls < min_tool_calls:
+                bump(self.skips, "below_min_tool_calls")
+                continue
+            roles: dict[str, int] = {}
+            for m in trace.messages:
+                roles[m.role] = roles.get(m.role, 0) + 1
+            rows.append(
+                oversize_row(
+                    {
+                        "session_id": sid,
+                        "model": trace.model,
+                        "title": (trace.title or "")[:120].strip(),
+                        "started_at": trace.started_at,
+                        "ended_at": trace.messages[-1].timestamp
+                        if trace.messages
+                        else None,
+                        "message_count": n_msgs,
+                        "tool_call_count": n_calls,
+                        "cwd": trace.cwd,
+                        "source": trace.source,
+                        "roles": roles,
+                    },
+                    max_messages,
+                )
+            )
+        rows.sort(key=lambda r: r["started_at"] or 0.0, reverse=True)
+        if limit is not None:
+            rows = rows[:limit]
+        return rows
+
+    def load_trace(self, session_id: str) -> Trace:
+        """Load one full trace as a normalized Trace."""
+        path = self._index().get(session_id)
+        if not path:
+            raise KeyError(f"claude session not found: {session_id}")
+        return self._parse_file(path, session_id)
+
+    # -------------------------------------------------------------- helpers
+
+    def stats(self) -> dict[str, Any]:
+        """Counts over the store(s), plus the skip census.
+
+        The parse loop runs against a SCRATCH counter (2026-10-24): stats()
+        re-parses every session to total messages/tool calls, and those
+        parses bump the row-level skip counters again — so a census taken
+        after stats() reported every reframed brief and dropped notice row
+        twice. stats() must observe the funnel, not re-count it.
+        """
+        idx = self._index()
+        n_msgs = 0
+        n_calls = 0
+        real_skips = self.skips
+        census = new_skips()
+        self.skips = census
+        try:
+            for _, path in idx.items():
+                try:
+                    t = self._parse_file(path)
+                except OSError:
+                    continue
+                n_msgs += len(t.messages)
+                n_calls += sum(len(m.tool_calls) for m in t.messages)
+        finally:
+            self.skips = real_skips
+        return {
+            "projects": sum(
+                len(
+                    [
+                        e
+                        for e in os.listdir(d)
+                        if os.path.isdir(os.path.join(d, e))
+                    ]
+                )
+                for d in self.projects_dirs
+                if os.path.isdir(d)
+            ),
+            "sessions": len(idx),
+            "messages": n_msgs,
+            "tool_calls": n_calls,
+            # the scratch counter, not the restored one: reading self.skips
+            # here returned the caller's counter and threw the census away
+            "skips": dict(census),
+        }

traceforge/readers/codex.py

@@ -0,0 +1,637 @@
+"""Codex trace reader — rollout JSONLs under ~/.codex/sessions (and exports).
+
+Verified schema (Codex Desktop 0.145.x, probed 2026-10-07 against the
+`exported-seed-datas` archives + the live store). Layout:
+  <root>/YYYY/MM/DD/rollout-<iso>-<thread-uuid>.jsonl
+one JSON object per line, top-level `type`:
+  session_meta   payload:{session_id, id (=thread id, unique per file),
+                   cwd, timestamp, originator, cli_version, source{...}}
+  turn_context   payload:{turn_id, cwd, model, effort, ...} (engine noise)
+  response_item  the actual conversation (payload.type below)
+  event_msg / world_state  engine telemetry — skipped
+
+response_item payload types:
+  message                    role:user|assistant|developer, content:
+                             [{type:"input_text"|"output_text", text}]
+  reasoning                  summary:[{type:"summary_text", text}] (when
+                             summaries are enabled; encrypted_content
+                             otherwise — skipped)
+  custom_tool_call           id, call_id, name, input (STRING of JS code
+                             for the `exec` tool — kept verbatim)
+  custom_tool_call_output    call_id, output:[{type:"input_text", text}]
+  function_call / function_call_output  same via call_id
+  agent_message / tool_search_call / ...  multi-agent chatter — skipped
+
+Probed skip rules (harness artifacts, never real user content):
+  user rows starting with <recommended_plugins>, <subagent_notification>,
+  <turn_aborted>, <environment_context>, <permissions, <user_instructions
+  user rows starting with '# AGENTS.md' or 'The following is the Codex'
+  (harness preamble), and ALL developer-role rows (engine instructions)
+Tool results attach via call_id -> the preceding call row. Session id =
+the session_meta payload.id (thread id in the filename), NOT session_id
+(forked rollouts share the parent's session_id; files would collide).
+
+Self-reference guard: rollouts whose session_meta cwd sits inside the
+improver workdir are dropped from listing (same doctrine as hermes).
+"""
+from __future__ import annotations
+
+import glob
+import json
+import os
+import re
+import sys
+import threading
+import time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from datetime import datetime
+from typing import Any, Optional
+
+from traceforge.readers._common import (
+    bump, in_workdir, new_skips, oversize_row, reframe_task_rows,
+    unwrap_dispatch_brief,
+)
+from traceforge.trace import Message, ToolCall, Trace
+
+_USER_ARTIFACT_PREFIXES = (
+    "<recommended_plugins>",
+    "<subagent_notification>",
+    "<turn_aborted>",
+    "<environment_context>",
+    "<permissions",
+    "<user_instructions",
+    "<plugins_available",
+)
+_USER_ARTIFACT_TEXT_PREFIXES = (
+    "# AGENTS.md",
+    "The following is the Codex",
+)
+_ROLLOUT_RE = re.compile(r"rollout-.*\.jsonl(?:\.zst)?$")
+_ROLLOUT_SUFFIXES = (".jsonl.zst", ".jsonl")
+
+
+def _strip_rollout_suffix(name: str) -> str:
+    """rollout-<iso>-<uuid>.jsonl[.zst] -> rollout-<iso>-<uuid>."""
+    for suf in _ROLLOUT_SUFFIXES:
+        if name.endswith(suf):
+            return name[: -len(suf)]
+    return name
+
+
+def _open_rollout(path: str):
+    """Open a rollout as text, transparently decompressing ``.jsonl.zst``.
+
+    Codex compresses older rollouts in place — 107 of 1387 files on this host
+    (measured 2026-10-22), all of them older than the live ones. The
+    decompressed bytes are the same JSONL, so only the opener changes.
+
+    Python 3.14 ships ``compression.zstd``; on 3.10-3.13 this needs the
+    ``zstandard`` package. Both are optional, so a missing backend raises a
+    clear error rather than silently skipping the file.
+    """
+    if not path.endswith(".zst"):
+        return open(path, "r", encoding="utf-8", errors="replace")
+    try:
+        from compression import zstd as _zstd  # Python 3.14+
+    except ImportError:
+        _zstd = None
+    if _zstd is not None:
+        return _zstd.open(path, "rt", encoding="utf-8", errors="replace")
+    try:
+        import zstandard  # type: ignore
+    except ImportError as exc:  # pragma: no cover - depends on interpreter
+        raise RuntimeError(
+            f"cannot read compressed rollout {path!r}: install the "
+            "'zstandard' package or run on Python 3.14+"
+        ) from exc
+    return zstandard.open(path, "rt", encoding="utf-8", errors="replace")
+_ISO_RE = re.compile(r"(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})")
+# Completion-report openings: work summaries, not task restatements.
+# Anchored at string start (case-insensitive); "Implemented …", "Wrote …".
+_REPORT_OPENING_RE = re.compile(
+    r"^\s*(?:i\s+)?(?:ich\s+)?(?:have\s+|hab\s+|habe\s+)?"
+    r"(?:implemented|created|fixed|added|built|completed|wrote|done)\b",
+    re.I,
+)
+
+
+def _iso_to_epoch(ts: Optional[str]) -> Optional[float]:
+    if not ts:
+        return None
+    try:
+        return datetime.fromisoformat(str(ts).replace("Z", "+00:00")).timestamp()
+    except ValueError:
+        return None
+
+
+def _unwrap_dispatch_brief(text: str) -> str:
+    """Return the real task from an Orca coordinator dispatch brief.
+
+    A codex subagent rollout opens with the coordinator's dispatch text: 6-7KB
+    of orchestration boilerplate (session/task ids, `orca orchestration` CLI
+    docs, worker_done rules) followed by a `=== TASK ===` block holding the
+    actual instruction. Promoted as-is, the trainee learns to emit harness
+    plumbing as the user turn (measured 2026-10-22: 81 raw traces, 6 already
+    promoted). The TASK block is the user's own words, so keep exactly that;
+    a brief with no TASK block yields "" and the row is dropped.
+
+    Thin alias over the shared helper (2026-10-24): the claude reader needs
+    the same unwrap, and two copies of the pattern drift. Kept as a module
+    name because tests and callers import it from here.
+    """
+    return unwrap_dispatch_brief(text)
+
+
+def recover_taskless_brief(messages: list, role_hint: str = "") -> bool:
+    """Synthesize a task row for a taskless child thread (in place).
+
+    Subagent handoff recovery (owner directive 2026-10-07: taskless child
+    threads are valuable work traces, don't drop them): when the rollout
+    carries no real user row, codex children open with the agent restating
+    its dispatched brief in task terms. That restatement becomes a
+    synthesized first user row so the trace is a trainable
+    (task → work → result) pair; the improve pipeline's reframe path and
+    judges still verify task↔work fit. Completion-REPORT-shaped openings
+    ("Implemented …", "Wrote …") are work summaries, not tasks — they stay
+    taskless.
+
+    Shared with the raw-pool repair pass (2026-10-24): dropping the
+    harness's orchestration-notification rows can leave a trace with no
+    user row at all, and a cached trace must land in the same state a
+    fresh pull would produce.
+
+    Returns True when a row was inserted.
+    """
+    if any(m.role == "user" for m in messages):
+        return False
+    first_asst = next(
+        (m for m in messages if m.role == "assistant" and (m.content or "")),
+        None,
+    )
+    if (
+        first_asst is None
+        or len(first_asst.content or "") < 80
+        or _REPORT_OPENING_RE.search(first_asst.content or "")
+    ):
+        return False
+    messages.insert(
+        0,
+        Message(
+            role="user",
+            content=role_hint + (first_asst.content or "").strip(),
+            timestamp=first_asst.timestamp,
+        ),
+    )
+    return True
+
+
+def _content_text(blocks: Any) -> str:
+    """Flatten content-block lists (message bodies, tool outputs)."""
+    if isinstance(blocks, str):
+        return blocks
+    if not isinstance(blocks, list):
+        return ""
+    parts: list[str] = []
+    for b in blocks:
+        if isinstance(b, dict):
+            t = b.get("text")
+            if t:
+                parts.append(str(t))
+        elif isinstance(b, str):
+            parts.append(b)
+    return "\n\n".join(p for p in parts if p)
+
+
+class CodexTraceReader:
+    """Reads real traces from Codex rollout session stores (read-only)."""
+
+    def __init__(self, roots: Optional[list[str]] = None):
+        if roots:
+            self.roots = [os.path.expanduser(r) for r in roots]
+        else:
+            self.roots = [os.path.expanduser("~/.codex/sessions")]
+        # files/rows dropped without a candidate row: reported by stats()
+        self.skips = new_skips()
+
+    # ---------------------------------------------------------------- index
+
+    def _agent_role(self, path: str) -> Optional[str]:
+        """Subagent role (agent_path/agent_nickname) from session_meta."""
+        try:
+            with _open_rollout(path) as f:
+                for line in f:
+                    try:
+                        d = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    if d.get("type") == "session_meta":
+                        pay = d.get("payload") or {}
+                        role = pay.get("agent_path") or pay.get("agent_nickname")
+                        if role:
+                            return str(role).strip("/")
+                        return None
+        except OSError:
+            pass
+        return None
+
+    def _rollout_files(self) -> list[str]:
+        files: list[str] = []
+        for root in self.roots:
+            if not os.path.isdir(root):
+                continue
+            for p in glob.glob(os.path.join(root, "**", "*.jsonl"), recursive=True):
+                if _ROLLOUT_RE.search(os.path.basename(p)):
+                    files.append(p)
+            for p in glob.glob(os.path.join(root, "**", "*.jsonl.zst"), recursive=True):
+                if _ROLLOUT_RE.search(os.path.basename(p)):
+                    files.append(p)
+        files.sort()
+        return files
+
+    def _index(self) -> dict[str, str]:
+        """thread id (session_meta payload.id) -> rollout path."""
+        idx: dict[str, str] = {}
+        for path in self._rollout_files():
+            sid = self._thread_id(path)
+            if sid:
+                idx[sid] = path
+            else:
+                bump(self.skips, "no_thread_id")
+        return idx
+
+    def _thread_id(self, path: str) -> Optional[str]:
+        """session_meta payload.id — the thread id from the filename.
+
+        The last uuid in the filename IS the thread id; parsing it avoids
+        scanning the file for sessions whose meta row was truncated away.
+        Fallback: scan for session_meta when the name is non-standard.
+        """
+        base = _strip_rollout_suffix(os.path.basename(path))
+        mm = re.search(r"([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$", base, re.I)
+        if mm:
+            return mm.group(1)
+        for line in _open_rollout(path):
+            try:
+                d = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            if d.get("type") == "session_meta":
+                pid = (d.get("payload") or {}).get("id")
+                if pid:
+                    return str(pid)
+        return None
+
+    # ---------------------------------------------------------------- parse
+
+    def _parse_file(self, path: str, session_id: Optional[str] = None) -> Trace:
+        sid = session_id or self._thread_id(path) or _strip_rollout_suffix(os.path.basename(path))
+        messages: list[Message] = []
+        call_names: dict[str, str] = {}  # call_id -> tool name
+        model: Optional[str] = None
+        cwd: Optional[str] = None
+        started: Optional[float] = None
+        last_ts: Optional[float] = None
+        title: Optional[str] = None  # first real user task, set below
+        pending_reasoning: Optional[str] = None  # reasoning buffered for the next assistant row
+
+        with _open_rollout(path) as f:
+            for line in f:
+                try:
+                    d = json.loads(line)
+                except json.JSONDecodeError:
+                    bump(self.skips, "unparseable_line")
+                    continue
+                if not isinstance(d, dict):
+                    continue
+                ts = _iso_to_epoch(d.get("timestamp"))
+                if started is None and ts is not None:
+                    started = ts
+                if ts is not None:
+                    last_ts = ts
+
+                t = d.get("type")
+                if t == "session_meta":
+                    pay = d.get("payload") or {}
+                    cwd = cwd or pay.get("cwd")
+                elif t == "turn_context":
+                    pay = d.get("payload") or {}
+                    if model is None and pay.get("model"):
+                        model = str(pay["model"])
+                    cwd = cwd or pay.get("cwd")
+                elif t != "response_item":
+                    continue
+
+                pay = d.get("payload") or {}
+                pt = pay.get("type")
+
+                if pt == "message":
+                    role = pay.get("role")
+                    text = _content_text(pay.get("content"))
+                    if not text:
+                        continue
+                    if role == "user":
+                        if text.startswith(_USER_ARTIFACT_PREFIXES) or text.startswith(
+                            _USER_ARTIFACT_TEXT_PREFIXES
+                        ):
+                            continue
+                        messages.append(
+                            Message(role="user", content=text, timestamp=ts)
+                        )
+                    elif role == "assistant":
+                        messages.append(
+                            Message(role="assistant", content=text, timestamp=ts)
+                        )
+                    # developer-role rows: engine instructions, skipped
+
+                elif pt == "reasoning":
+                    # codex emits reasoning BEFORE its assistant row
+                    # (reasoning -> message / custom_tool_call). Buffer and
+                    # attach to the next assistant message. Encrypted-only
+                    # reasoning rows produce no text and are skipped.
+                    s = pay.get("summary") or []
+                    text = _content_text(s)
+                    if text:
+                        pending_reasoning = (
+                            f"{pending_reasoning}\n\n{text}"
+                            if pending_reasoning
+                            else text
+                        )
+
+                elif pt in ("custom_tool_call", "function_call"):
+                    name = str(pay.get("name") or "")
+                    call_id = str(pay.get("call_id") or pay.get("id") or "")
+                    if call_id:
+                        call_names[call_id] = name
+                    inp = pay.get("input")
+                    if isinstance(inp, str):
+                        # `exec` custom tool calls carry raw JS code, not a
+                        # JSON object — normalize to {"code": ...} so the
+                        # structural gate (JSON tool args) stays satisfiable
+                        # while the code remains byte-verbatim.
+                        args = json.dumps({"code": inp}, ensure_ascii=False)
+                    else:
+                        args = json.dumps(inp or {}, ensure_ascii=False)
+                    messages.append(
+                        Message(
+                            role="assistant",
+                            content=None,
+                            reasoning=pending_reasoning or None,
+                            tool_calls=[
+                                ToolCall(id=call_id, name=name, arguments=args)
+                            ],
+                            timestamp=ts,
+                        )
+                    )
+                    pending_reasoning = None
+
+                elif pt in ("custom_tool_call_output", "function_call_output"):
+                    call_id = str(pay.get("call_id") or "")
+                    out = pay.get("output")
+                    body = _content_text(out) if isinstance(out, list) else (
+                        str(out) if out is not None else ""
+                    )
+                    messages.append(
+                        Message(
+                            role="tool",
+                            content=body or None,
+                            tool_call_id=call_id,
+                            tool_name=call_names.get(call_id),
+                            timestamp=ts,
+                        )
+                    )
+
+        # Harness plumbing in user rows (2026-10-24): the coordinator's
+        # dispatch brief (keep its `=== TASK ===` block, drop the 6-7KB of
+        # Orca boilerplate) and the harness's "You have N orchestration
+        # messages" notifications. Both are plumbing where the user's own
+        # words belong, and the validate gate rejects the trace for either.
+        # Runs BEFORE the title pick so the title is the real task.
+        messages = reframe_task_rows(messages, self.skips)
+
+        # title: first real user task (codex has no title rows)
+        for m in messages:
+            if m.role == "user" and m.content:
+                title = " ".join(m.content.split())[:120]
+                break
+
+        # Subagent handoff recovery (owner directive 2026-10-07: taskless
+        # child threads are valuable work traces, don't drop them): when
+        # the rollout carries no real user row, codex children open with
+        # the agent restating its dispatched brief in task terms. That
+        # restatement becomes a synthesized first user row so the trace
+        # is a trainable (task → work → result) pair; the improve
+        # pipeline's reframe path and judges still verify task↔work fit.
+        # Completion-REPORT-shaped openings ("Implemented …", "Wrote …")
+        # are work summaries, not tasks — they stay taskless.
+        if not any(m.role == "user" for m in messages):
+            role = self._agent_role(path)  # extra file read: taskless only
+            recover_taskless_brief(
+                messages, f"[subagent brief — role: {role}]\n" if role else ""
+            )
+
+        return Trace(
+            session_id=sid,
+            model=model,
+            title=title,
+            started_at=started or last_ts,
+            cwd=cwd,
+            messages=messages,
+            source="codex",
+        )
+
+    # ------------------------------------------------------------- sessions
+
+    def _cache_path(self) -> str:
+        """Per-roots summary cache file (hash of the roots list)."""
+        import hashlib
+
+        key = hashlib.sha1(
+            "\x00".join(self.roots).encode("utf-8", "replace")
+        ).hexdigest()[:12]
+        base = next(
+            (r for r in self.roots if os.path.isdir(r)), None
+        ) or os.getcwd()
+        d = os.path.join(os.path.dirname(os.path.abspath(base)), ".cache")
+        try:
+            os.makedirs(d, exist_ok=True)
+        except OSError:
+            return os.path.join(
+                os.environ.get("TMPDIR", os.getcwd()), f".tf-codex-{key}.json"
+            )
+        return os.path.join(d, f"codex-summaries-{key}.json")
+
+    def _load_cache(self) -> dict:
+        try:
+            with open(self._cache_path(), "r", encoding="utf-8") as f:
+                d = json.load(f)
+            if isinstance(d, dict) and d.get("v") == 1:
+                return d.get("entries", {})
+        except (OSError, ValueError):
+            pass
+        return {}
+
+    def _save_cache(self, entries: dict) -> None:
+        try:
+            tmp = self._cache_path() + ".tmp"
+            with open(tmp, "w", encoding="utf-8") as f:
+                json.dump({"v": 1, "entries": entries}, f,
+                          separators=(",", ":"))
+            os.replace(tmp, self._cache_path())
+        except OSError:
+            pass  # cache is best-effort, never fatal
+
+    def list_sessions(
+        self,
+        *,
+        min_messages: int = 10,
+        max_messages: int = 120,
+        min_tool_calls: int = 2,
+        exclude_cwd: Optional[str] = None,
+        limit: Optional[int] = None,
+        status_cb=None,
+        use_cache: bool = True,
+    ) -> list[dict[str, Any]]:
+        """Candidate traces for improvement, newest first (summaries only).
+
+        Per-file summaries are cached (path -> {sig: size+mtime, row})
+        under the store's parent .cache dir; only new/changed files
+        parse. Parsing is thread-parallel — ~3.8k rollouts ≈ 10 GB
+        parse in ~20s wall on the first run, near-instant after.
+        ``status_cb(msg)`` gets a discovery heartbeat while it runs.
+
+        ``use_cache=False`` forces a full parse (2026-10-24). The row-level
+        skip counters are bumped during parsing, so a warm cache made the
+        census report near-zero: measured, 251 rollouts carry a dispatch
+        brief and the cached run counted 0 reframes. ``stats()`` passes
+        False so its census is authoritative.
+
+        ``max_messages`` is a SOFT cap (2026-10-23): a rollout above it is
+        still listed, flagged ``oversize``. The hard cap it replaced
+        discarded 1,246 real codex rollouts (median 210 messages, 98 tool
+        calls) before they could reach ``split_long``.
+        """
+        idx = self._index()
+        cached = self._load_cache() if use_cache else {}
+        entries: dict[str, Any] = {}
+        stale: list[tuple[str, str, str]] = []
+        for sid, path in idx.items():
+            try:
+                sig = f"{os.path.getsize(path):x}-{int(os.path.getmtime(path)):x}"
+            except OSError:
+                # stat failed: the file vanished or is locked. Counted —
+                # this used to drop the rollout with no trace of it.
+                bump(self.skips, "unreadable_file")
+                continue
+            c = cached.get(path)
+            if c and c.get("sig") == sig:
+                entries[path] = c
+            else:
+                stale.append((sid, path, sig))
+        rows: list[dict[str, Any]] = [
+            dict(e["row"]) for e in entries.values()
+        ]
+        if stale:
+            t0 = time.monotonic()
+            done = 0
+            lock = threading.Lock()
+
+            def _parse_one(item):
+                sid, path, sig = item
+                try:
+                    trace = self._parse_file(path, sid)
+                except OSError:
+                    bump(self.skips, "unreadable_file")
+                    return None
+                return path, sig, trace
+
+            workers = min(8, max(1, len(stale)))
+            with ThreadPoolExecutor(max_workers=workers) as ex:
+                futs = [ex.submit(_parse_one, it) for it in stale]
+                for f in as_completed(futs):
+                    got = f.result()
+                    with lock:
+                        done += 1
+                        if status_cb and (
+                            done % 250 == 0 or done == len(stale)
+                        ):
+                            status_cb(
+                                f"discovery: {done}/{len(stale)} rollouts "
+                                f"({time.monotonic() - t0:.0f}s)"
+                            )
+                    if got is None:
+                        continue
+                    path, sig, trace = got
+                    roles: dict[str, int] = {}
+                    for m in trace.messages:
+                        roles[m.role] = roles.get(m.role, 0) + 1
+                    row = {
+                        "session_id": trace.session_id,
+                        "model": trace.model,
+                        "title": (trace.title or "")[:120].strip(),
+                        "started_at": trace.started_at,
+                        "ended_at": trace.messages[-1].timestamp
+                        if trace.messages
+                        else None,
+                        "message_count": len(trace.messages),
+                        "tool_call_count": sum(
+                            len(m.tool_calls) for m in trace.messages
+                        ),
+                        "cwd": trace.cwd,
+                        "source": "codex",
+                        "roles": roles,
+                    }
+                    rows.append(row)
+                    entries[path] = {"sig": sig, "row": row}
+            self._save_cache(entries)
+        out: list[dict[str, Any]] = []
+        for r in rows:
+            if exclude_cwd and in_workdir(r["cwd"], exclude_cwd):
+                bump(self.skips, "cwd_self_reference")
+                continue
+            if r["message_count"] < min_messages:
+                bump(self.skips, "below_min_messages")
+                continue
+            if r["tool_call_count"] < min_tool_calls:
+                bump(self.skips, "below_min_tool_calls")
+                continue
+            out.append(oversize_row(r, max_messages))
+        out.sort(key=lambda r: r["started_at"] or 0.0, reverse=True)
+        if limit is not None:
+            out = out[:limit]
+        return out
+
+    def load_trace(self, session_id: str) -> Trace:
+        """Load one full trace as a normalized Trace."""
+        path = self._index().get(session_id)
+        if not path:
+            raise KeyError(f"codex session not found: {session_id}")
+        return self._parse_file(path, session_id)
+
+    # -------------------------------------------------------------- helpers
+
+    def stats(self) -> dict[str, Any]:
+        """Counts over the store(s), plus the skip census.
+
+        Runs against a SCRATCH counter (2026-10-24): stats() re-walks the
+        index and re-lists sessions, which bumps the file-level counters
+        again — a census taken after stats() reported every skip twice.
+        """
+        real_skips = self.skips
+        census = new_skips()
+        self.skips = census
+        try:
+            rows = self.list_sessions(
+                min_messages=0, max_messages=10**9, min_tool_calls=0,
+                use_cache=False,
+            )
+        finally:
+            self.skips = real_skips
+        return {
+            "roots": len(self.roots),
+            "sessions": len(rows),
+            "messages": sum(r["message_count"] for r in rows),
+            "tool_calls": sum(r["tool_call_count"] for r in rows),
+            # the scratch counter, not the restored one: reading self.skips
+            # here returned the caller's counter and threw the census away
+            "skips": dict(census),
+        }

traceforge/readers/hermes.py

@@ -0,0 +1,319 @@
+"""Hermes Agent trace reader — reads sessions+messages from state.db.
+
+Verified schema (Hermes state.db, 2026-09-26):
+  sessions(id, model, title, display_name, started_at, ended_at,
+           message_count, tool_call_count, input_tokens, output_tokens,
+           cwd, source, ...)
+  messages(id, session_id, role, content, tool_call_id, tool_calls,
+           tool_name, timestamp, reasoning, reasoning_content, active,
+           compacted, display_order, ...)
+
+display_order is NULL on ~47% of rows -> order by (timestamp, id).
+tool_calls is a JSON array: [{"id","call_id","type":"function",
+  "function":{"name","arguments"}}]. Tool results are role='tool' rows
+with tool_call_id + tool_name, paired against the issuing assistant turn.
+"""
+from __future__ import annotations
+
+import os
+import sqlite3
+from typing import Any, Optional
+
+from traceforge.readers._common import bump, in_workdir, new_skips, oversize_row
+from traceforge.trace import Message, ToolCall, Trace, json_loads_lenient
+
+# Sources that carry a real user-facing agent session. 'oneshot' is
+# traceforge's OWN improver runtime (6,625 sessions in the seed archive,
+# 1,578 of them long enough to look like candidates) and 'tool' is a
+# 2-message probe — both stay out. 'subagent' / 'kanban' / 'local' are
+# real work sessions: the subagent ones are delegate_task children whose
+# transcripts are the densest multi-agent signal in the store (MASS,
+# arXiv 2610.12176 — the claude reader already indexes its sidechains
+# for the same reason), and excluding them here was an inconsistency,
+# not a policy. Measured 2026-10-23: 43 such sessions pass every other
+# filter and were being dropped by the whitelist alone.
+_CANDIDATE_SOURCES = ("desktop", "cli", "tui", "subagent", "local", "kanban")
+
+
+class HermesTraceReader:
+    """Reads real traces from one or more Hermes Agent state.dbs (read-only).
+
+    Multi-db support (2026-10-13): a list of db paths is searched
+    together — the live store plus seed archives like
+    harness-traces/hermes/state.db. Session ids are uuids, unique per
+    store with zero expected cross-store overlap (verified against the
+    seed archive); a hit in the FIRST db wins on collision.
+    """
+
+    def __init__(self, state_db: str | list[str]):
+        if isinstance(state_db, (list, tuple)):
+            self.state_dbs = [os.path.abspath(d) for d in state_db]
+        else:
+            self.state_dbs = [os.path.abspath(state_db)]
+        # rows dropped without a candidate row (self-reference guard,
+        # inactive turns): reported by stats() instead of vanishing.
+        self.skips = new_skips()
+        # Last filter settings list_sessions ran with — the SQL-level drop
+        # census in stats() attributes drops in the same order the WHERE
+        # clause applies, so it needs to know what the clause was.
+        self._last_min_messages = 10
+        self._last_min_tool_calls = 2
+        self._last_require_ended = True
+        self._last_max_input_tokens: Optional[int] = None
+        self._last_exclude_cwd: Optional[str] = None
+
+    @property
+    def state_db(self) -> str:
+        """First db (backwards compat for callers/tests)."""
+        return self.state_dbs[0]
+
+    def _connect(self, db: str) -> sqlite3.Connection:
+        if not os.path.exists(db):
+            raise FileNotFoundError(f"Hermes state.db not found at {db}")
+        return sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+
+    def _existing_dbs(self) -> list[str]:
+        """All configured dbs that exist on disk (seed archives may be
+        absent; the live store always is)."""
+        return [d for d in self.state_dbs if os.path.exists(d)]
+
+    def _iter_dbs(self) -> tuple[str, sqlite3.Connection]:
+        """(db, conn) for every existing store; caller closes each conn."""
+        for db in self._existing_dbs():
+            yield db, self._connect(db)
+
+    # ------------------------------------------------------------ sessions
+
+    def list_sessions(
+        self,
+        *,
+        min_messages: int = 10,
+        max_messages: int = 120,
+        min_tool_calls: int = 2,
+        max_input_tokens: Optional[int] = None,
+        source: Optional[str] = None,
+        require_ended: bool = True,
+        exclude_cwd: Optional[str] = None,
+        limit: Optional[int] = None,
+    ) -> list[dict[str, Any]]:
+        """Candidate traces for improvement, newest first (summaries only).
+
+        ``exclude_cwd`` roots out traceforge's own improver children even
+        when mis-tagged: every improver stage runs with cwd inside the
+        improver workdir, so a cwd there means self-referential data no
+        matter what ``source`` says.
+
+        ``max_messages`` is a SOFT cap (2026-10-23): a session above it is
+        still listed, flagged ``oversize``. The hard cap it replaced
+        discarded 563 real hermes sessions (median 187 messages, 99 tool
+        calls) before they could reach ``split_long`` — the stage that
+        actually bounds what the pipeline processes, by rendered size
+        rather than by message count. ``min_messages`` / ``min_tool_calls``
+        stay hard: they filter degenerate stubs, not long work.
+        """
+        q = (
+                "SELECT s.id, s.model, s.title, s.display_name, s.started_at,"
+                " s.ended_at, s.message_count, s.tool_call_count,"
+                " s.input_tokens, s.output_tokens, s.cwd, s.source"
+                " FROM sessions s"
+                " WHERE s.message_count >= ? AND s.tool_call_count >= ?"
+                " AND s.source IN (" + ",".join("?" * len(_CANDIDATE_SOURCES)) + ")"
+            )
+        # remembered for stats()'s SQL-level drop census
+        self._last_min_messages = min_messages
+        self._last_min_tool_calls = min_tool_calls
+        self._last_require_ended = require_ended
+        self._last_max_input_tokens = max_input_tokens
+        self._last_exclude_cwd = exclude_cwd
+        params: list[Any] = [min_messages, min_tool_calls, *_CANDIDATE_SOURCES]
+        if exclude_cwd:
+            # Self-reference guard: improver children run with cwd inside
+            # the improver workdir; a mis-tagged source (e.g. inherited
+            # 'desktop') must never let them become candidates.
+            q += " AND (s.cwd IS NULL OR s.cwd NOT LIKE ?)"
+            params.append(exclude_cwd.rstrip("/\\") + "/%")
+        if require_ended:
+            q += " AND s.ended_at IS NOT NULL"
+        if source is not None:
+            q += " AND s.source = ?"
+            params.append(source)
+        if max_input_tokens is not None:
+            q += " AND s.input_tokens <= ?"
+            params.append(max_input_tokens)
+        q += " ORDER BY s.started_at DESC"
+        # (LIMIT applied per-db below, after the merge)
+        rows: list[dict[str, Any]] = []
+        seen: set[str] = set()
+        for _db, conn in self._iter_dbs():
+            try:
+                got = conn.execute(q, params).fetchall()
+            finally:
+                conn.close()
+            for r in got:
+                if r[0] in seen:
+                    continue
+                seen.add(r[0])
+                if exclude_cwd and in_workdir(r[10], exclude_cwd):
+                    bump(self.skips, "cwd_self_reference")
+                    continue
+                rows.append(
+                    oversize_row(
+                        {
+                            "session_id": r[0],
+                            "model": r[1],
+                            "title": (r[2] or r[3] or "").strip(),
+                            "started_at": r[4],
+                            "ended_at": r[5],
+                            "message_count": r[6],
+                            "tool_call_count": r[7],
+                            "input_tokens": r[8],
+                            "output_tokens": r[9],
+                            "cwd": r[10],
+                            "source": r[11],
+                        },
+                        max_messages,
+                    )
+                )
+        rows.sort(key=lambda r: r["started_at"] or 0.0, reverse=True)
+        if limit is not None:
+            rows = rows[:limit]
+        return rows
+
+    # ------------------------------------------------------------- messages
+
+    MSG_QUERY = (
+        "SELECT id, role, content, tool_call_id, tool_calls, tool_name,"
+        " timestamp, reasoning, reasoning_content, active, compacted"
+        " FROM messages WHERE session_id = ?"
+        " ORDER BY timestamp ASC, id ASC"
+    )
+
+    def load_trace(self, session_id: str) -> Trace:
+        """Load one full trace as a normalized Trace (searches all dbs)."""
+        for _db, conn in self._iter_dbs():
+            try:
+                sess = conn.execute(
+                    "SELECT id, model, title, display_name, started_at, cwd,"
+                    " source FROM sessions WHERE id = ?",
+                    (session_id,),
+                ).fetchone()
+                if not sess:
+                    continue
+                rows = conn.execute(self.MSG_QUERY, (session_id,)).fetchall()
+                return Trace(
+                    session_id=sess[0],
+                    model=sess[1],
+                    title=(sess[2] or sess[3] or "").strip() or None,
+                    started_at=sess[4],
+                    cwd=sess[5],
+                    messages=list(self._parse_rows(rows)),
+                    source="hermes" if not sess[6] else str(sess[6]),
+                )
+            finally:
+                conn.close()
+        raise KeyError(f"session not found: {session_id}")
+
+    def _parse_rows(self, rows: list[sqlite3.Row]) -> list[Message]:
+        out: list[Message] = []
+        for (mid, role, content, tc_id, tc_json, tool_name, ts,
+             reasoning, reasoning_content, active, compacted) in rows:
+            if not active:
+                bump(self.skips, "inactive_turn")
+                continue  # rewound/deleted turns
+            m = Message(role=role, content=content, timestamp=ts)
+            if role == "tool":
+                m.tool_call_id = tc_id
+                m.tool_name = tool_name
+            elif role == "assistant" and tc_json:
+                calls = json_loads_lenient(tc_json)
+                if isinstance(calls, list):
+                    for c in calls:
+                        fn = c.get("function") or {}
+                        m.tool_calls.append(
+                            ToolCall(
+                                id=str(c.get("call_id") or c.get("id") or ""),
+                                name=str(fn.get("name") or ""),
+                                arguments=fn.get("arguments") or "{}",
+                            )
+                        )
+            reasoning = reasoning or reasoning_content
+            if reasoning:
+                m.reasoning = reasoning
+            out.append(m)
+        return out
+
+    # -------------------------------------------------------------- helpers
+
+    def _sql_drop_census(self) -> dict[str, int]:
+        """Attribute every dropped session to the filter that took it.
+
+        ``list_sessions`` applies min_messages / min_tool_calls / the source
+        whitelist / require_ended / max_input_tokens / the cwd guard inside
+        ONE SQL query, so none of those drops can bump a counter — measured
+        2026-10-24: 6,741 of 9,110 sessions vanished with ``skips == {}``.
+        A skip census that reports nothing for the store that drops the most
+        is worse than no census, so the attribution runs here, in the same
+        order the WHERE clause applies.
+
+        Read-only and cheap (one row scan per db, ~9k rows).
+        """
+        census: dict[str, int] = {}
+        seen: set[str] = set()
+
+        def bump_census(reason: str) -> None:
+            census[reason] = census.get(reason, 0) + 1
+
+        for _db, conn in self._iter_dbs():
+            try:
+                rows = conn.execute(
+                    "SELECT id, message_count, tool_call_count, source,"
+                    " ended_at, cwd, input_tokens FROM sessions"
+                ).fetchall()
+            finally:
+                conn.close()
+            for (sid, mc, tc, src, ended, cwd, itok) in rows:
+                if sid in seen:
+                    bump_census("cross_db_duplicate")
+                    continue
+                seen.add(sid)
+                if src not in _CANDIDATE_SOURCES:
+                    bump_census(f"source_not_candidate:{src}")
+                elif (mc or 0) < self._last_min_messages:
+                    bump_census("below_min_messages")
+                elif (tc or 0) < self._last_min_tool_calls:
+                    bump_census("below_min_tool_calls")
+                elif self._last_require_ended and ended is None:
+                    bump_census("not_ended")
+                elif (self._last_max_input_tokens is not None
+                      and (itok or 0) > self._last_max_input_tokens):
+                    bump_census("over_max_input_tokens")
+                elif self._last_exclude_cwd and cwd and in_workdir(
+                    cwd, self._last_exclude_cwd
+                ):
+                    bump_census("cwd_self_reference")
+        return census
+
+    def stats(self) -> dict[str, Any]:
+        """Quick counts over the store(s) for sanity checks.
+
+        ``skips`` merges the row-level counters with the SQL-level drop
+        census, so the store's whole funnel is visible in one place.
+        """
+        out: dict[str, Any] = {"dbs": len(self._existing_dbs())}
+        for key in ("sessions", "messages", "tool_rows"):
+            out[key] = 0
+        for _db, conn in self._iter_dbs():
+            try:
+                one = lambda sql: conn.execute(sql).fetchone()[0]
+                out["sessions"] += one("SELECT COUNT(*) FROM sessions")
+                out["messages"] += one("SELECT COUNT(*) FROM messages")
+                out["tool_rows"] += one(
+                    "SELECT COUNT(*) FROM messages WHERE role='tool'"
+                )
+            finally:
+                conn.close()
+        skips = dict(self.skips)
+        for reason, n in self._sql_drop_census().items():
+            skips[reason] = skips.get(reason, 0) + n
+        out["skips"] = skips
+        return out

traceforge/readers/nemotron.py

@@ -0,0 +1,491 @@
+"""Nemotron SFT dataset reader — NVIDIA's published agentic SFT corpora.
+
+These are not live agent stores. Each row is an already-finished
+``(messages, tools)`` pair from a released SFT dataset, so the reader's
+job is narrower than the harness readers': normalize the row into a
+:class:`~traceforge.trace.Trace` and serve it by session id.
+
+Why they belong in the pool at all: the live stores are ~95% coding and
+English, and the corpus's weak buckets (research, data-analysis, ops,
+office-docs) cannot be filled by re-reading the same harness. The
+Nemotron agentic subsets carry tool-call trajectories from other
+harnesses and other domains (SEC filings, customer support, weather),
+which is exactly the coverage the classifier reports as missing.
+
+Three normalizations the raw rows need:
+
+1. ``messages`` is a JSON *string* in some subsets (SWE-v3.5) and a list
+   in others — both are accepted.
+2. SWE-v3.5 tool rows carry only ``content``: no ``tool_call_id`` and no
+   ``name``. They are paired positionally against the issuing assistant
+   turn's calls (verified 1:1 in order on the released shards).
+3. ``tools`` is a separate column with no counterpart in the Trace
+   model, and the export has no field for it either. Dropping it would
+   leave tool calls pointing at tools nothing defines, so the schemas are
+   folded into the system row as a compact block (name, short
+   description, parameter names) — capped, because SWE-v3.5 ships 33 KB
+   of JSON schema per row and the OpenCode system prompt already
+   describes the tools in prose.
+
+Provenance is carried twice: ``source`` (``nemotron-swe`` …) for the
+benchmark's ``by_source`` grouping, and the session-id namespace
+(``nemotron-swe-<uuid>``) because the improve stage overwrites ``source``
+with ``traceforge-splice`` while the id survives verbatim.
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+from typing import Any, Optional
+
+from traceforge.readers._common import bump, new_skips, oversize_row
+from traceforge.trace import VALID_ROLES, Message, ToolCall, Trace
+
+# Subset -> provenance tag. The tag is the reader's contract with the
+# benchmark: `by_source` groups on it, so it must stay stable.
+SUBSET_SOURCES = {
+    "swe": "nemotron-swe",
+    "agentic-tc": "nemotron-agentic-tc",
+    "agentic-ka": "nemotron-agentic-ka",
+}
+
+# Cap on the folded tool-schema block. SWE-v3.5 ships 33 KB of JSON schema
+# per row; the OpenCode system prompt already names and describes every
+# tool, so the block only has to make the calls resolvable.
+TOOLS_BLOCK_MAX_CHARS = 4000
+
+
+# --------------------------------------------------------------- row parsing
+
+def as_list(value: Any) -> Optional[list]:
+    """A JSON list, whether the column stores a list or a JSON string."""
+    if isinstance(value, list):
+        return value
+    if isinstance(value, str):
+        try:
+            got = json.loads(value)
+        except json.JSONDecodeError:
+            return None
+        return got if isinstance(got, list) else None
+    return None
+
+
+def content_text(value: Any) -> str:
+    """Message content as text.
+
+    OpenAI-style content may be a string or a list of parts
+    (``{"type": "text", "text": …}``); the datasets use strings, but a
+    part list must not silently become ``str(list)``.
+    """
+    if value is None:
+        return ""
+    if isinstance(value, str):
+        return value
+    if isinstance(value, list):
+        out = []
+        for part in value:
+            if isinstance(part, str):
+                out.append(part)
+            elif isinstance(part, dict):
+                out.append(str(part.get("text") or part.get("content") or ""))
+        return "\n".join(p for p in out if p)
+    return str(value)
+
+
+def _arguments_text(value: Any) -> str:
+    """Tool-call arguments as the JSON string the Trace model stores."""
+    if value is None:
+        return "{}"
+    if isinstance(value, str):
+        return value
+    try:
+        return json.dumps(value, ensure_ascii=False)
+    except (TypeError, ValueError):
+        return "{}"
+
+
+def tools_block(tools: Optional[list], max_chars: int = TOOLS_BLOCK_MAX_CHARS) -> str:
+    """Compact ``## Available tools`` block for the system row.
+
+    Name + first line of the description + parameter names. Full JSON
+    schemas are deliberately not reproduced: they are 33 KB on SWE-v3.5
+    and the arguments in the trace are what the trainee has to match.
+    """
+    if not tools:
+        return ""
+    lines = ["## Available tools", ""]
+    for t in tools:
+        if not isinstance(t, dict):
+            continue
+        fn = t.get("function") if isinstance(t.get("function"), dict) else t
+        if not isinstance(fn, dict):
+            continue
+        name = str(fn.get("name") or "").strip()
+        if not name:
+            continue
+        desc = str(fn.get("description") or "").strip().split("\n")[0][:160]
+        params = fn.get("parameters") or {}
+        props = params.get("properties") if isinstance(params, dict) else None
+        arg_names = ", ".join(props.keys()) if isinstance(props, dict) else ""
+        required = params.get("required") if isinstance(params, dict) else None
+        req = ", ".join(str(r) for r in required) if isinstance(required, list) else ""
+        line = f"- `{name}`"
+        if desc:
+            line += f": {desc}"
+        if arg_names:
+            line += f" (args: {arg_names}"
+            line += f"; required: {req})" if req else ")"
+        lines.append(line)
+        if sum(len(x) for x in lines) > max_chars:
+            lines.append("- … (further tool schemas omitted)")
+            break
+    return "\n".join(lines)
+
+
+def _normalize_calls(raw_calls: list, skips, counter: list[int]) -> list[ToolCall]:
+    """Assistant tool calls -> ToolCall list, synthesizing missing ids.
+
+    ``counter`` is a one-element list shared across the whole row: SWE-v3.5
+    ships no call ids at all, and a per-message ``call_0`` would collide
+    across turns (the export pairs results by id).
+    """
+    out: list[ToolCall] = []
+    for tc in raw_calls:
+        if not isinstance(tc, dict):
+            continue
+        fn = tc.get("function") if isinstance(tc.get("function"), dict) else tc
+        if not isinstance(fn, dict):
+            continue
+        name = str(fn.get("name") or "").strip()
+        if not name:
+            bump(skips, "tool_call_without_name")
+            continue
+        call_id = str(tc.get("id") or tc.get("call_id") or "").strip()
+        if not call_id:
+            call_id = f"call_{counter[0]}"
+            counter[0] += 1
+        out.append(
+            ToolCall(
+                id=call_id,
+                name=name,
+                arguments=_arguments_text(fn.get("arguments")),
+            )
+        )
+    return out
+
+
+def _title_from(text: str) -> Optional[str]:
+    """First substantive line of the task, for the listing.
+
+    SWE-v3.5 opens with an ``<uploaded_files>`` envelope whose next line is
+    a bare ``/workspace/repo`` path, so neither the literal first line nor
+    the first non-tag line is a title. Skip markup, bare paths and short
+    fragments; fall back to the first non-empty line.
+    """
+    fallback: Optional[str] = None
+    for line in (text or "").splitlines():
+        s = line.strip().strip('"').strip()
+        if not s:
+            continue
+        if fallback is None:
+            fallback = s[:80]
+        if s.startswith("<") or s.startswith("```"):
+            continue
+        if " " not in s and ("/" in s or "\\" in s):
+            continue  # a bare path, not prose
+        if len(s) < 20 or len(s.split()) < 3:
+            continue
+        return s[:80]
+    return fallback
+
+
+def normalize_row(
+    row: dict,
+    subset: str,
+    *,
+    skips=None,
+    embed_tools: bool = True,
+) -> Optional[Trace]:
+    """One dataset row -> Trace, or None when the row cannot be a trace.
+
+    Returns None (and bumps a skip counter) for a row with no messages, no
+    user turn, or no assistant turn — those are not improvable traces and
+    must not reach the raw pool.
+    """
+    source = SUBSET_SOURCES.get(subset, f"nemotron-{subset}")
+    raw_msgs = as_list(row.get("messages"))
+    if not raw_msgs:
+        bump(skips, "no_messages")
+        return None
+
+    messages: list[Message] = []
+    pending: list[ToolCall] = []  # calls awaiting their result row
+    counter = [0]  # synthesized call-id counter, shared across the row
+    for raw in raw_msgs:
+        if not isinstance(raw, dict):
+            bump(skips, "non_dict_message")
+            continue
+        role = str(raw.get("role") or "").strip()
+        if role not in VALID_ROLES:
+            bump(skips, f"unknown_role:{role or '?'}")
+            continue
+        text = content_text(raw.get("content"))
+        if role == "tool":
+            if not pending:
+                # Orphan result: no issuing call in this row. Keeping it
+                # would fail the pairing gate on every attempt.
+                bump(skips, "orphan_tool_result")
+                continue
+            call = pending.pop(0)
+            messages.append(
+                Message(
+                    role="tool",
+                    content=text,
+                    tool_call_id=str(raw.get("tool_call_id") or call.id),
+                    tool_name=str(raw.get("name") or call.name),
+                )
+            )
+            continue
+        if role == "assistant":
+            calls = _normalize_calls(raw.get("tool_calls") or [], skips, counter)
+            reasoning = raw.get("reasoning_content") or raw.get("reasoning")
+            if not text and not calls:
+                bump(skips, "empty_assistant_row")
+                continue
+            pending.extend(calls)
+            messages.append(
+                Message(
+                    role="assistant",
+                    content=text or None,
+                    reasoning=str(reasoning) if reasoning else None,
+                    tool_calls=calls,
+                )
+            )
+            continue
+        if role == "system" and not text.strip():
+            bump(skips, "empty_system_row")
+            continue
+        messages.append(Message(role=role, content=text))
+
+    if pending:
+        # Calls with no result: the row was cut mid-run. Drop the calls so
+        # the trace does not carry an unanswered invocation.
+        bump(skips, "unanswered_tool_call", len(pending))
+        dead = {c.id for c in pending}
+        for m in messages:
+            if m.role == "assistant" and m.tool_calls:
+                m.tool_calls = [c for c in m.tool_calls if c.id not in dead]
+
+    roles = {m.role for m in messages}
+    if "user" not in roles:
+        bump(skips, "no_user_message")
+        return None
+    if "assistant" not in roles:
+        bump(skips, "no_assistant_message")
+        return None
+
+    if embed_tools:
+        block = tools_block(as_list(row.get("tools")))
+        if block:
+            if messages and messages[0].role == "system":
+                messages[0].content = (messages[0].content or "").rstrip() + "\n\n" + block
+            else:
+                messages.insert(0, Message(role="system", content=block))
+
+    uuid = str(row.get("uuid") or "").strip()
+    if not uuid:
+        uuid = hashlib.sha1(
+            json.dumps(row, sort_keys=True, ensure_ascii=False).encode("utf-8")
+        ).hexdigest()[:16]
+    first_user = next((m.content for m in messages if m.role == "user"), "") or ""
+    return Trace(
+        session_id=f"{source}-{uuid}",
+        title=_title_from(first_user),
+        messages=messages,
+        source=source,
+    )
+
+
+# ------------------------------------------------------------------- reader
+
+class NemotronTraceReader:
+    """Reads normalized Nemotron SFT traces from one or more JSONL roots.
+
+    Each root holds one ``<subset>.jsonl`` per dataset subset, one Trace
+    dict per line (written by ``scripts/fetch_nemotron.py``). A byte-offset
+    index per file is cached next to the data, so ``load_trace`` seeks
+    instead of re-parsing a multi-hundred-MB file.
+    """
+
+    def __init__(self, roots: Optional[list[str]] = None):
+        if isinstance(roots, str):
+            roots = [roots]
+        self.roots = [os.path.abspath(os.path.expanduser(r)) for r in (roots or [])]
+        self.skips = new_skips()
+
+    # ------------------------------------------------------------- indexing
+
+    def _files(self) -> list[str]:
+        out: list[str] = []
+        for root in self.roots:
+            if not os.path.isdir(root):
+                continue
+            for name in sorted(os.listdir(root)):
+                if name.endswith(".jsonl"):
+                    out.append(os.path.join(root, name))
+        return out
+
+    def _index_path(self, path: str) -> str:
+        root = os.path.dirname(path)
+        return os.path.join(root, ".index", os.path.basename(path) + ".idx.json")
+
+    def _build_index(self, path: str) -> dict[str, Any]:
+        """Byte offsets + summary per row, cached on (size, mtime)."""
+        try:
+            st = os.stat(path)
+        except OSError:
+            bump(self.skips, "unreadable_file")
+            return {"sig": "", "rows": []}
+        sig = f"{st.st_size:x}-{int(st.st_mtime):x}"
+        cache_path = self._index_path(path)
+        try:
+            with open(cache_path, "r", encoding="utf-8") as f:
+                cached = json.load(f)
+            if cached.get("sig") == sig:
+                return cached
+        except (OSError, json.JSONDecodeError):
+            pass
+
+        rows: list[dict[str, Any]] = []
+        with open(path, "rb") as f:
+            offset = 0
+            for raw in f:
+                length = len(raw)
+                if raw.strip():
+                    try:
+                        d = json.loads(raw.decode("utf-8"))
+                    except (json.JSONDecodeError, UnicodeDecodeError):
+                        bump(self.skips, "unparseable_row")
+                        offset += length
+                        continue
+                    msgs = d.get("messages") or []
+                    rows.append(
+                        {
+                            "sid": d.get("session_id"),
+                            "offset": offset,
+                            "length": length,
+                            "n_messages": len(msgs),
+                            "n_tool_calls": sum(
+                                len(m.get("tool_calls") or []) for m in msgs
+                            ),
+                            "chars": sum(len(m.get("content") or "") for m in msgs),
+                            "title": d.get("title"),
+                            "source": d.get("source"),
+                            "started_at": d.get("started_at"),
+                        }
+                    )
+                offset += length
+        index = {"sig": sig, "rows": rows}
+        try:
+            os.makedirs(os.path.dirname(cache_path), exist_ok=True)
+            tmp = cache_path + ".tmp"
+            with open(tmp, "w", encoding="utf-8") as f:
+                json.dump(index, f)
+            os.replace(tmp, cache_path)
+        except OSError:
+            pass  # a read-only seed dir still works, just slower
+        return index
+
+    def _index(self) -> dict[str, dict[str, Any]]:
+        """session id -> index entry (with its file path)."""
+        out: dict[str, dict[str, Any]] = {}
+        for path in self._files():
+            for row in self._build_index(path).get("rows", []):
+                sid = row.get("sid")
+                if not sid:
+                    bump(self.skips, "row_without_session_id")
+                    continue
+                entry = dict(row)
+                entry["path"] = path
+                out[sid] = entry
+        return out
+
+    # ------------------------------------------------------------ sessions
+
+    def list_sessions(
+        self,
+        *,
+        min_messages: int = 10,
+        max_messages: int = 120,
+        min_tool_calls: int = 2,
+        limit: Optional[int] = None,
+        status_cb=None,
+        **_ignored,
+    ) -> list[dict[str, Any]]:
+        """Candidate traces, newest first (summaries only).
+
+        ``max_messages`` is the same SOFT cap the other readers use: an
+        over-cap row is listed with ``oversize`` and bounded later by
+        ``split_long``. ``min_messages`` / ``min_tool_calls`` stay hard.
+        """
+        if status_cb:
+            status_cb(f"indexing {len(self._files())} nemotron file(s)")
+        rows: list[dict[str, Any]] = []
+        for sid, e in self._index().items():
+            if e["n_messages"] < min_messages:
+                bump(self.skips, "below_min_messages")
+                continue
+            if e["n_tool_calls"] < min_tool_calls:
+                bump(self.skips, "below_min_tool_calls")
+                continue
+            rows.append(
+                oversize_row(
+                    {
+                        "session_id": sid,
+                        "model": None,
+                        "title": e.get("title") or "",
+                        "started_at": e.get("started_at"),
+                        "ended_at": None,
+                        "message_count": e["n_messages"],
+                        "tool_call_count": e["n_tool_calls"],
+                        "input_tokens": None,
+                        "output_tokens": None,
+                        "cwd": None,
+                        "source": e.get("source") or "nemotron",
+                    },
+                    max_messages,
+                )
+            )
+        rows.sort(key=lambda r: (r["started_at"] or 0.0, r["session_id"]), reverse=True)
+        if limit is not None:
+            rows = rows[:limit]
+        return rows
+
+    # ------------------------------------------------------------- loading
+
+    def load_trace(self, session_id: str) -> Trace:
+        entry = self._index().get(session_id)
+        if entry is None:
+            raise KeyError(f"session not found: {session_id}")
+        with open(entry["path"], "rb") as f:
+            f.seek(entry["offset"])
+            raw = f.read(entry["length"])
+        return Trace.from_dict(json.loads(raw.decode("utf-8")))
+
+    # --------------------------------------------------------------- stats
+
+    def stats(self) -> dict[str, Any]:
+        files = self._files()
+        idx = self._index()
+        return {
+            "files": len(files),
+            "traces": len(idx),
+            "messages": sum(e["n_messages"] for e in idx.values()),
+            "tool_calls": sum(e["n_tool_calls"] for e in idx.values()),
+            "by_source": {
+                src: sum(1 for e in idx.values() if e.get("source") == src)
+                for src in sorted({e.get("source") or "?" for e in idx.values()})
+            },
+            "skips": dict(self.skips),
+        }

traceforge/space/__init__.py

@@ -0,0 +1,23 @@
+"""Portable traceforge: content-addressed seeds, a queue, and a Gradio UI.
+
+The package is import-safe without Gradio — `app.py` imports it lazily, so
+the CLI and the tests never pay for the UI dependency.
+
+    from traceforge.space import TraceStore, SpacePipeline, seed_hash
+"""
+from traceforge.space.hashing import (canonical_messages, fingerprint,
+                                      output_hash, seed_hash, short_hash,
+                                      tool_names)
+from traceforge.space.ingest import parse_records, read_upload, trace_from_record
+from traceforge.space.pipeline import (SpacePipeline, pipeline_ready,
+                                       space_config)
+from traceforge.space.store import (DONE, FAILED, QUEUED, REJECTED, RUNNING,
+                                    TraceStore)
+
+__all__ = [
+    "TraceStore", "SpacePipeline", "space_config", "pipeline_ready",
+    "seed_hash", "output_hash", "short_hash", "fingerprint",
+    "canonical_messages", "tool_names", "trace_from_record",
+    "parse_records", "read_upload",
+    "QUEUED", "RUNNING", "DONE", "FAILED", "REJECTED",
+]

traceforge/space/app.py

@@ -0,0 +1,465 @@
+"""Gradio app: submit a trace, sponsor a dataset, read the dashboard.
+
+The UI is deliberately thin — every decision lives in `SpacePipeline` and
+`TraceStore`, so the same code runs locally (`python app.py`) and on a
+Hugging Face Space (`app.py` at the repo root).
+
+Tabs
+  Submit     one trace (JSON / JSONL / pasted) -> hash -> queue -> output
+  Sponsor    a JSONL of traces -> hashed, deduped, queued, attributed
+  Dashboard  public stats: corpus, time sensitivity, tasks, sponsors, queue
+  Browse     the stored outputs, by hash
+"""
+from __future__ import annotations
+
+import html
+import json
+import os
+from typing import Any, Optional
+
+from traceforge.space.hashing import short_hash
+from traceforge.space.ingest import parse_records, read_upload
+from traceforge.space.jobs import get_worker
+from traceforge.space.pipeline import SpacePipeline, pipeline_ready, space_config
+from traceforge.space.stats import bar_rows, dashboard, pct
+from traceforge.space.store import DONE, FAILED, QUEUED, RUNNING, TraceStore
+
+REFRESH_S = 8.0
+
+CSS = """
+.tf-wrap { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
+.tf-kpis { display:grid; grid-template-columns:repeat(auto-fit,minmax(130px,1fr));
+           gap:8px; margin:6px 0 14px; }
+.tf-kpi { background:#161b22; border:1px solid #2a3140; border-radius:8px;
+          padding:9px 11px; }
+.tf-kpi .v { font-size:21px; font-weight:600; color:#e6edf3; line-height:1.2; }
+.tf-kpi .k { font-size:11px; color:#8b949e; text-transform:uppercase;
+             letter-spacing:.04em; }
+.tf-kpi .s { font-size:11px; color:#6e7681; }
+.tf-ok { color:#3fb950; } .tf-warn { color:#d29922; } .tf-bad { color:#f85149; }
+.tf-bar-row { display:grid; grid-template-columns:190px 1fr 64px; gap:8px;
+              align-items:center; font-size:12px; color:#9aa4b2; margin:3px 0; }
+.tf-bar-outer { background:#1b2029; border-radius:6px; height:14px;
+                overflow:hidden; border:1px solid #2a3140; }
+.tf-bar-inner { height:100%; background:linear-gradient(90deg,#2f81f7,#3fb950); }
+.tf-head { font-size:13px; color:#8b949e; margin:2px 0 8px; }
+.tf-pill { padding:2px 8px; border-radius:999px; font-size:11px;
+           border:1px solid #2a3140; }
+"""
+
+
+# --------------------------------------------------------------------- html
+
+def _esc(s: Any) -> str:
+    return html.escape(str(s if s is not None else ""))
+
+
+def _kpi(value: Any, key: str, sub: str = "", cls: str = "") -> str:
+    sub_html = f'<div class="s">{_esc(sub)}</div>' if sub else ""
+    return (f'<div class="tf-kpi"><div class="v {cls}">{_esc(value)}</div>'
+            f'<div class="k">{_esc(key)}</div>{sub_html}</div>')
+
+
+def _bars(counts: dict, total: Optional[int] = None) -> str:
+    rows = bar_rows(counts, total=total)
+    if not rows:
+        return '<div class="tf-head">no data yet</div>'
+    out = ['<div class="tf-wrap">']
+    for label, n, p in rows:
+        out.append(
+            '<div class="tf-bar-row">'
+            f'<span>{_esc(label)}</span>'
+            f'<span class="tf-bar-outer"><span class="tf-bar-inner" '
+            f'style="width:{max(0.0, min(100.0, p)):.1f}%"></span></span>'
+            f'<span>{n} · {p:.0f}%</span></div>'
+        )
+    out.append("</div>")
+    return "".join(out)
+
+
+def _status_pill(status: str) -> str:
+    cls = {DONE: "tf-ok", FAILED: "tf-bad", RUNNING: "tf-warn",
+           QUEUED: "tf-warn"}.get(status, "")
+    return f'<span class="tf-pill {cls}">{_esc(status)}</span>'
+
+
+def _kpi_html(space: dict, ref: dict, worker: dict, ready: tuple[bool, str]) -> str:
+    ok, reason = ready
+    cards = [
+        _kpi(f"{space['seeds']:,}", "seeds", "distinct content hashes"),
+        _kpi(f"{space['done']:,}", "outputs",
+             f"{pct(space['done'], space['seeds']):.0f}% of seeds"),
+        _kpi(f"{space['queued']:,}", "queued",
+             f"{space['running']} running"),
+        _kpi(f"{space['failed']:,}", "failed", "rejected or errored"),
+        _kpi(f"{len(space['sponsors'])}", "sponsors",
+             f"{sum(s['seeds'] for s in space['sponsors'])} seeds"),
+        _kpi(f"{space['duration_s']['median']:.0f}s", "median runtime",
+             f"p90 {space['duration_s']['p90']:.0f}s"),
+    ]
+    if ref.get("available"):
+        cards.append(_kpi(f"{ref.get('traces', 0):,}", "reference corpus",
+                          "published ideal traces"))
+    head = (
+        '<div class="tf-wrap"><div class="tf-head">'
+        f'<span class="tf-pill {"tf-ok" if ok else "tf-bad"}">'
+        f'{"pipeline ready" if ok else "pipeline not configured"}</span> '
+        f'{_esc(reason)} · worker {"alive" if worker.get("alive") else "stopped"}'
+        f' · queue {worker.get("queued", 0)}'
+        "</div></div>"
+    )
+    return head + '<div class="tf-wrap"><div class="tf-kpis">' + "".join(cards) + "</div></div>"
+
+
+def _ts_html(space: dict, ref: dict) -> str:
+    """Time sensitivity: the metric that says whether an answer decays."""
+    parts = ['<div class="tf-wrap"><div class="tf-head">'
+             "time sensitivity — a temporal fact answered from weights decays; "
+             "the same fact fetched with a tool does not</div>"]
+    space_ts = space.get("time_sensitivity") or {}
+    if space_ts:
+        parts.append("<b>this Space</b>")
+        parts.append(_bars(space_ts))
+    census = (ref or {}).get("time_sensitivity") or {}
+    if census.get("available"):
+        parts.append(f"<b>reference corpus</b> ({census.get('total', 0):,} traces)")
+        parts.append(_bars(census.get("classes") or {}))
+        parts.append(
+            f'<div class="tf-head">needs rewrite (tool available but never '
+            f'called): {census.get("needs_rewrite", 0):,}</div>')
+    if not space_ts and not census.get("available"):
+        parts.append('<div class="tf-head">no data yet</div>')
+    parts.append("</div>")
+    return "".join(parts)
+
+
+# ------------------------------------------------------------------- tables
+
+def _rows(records: list[dict], cols: list[str]) -> list[list]:
+    out = []
+    for r in records:
+        row = []
+        for c in cols:
+            v = r.get(c)
+            if c == "hash":
+                v = short_hash(v or "")
+            elif c == "time_sensitivity":
+                v = (v or {}).get("class", "")
+            elif isinstance(v, float):
+                v = round(v, 1)
+            row.append(v if v is not None else "")
+        out.append(row)
+    return out
+
+
+# ---------------------------------------------------------------------- app
+
+def build_app(store: Optional[TraceStore] = None,
+              pipeline: Optional[SpacePipeline] = None):
+    import gradio as gr
+
+    cfg = pipeline.cfg if pipeline else space_config()
+    store = store or (pipeline.store if pipeline else TraceStore(
+        os.path.join(cfg.data_dir)))
+    pipeline = pipeline or SpacePipeline(store, cfg)
+    worker = get_worker(pipeline)
+    ready = pipeline_ready(cfg)
+    dataset_dir = os.path.join(os.path.dirname(cfg.data_dir), "dataset")
+    cache_path = os.path.join(store.root, "reference_census.json")
+
+    # ------------------------------------------------------------- submit
+
+    def on_submit(file_path, pasted, sponsor, sync):
+        if not ready:
+            return ("**pipeline not configured** — this Space has no LLM endpoint, "
+                    "so nothing was queued. Set `TRACEFORGE_BASE_URL` and "
+                    "`TRACEFORGE_LLM_KEY` in the Space settings and restart.", None, None)
+        if file_path:
+            records, errors = read_upload(file_path)
+        else:
+            records, errors = parse_records(pasted or "")
+        if errors:
+            return (f"**parse errors**\n\n" + "\n".join(f"- {e}" for e in errors[:10]),
+                    None, None)
+        if not records:
+            return "**nothing to submit** — upload a file or paste JSON", None, None
+        if len(records) > 1:
+            return (f"**{len(records)} records** — the Submit tab takes one trace. "
+                    "Use **Sponsor a dataset** for a batch.", None, None)
+        try:
+            res = pipeline.submit(records[0], sponsor=sponsor)
+        except Exception as e:
+            return f"**submit failed**: {e}", None, None
+        h = res["hash"]
+        if res.get("duplicate") and res.get("status") == DONE:
+            path = store.output_path(h)
+            return (f"**already processed** — this seed hash already has an output.\n\n"
+                    f"`{h}`\n\nNo second output was created.", h,
+                    path if os.path.exists(path) else None)
+        if sync:
+            res = pipeline.process(h)
+        rec = pipeline.result(h)
+        md = _result_md(rec)
+        path = store.output_path(h) if rec.get("status") == DONE else None
+        return md, h, path if path and os.path.exists(path) else None
+
+    def on_check(h):
+        if not h:
+            return "no hash yet — submit a trace first", None
+        rec = pipeline.result(h)
+        path = store.output_path(h) if rec.get("status") == DONE else None
+        return _result_md(rec), path if path and os.path.exists(path) else None
+
+    def _result_md(rec: dict) -> str:
+        h = rec.get("hash") or ""
+        st = rec.get("status") or "unknown"
+        lines = [f"### {_status_pill(st)} `{h}`", ""]
+        if rec.get("duplicate"):
+            lines.append("_duplicate submission — the existing record was returned_")
+            lines.append("")
+        if rec.get("task"):
+            lines.append(f"- task: **{rec['task']}** · language: **{rec.get('language', '?')}**")
+        if rec.get("n_messages"):
+            lines.append(f"- seed: {rec['n_messages']} messages · "
+                         f"{rec.get('n_tool_calls', 0)} tool calls")
+        if rec.get("n_messages_out"):
+            lines.append(f"- output: {rec['n_messages_out']} messages")
+        if rec.get("route"):
+            lines.append(f"- route: `{rec['route']}` · attempts: {rec.get('attempts', 0)}")
+        if rec.get("duration_s"):
+            lines.append(f"- runtime: {rec['duration_s']:.0f}s")
+        ts = rec.get("time_sensitivity") or {}
+        if ts.get("class"):
+            lines.append(f"- time sensitivity: **{ts['class']}** — {ts.get('label', '')}")
+            if ts.get("temporal_facts"):
+                lines.append(f"  - temporal facts: {', '.join(ts['temporal_facts'])}")
+            if ts.get("retrieval_available"):
+                lines.append(f"  - retrieval available: {', '.join(ts['retrieval_available'])}")
+            if ts.get("needs_rewrite"):
+                lines.append("  - the raw trace answered from weights although a "
+                             "retrieval tool was available")
+        ts2 = rec.get("time_sensitivity_improved") or {}
+        if ts2.get("class") and ts2.get("class") != ts.get("class"):
+            lines.append(f"- after rewrite: **{ts2['class']}**")
+        if rec.get("error"):
+            lines.append(f"- error: `{rec['error'][:600]}`")
+        if st == QUEUED:
+            lines.append("")
+            lines.append("_queued — the worker picks it up in submission order. "
+                         "Press **Check status** to refresh._")
+        return "\n".join(lines)
+
+    # ------------------------------------------------------------ sponsor
+
+    def on_sponsor(file_path, pasted, sponsor):
+        if not ready:
+            return ("**pipeline not configured** — this Space has no LLM endpoint, "
+                    "so nothing was queued. Set `TRACEFORGE_BASE_URL` and "
+                    "`TRACEFORGE_LLM_KEY` in the Space settings and restart.", [], "")
+        if file_path:
+            records, errors = read_upload(file_path)
+        else:
+            records, errors = parse_records(pasted or "")
+        if errors:
+            return (f"**parse errors**\n\n" + "\n".join(f"- {e}" for e in errors[:10]),
+                    [], "")
+        if not records:
+            return "**nothing to sponsor** — upload a JSONL of traces", [], ""
+        results = pipeline.submit_many(records, sponsor=sponsor)
+        new = sum(1 for r in results if not r.get("duplicate") and r.get("hash"))
+        dup = sum(1 for r in results if r.get("duplicate"))
+        err = sum(1 for r in results if not r.get("hash"))
+        rows = []
+        for r in results:
+            rows.append([
+                short_hash(r.get("hash") or ""),
+                r.get("status") or "error",
+                "duplicate" if r.get("duplicate") else ("new" if r.get("hash") else "error"),
+                r.get("n_messages", ""),
+                r.get("task", "")[:60] if r.get("task") else "",
+                (r.get("time_sensitivity") or {}).get("class", ""),
+                (r.get("error") or "")[:120],
+            ])
+        md = (f"**{len(records)} records** — {new} new, {dup} already known, "
+              f"{err} rejected. Sponsor: **{_esc(sponsor or 'anonymous')}**.\n\n"
+              "New seeds are queued; each one produces at most one output.")
+        return md, rows, sponsor or "anonymous"
+
+    # ---------------------------------------------------------- dashboard
+
+    def on_dashboard():
+        d = dashboard(store, dataset_dir=dataset_dir, cache_path=cache_path)
+        space, ref = d["space"], d["reference"]
+        w = worker.status()
+        kpis = _kpi_html(space, ref, w, ready)
+        ts = _ts_html(space, ref)
+        tasks = bar_rows(space.get("tasks") or {})
+        langs = bar_rows(space.get("languages") or {})
+        sponsors = [[s["sponsor"], s["seeds"], s["done"], s["failed"]]
+                    for s in space.get("sponsors") or []]
+        recent = _rows(d["recent"], ["hash", "status", "sponsor", "task",
+                                     "time_sensitivity", "duration_s"])
+        queue = _rows(d["queue"], ["hash", "sponsor", "task", "n_messages"])
+        ref_rows = []
+        if ref.get("available"):
+            card = ref.get("card") or {}
+            for k, v in (card.get("tasks") or {}).items():
+                ref_rows.append([k, v, pct(v, card.get("traces", 0))])
+        return kpis, ts, tasks, langs, sponsors, recent, queue, ref_rows
+
+    # ------------------------------------------------------------- browse
+
+    def browse_choices():
+        done = [r for r in store.all() if r.get("status") == DONE]
+        done.sort(key=lambda r: r.get("updated_at") or 0, reverse=True)
+        return gr.update(choices=[r["hash"] for r in done][:500])
+
+    def on_browse(h):
+        if not h:
+            return None, None
+        out = store.load_output(h)
+        if out is None:
+            return None, None
+        return out.to_dict(), store.output_path(h)
+
+    # ---------------------------------------------------------------- UI
+
+    with gr.Blocks(title="traceforge — ideal-world agent traces") as app:
+        gr.Markdown(
+            "# traceforge\n"
+            "Turn a real agent trace into its **ideal-world reference**: the same "
+            "work, with the dead ends removed, gated for structure and grounding.\n\n"
+            "Every seed is identified by a **content hash** — submitting the same "
+            "trace twice returns the existing output instead of creating a second one."
+        )
+
+        with gr.Tab("Submit a trace"):
+            with gr.Row():
+                with gr.Column(scale=3):
+                    file_in = gr.File(label="trace file (.json / .jsonl)",
+                                      file_types=[".json", ".jsonl"],
+                                      type="filepath")
+                    paste_in = gr.Textbox(
+                        label="…or paste a trace (JSON, JSONL, or a message list)",
+                        lines=10, placeholder='{"messages": [{"role": "user", ...}]}')
+                with gr.Column(scale=2):
+                    sponsor_in = gr.Textbox(label="your name / handle (optional)",
+                                            max_lines=1)
+                    sync_in = gr.Checkbox(
+                        label="process synchronously (debug — blocks until done)",
+                        value=os.environ.get("TRACEFORGE_SPACE_SYNC") == "1")
+                    btn_submit = gr.Button("Submit", variant="primary")
+                    btn_check = gr.Button("Check status")
+                    last_hash = gr.State("")
+            status_md = gr.Markdown("_no submission yet_")
+            out_file = gr.File(label="improved trace (download)", interactive=False)
+
+            btn_submit.click(on_submit,
+                             inputs=[file_in, paste_in, sponsor_in, sync_in],
+                             outputs=[status_md, last_hash, out_file])
+            btn_check.click(on_check, inputs=[last_hash],
+                            outputs=[status_md, out_file])
+
+        with gr.Tab("Sponsor a dataset"):
+            gr.Markdown(
+                "Upload a JSONL of traces. Each row is hashed; **new** seeds are "
+                "queued for the pipeline and attributed to you, **known** seeds are "
+                "skipped. The processed rows join the public output dataset."
+            )
+            with gr.Row():
+                with gr.Column(scale=3):
+                    sp_file = gr.File(label="dataset (.jsonl)", file_types=[".jsonl"],
+                                      type="filepath")
+                    sp_paste = gr.Textbox(label="…or paste JSONL", lines=8)
+                with gr.Column(scale=2):
+                    sp_name = gr.Textbox(label="sponsor name", max_lines=1)
+                    sp_btn = gr.Button("Sponsor", variant="primary")
+            sp_md = gr.Markdown("_no dataset sponsored yet_")
+            sp_table = gr.Dataframe(
+                headers=["hash", "status", "new?", "messages", "task",
+                         "time sensitivity", "error"],
+                interactive=False, wrap=True, max_height=420,
+            )
+            sp_credit = gr.State("")
+            sp_btn.click(on_sponsor, inputs=[sp_file, sp_paste, sp_name],
+                         outputs=[sp_md, sp_table, sp_credit])
+
+        with gr.Tab("Dashboard"):
+            gr.Markdown("Public statistics. Refreshes automatically.")
+            dash_kpis = gr.HTML()
+            dash_ts = gr.HTML()
+            with gr.Row():
+                with gr.Column():
+                    gr.Markdown("### task mix (this Space)")
+                    dash_tasks = gr.Dataframe(
+                        headers=["task", "seeds", "pct"], interactive=False,
+                        wrap=True, max_height=300)
+                with gr.Column():
+                    gr.Markdown("### languages")
+                    dash_langs = gr.Dataframe(
+                        headers=["language", "seeds", "pct"], interactive=False,
+                        wrap=True, max_height=300)
+            with gr.Row():
+                with gr.Column():
+                    gr.Markdown("### sponsors")
+                    dash_sponsors = gr.Dataframe(
+                        headers=["sponsor", "seeds", "done", "failed"],
+                        interactive=False, wrap=True, max_height=300)
+                with gr.Column():
+                    gr.Markdown("### reference corpus — task mix")
+                    dash_ref = gr.Dataframe(
+                        headers=["task", "traces", "pct"], interactive=False,
+                        wrap=True, max_height=300)
+            gr.Markdown("### recent outputs")
+            dash_recent = gr.Dataframe(
+                headers=["hash", "status", "sponsor", "task",
+                         "time sensitivity", "runtime s"],
+                interactive=False, wrap=True, max_height=320)
+            gr.Markdown("### queue")
+            dash_queue = gr.Dataframe(
+                headers=["hash", "sponsor", "task", "messages"],
+                interactive=False, wrap=True, max_height=240)
+
+            dash_outputs = [dash_kpis, dash_ts, dash_tasks, dash_langs,
+                            dash_sponsors, dash_recent, dash_queue, dash_ref]
+            timer = gr.Timer(REFRESH_S)
+            timer.tick(on_dashboard, outputs=dash_outputs)
+            app.load(on_dashboard, outputs=dash_outputs)
+
+        with gr.Tab("Browse outputs"):
+            with gr.Row():
+                pick = gr.Dropdown(label="processed seed (by hash)", choices=[],
+                                   scale=4)
+                btn_reload = gr.Button("Reload list", scale=1)
+            preview = gr.JSON(label="improved trace", max_height=520)
+            dl = gr.File(label="download", interactive=False)
+            btn_reload.click(browse_choices, outputs=[pick])
+            pick.change(on_browse, inputs=[pick], outputs=[preview, dl])
+            app.load(browse_choices, outputs=[pick])
+
+    return app
+
+
+def main(argv: Optional[list[str]] = None) -> int:
+    import argparse
+
+    import gradio as gr
+
+    ap = argparse.ArgumentParser(description="traceforge Gradio app")
+    ap.add_argument("--host", default=os.environ.get("GRADIO_SERVER_NAME",
+                                                     "127.0.0.1"))
+    ap.add_argument("--port", type=int,
+                    default=int(os.environ.get("GRADIO_SERVER_PORT", "7860")))
+    ap.add_argument("--share", action="store_true")
+    args = ap.parse_args(argv)
+
+    app = build_app()
+    app.queue(default_concurrency_limit=4)
+    app.launch(server_name=args.host, server_port=args.port,
+               share=args.share, inbrowser=False, show_error=True,
+               css=CSS, theme=gr.themes.Base())
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

traceforge/space/hashing.py

@@ -0,0 +1,214 @@
+"""Content-addressed identity for a seed trace.
+
+The Space must never turn one seed trace into two outputs. The key is a
+hash over the *conversation content*, not over the harness metadata:
+
+  * session id, model name, cwd, timestamps and tool-call ids are excluded —
+    the same conversation exported twice by two harnesses is one seed;
+  * message text is normalised (line endings, trailing spaces, blank runs)
+    so a re-indent or a CRLF conversion does not mint a new seed;
+  * tool-call arguments are canonicalised as JSON (sorted keys, no spaces)
+    so `{"a": 1, "b": 2}` and `{"b":2,"a":1}` are the same call;
+  * the AVAILABLE tool names are included. A trace where the agent had
+    `web_search` and one where it did not are different seeds: the first is
+    time-anchored, the second is time-sensitive, and they must not collapse
+    into one output.
+
+    from traceforge.space.hashing import seed_hash, fingerprint
+    seed_hash(trace)        # 64-char sha256
+    fingerprint(trace)      # hash + the shape the dashboard reports
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from typing import Any, Optional
+
+# Roles the canonical form keeps. Anything else is harness noise.
+_KEEP_ROLES = ("system", "user", "assistant", "tool")
+
+_BLANK_RUN_RE = re.compile(r"\n{3,}")
+
+
+# ------------------------------------------------------------------ helpers
+
+def _messages(obj: Any) -> list[dict]:
+    """Accept a Trace, an exported record, or a bare message list."""
+    if isinstance(obj, list):
+        return [m for m in obj if isinstance(m, dict)]
+    if hasattr(obj, "messages"):
+        return [m.to_dict() if hasattr(m, "to_dict") else m for m in obj.messages]
+    if isinstance(obj, dict):
+        msgs = obj.get("messages") or []
+        return [m for m in msgs if isinstance(m, dict)]
+    return []
+
+
+def _norm_text(text: Any) -> str:
+    """Whitespace-normalise message text without changing its words."""
+    if text is None:
+        return ""
+    if not isinstance(text, str):
+        # OpenAI content parts: [{"type": "text", "text": "..."}]
+        if isinstance(text, list):
+            parts = []
+            for p in text:
+                if isinstance(p, dict):
+                    parts.append(str(p.get("text") or p.get("content") or ""))
+                else:
+                    parts.append(str(p))
+            text = "\n".join(parts)
+        else:
+            text = str(text)
+    text = text.replace("\r\n", "\n").replace("\r", "\n")
+    text = "\n".join(line.rstrip() for line in text.split("\n"))
+    text = _BLANK_RUN_RE.sub("\n\n", text)
+    return text.strip()
+
+
+def _canon_arguments(raw: Any) -> str:
+    """Canonical JSON for a tool call's arguments.
+
+    Accepts the OpenAI string form, a mapping, or a malformed blob. A blob
+    that does not parse is normalised as text — it is still part of the
+    seed's identity, just not comparable field-by-field.
+    """
+    if isinstance(raw, (dict, list)):
+        return json.dumps(raw, sort_keys=True, separators=(",", ":"),
+                          ensure_ascii=False)
+    s = (raw or "").strip()
+    if not s:
+        return "{}"
+    try:
+        return json.dumps(json.loads(s), sort_keys=True,
+                          separators=(",", ":"), ensure_ascii=False)
+    except (ValueError, TypeError):
+        return _norm_text(s)
+
+
+def _tool_call(tc: Any) -> Optional[dict]:
+    """Normalise one tool call from either schema.
+
+    Flat (traceforge): {id, name, arguments}
+    OpenAI:            {id, type: "function", function: {name, arguments}}
+    """
+    if not isinstance(tc, dict):
+        return None
+    fn_raw = tc.get("function")
+    fn: dict = fn_raw if isinstance(fn_raw, dict) else {}
+    name = (fn.get("name") or tc.get("name") or "").strip()
+    if not name:
+        return None
+    args = fn.get("arguments") if fn else tc.get("arguments")
+    return {"name": name, "arguments": _canon_arguments(args)}
+
+
+def tool_names(obj: Any) -> list[str]:
+    """Names of the tools the harness made available, sorted.
+
+    Only a record with a `tools` array can answer this; a bare Trace cannot
+    (the model was never told which tools existed). An empty list is the
+    honest answer, not a claim that no tools were available.
+    """
+    if not isinstance(obj, dict):
+        return []
+    names: set[str] = set()
+    for t in obj.get("tools") or []:
+        if not isinstance(t, dict):
+            continue
+        fn = t.get("function") if isinstance(t.get("function"), dict) else {}
+        name = (fn.get("name") or t.get("name") or "").strip()
+        if name:
+            names.add(name)
+    return sorted(names)
+
+
+# --------------------------------------------------------------- canonical
+
+def canonical_messages(obj: Any) -> list[dict]:
+    """The conversation as a stable, metadata-free structure."""
+    out: list[dict] = []
+    for m in _messages(obj):
+        role = str(m.get("role") or "").strip().lower()
+        if role not in _KEEP_ROLES:
+            continue
+        rec: dict[str, Any] = {"role": role}
+        content = _norm_text(m.get("content"))
+        if content:
+            rec["content"] = content
+        calls = [c for c in (_tool_call(tc) for tc in (m.get("tool_calls") or []))
+                 if c]
+        if calls:
+            rec["tool_calls"] = calls
+        if role == "tool":
+            name = (m.get("tool_name") or m.get("name") or "").strip()
+            if name:
+                rec["tool_name"] = name
+        # A row with no content and no calls carries nothing.
+        if len(rec) == 1:
+            continue
+        out.append(rec)
+    return out
+
+
+def canonical_payload(obj: Any) -> dict:
+    return {
+        "messages": canonical_messages(obj),
+        "tools": tool_names(obj),
+    }
+
+
+def seed_hash(obj: Any) -> str:
+    """Stable sha256 of a seed trace's content. Same content -> same hash."""
+    payload = json.dumps(canonical_payload(obj), sort_keys=True,
+                         separators=(",", ":"), ensure_ascii=False)
+    return hashlib.sha256(payload.encode("utf-8")).hexdigest()
+
+
+def output_hash(obj: Any) -> str:
+    """Hash of an improved trace, for integrity checks on the stored output."""
+    return seed_hash(obj)
+
+
+def short_hash(h: str, n: int = 12) -> str:
+    return (h or "")[:n]
+
+
+# ------------------------------------------------------------- fingerprint
+
+def fingerprint(obj: Any) -> dict:
+    """Hash plus the shape the dashboard and the index row report."""
+    msgs = _messages(obj)
+    n_calls = 0
+    n_users = 0
+    chars = 0
+    for m in msgs:
+        chars += len(_norm_text(m.get("content")))
+        if m.get("role") == "user":
+            n_users += 1
+        for tc in (m.get("tool_calls") or []):
+            n_calls += 1
+            fn = tc.get("function") if isinstance(tc.get("function"), dict) else {}
+            chars += len(str(fn.get("arguments") or tc.get("arguments") or ""))
+    task = ""
+    for m in msgs:
+        if m.get("role") == "user":
+            task = _norm_text(m.get("content"))
+            break
+    lang = "en"
+    try:
+        from traceforge.classify import detect_language
+        lang = detect_language(task)
+    except Exception:
+        pass
+    return {
+        "hash": seed_hash(obj),
+        "n_messages": len(msgs),
+        "n_tool_calls": n_calls,
+        "n_user_turns": n_users,
+        "chars": chars,
+        "task": task[:400],
+        "language": lang,
+        "tools": tool_names(obj),
+    }

traceforge/space/ingest.py

@@ -0,0 +1,175 @@
+"""Turn whatever a user uploads into a `Trace`.
+
+Three shapes arrive in practice:
+
+  1. traceforge native — `{"session_id", "messages": [...], "source", ...}`
+  2. OpenAI chat JSONL — `{"messages": [...], "tools": [...]}`
+  3. a bare message list — `[{...}, {...}]`
+
+The normaliser accepts all three and both tool-call schemas (flat
+`{id,name,arguments}` and OpenAI `{type:"function",function:{...}}`), so a
+user does not have to convert their export before submitting it.
+
+`parse_records` is the batch entry point: it reads a JSON file, a JSONL file
+or a pasted blob and yields `(record, trace)` pairs. A malformed row is
+reported, never silently dropped.
+"""
+from __future__ import annotations
+
+import json
+import os
+from typing import Any, Iterable, Iterator, Optional
+
+from traceforge.trace import Message, ToolCall, Trace
+
+
+# ------------------------------------------------------------------ helpers
+
+def _content_text(content: Any) -> Optional[str]:
+    if content is None:
+        return None
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts: list[str] = []
+        for p in content:
+            if isinstance(p, dict):
+                if p.get("type") in (None, "text", "input_text", "output_text"):
+                    parts.append(str(p.get("text") or ""))
+                elif p.get("type") == "image_url":
+                    parts.append("[image]")
+            else:
+                parts.append(str(p))
+        return "\n".join(x for x in parts if x)
+    return str(content)
+
+
+def _tool_call(tc: Any, idx: int) -> Optional[ToolCall]:
+    if not isinstance(tc, dict):
+        return None
+    fn_raw = tc.get("function")
+    fn: dict = fn_raw if isinstance(fn_raw, dict) else {}
+    name = (fn.get("name") or tc.get("name") or "").strip()
+    if not name:
+        return None
+    args = fn.get("arguments") if fn else tc.get("arguments")
+    if isinstance(args, (dict, list)):
+        args = json.dumps(args, ensure_ascii=False)
+    return ToolCall(
+        id=str(tc.get("id") or f"call_{idx}"),
+        name=name,
+        arguments=args if isinstance(args, str) and args.strip() else "{}",
+    )
+
+
+def message_from_dict(d: dict, idx: int = 0) -> Optional[Message]:
+    if not isinstance(d, dict):
+        return None
+    role = str(d.get("role") or "").strip().lower()
+    if role not in ("system", "user", "assistant", "tool"):
+        return None
+    calls = [c for c in (_tool_call(tc, i)
+                         for i, tc in enumerate(d.get("tool_calls") or [])) if c]
+    return Message(
+        role=role,
+        content=_content_text(d.get("content")),
+        reasoning=d.get("reasoning") if isinstance(d.get("reasoning"), str) else None,
+        tool_calls=calls,
+        tool_call_id=(str(d["tool_call_id"]) if d.get("tool_call_id") is not None
+                      else None),
+        tool_name=(str(d.get("tool_name") or d.get("name") or "") or None),
+        timestamp=(float(d["timestamp"]) if isinstance(d.get("timestamp"), (int, float))
+                   else None),
+    )
+
+
+def trace_from_record(rec: Any, *, fallback_id: str = "upload",
+                      source: str = "space-upload") -> Trace:
+    """Build a Trace from a native record, an OpenAI row, or a message list."""
+    if isinstance(rec, list):
+        meta: dict = {}
+        msgs_raw = rec
+    elif isinstance(rec, dict):
+        meta = rec
+        msgs_raw = rec.get("messages") or []
+    else:
+        raise ValueError(f"unsupported record type: {type(rec).__name__}")
+
+    msgs = [m for m in (message_from_dict(d, i)
+                        for i, d in enumerate(msgs_raw)) if m]
+    if not msgs:
+        raise ValueError("record has no usable messages")
+
+    sid = str(meta.get("session_id") or meta.get("id") or fallback_id)
+    return Trace(
+        session_id=sid,
+        model=meta.get("model"),
+        title=meta.get("title"),
+        started_at=(meta.get("started_at")
+                    if isinstance(meta.get("started_at"), (int, float)) else None),
+        cwd=meta.get("cwd"),
+        messages=msgs,
+        source=str(meta.get("source") or source),
+        improved=bool(meta.get("improved", False)),
+        parent_session_id=meta.get("parent_session_id"),
+        task=meta.get("task"),
+    )
+
+
+# ------------------------------------------------------------------- input
+
+def _iter_jsonl(text: str) -> Iterator[Any]:
+    for line in text.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        yield json.loads(line)
+
+
+def parse_records(text: str) -> tuple[list[Any], list[str]]:
+    """Parse a pasted/uploaded blob into records.
+
+    Accepts a JSON array, a single JSON object, or JSONL. Returns
+    `(records, errors)`; a row that fails to parse is reported by line.
+    """
+    text = (text or "").strip()
+    if not text:
+        return [], ["empty input"]
+    # A single JSON document (object or array) is the common case.
+    try:
+        doc = json.loads(text)
+        if isinstance(doc, list):
+            # A bare message list is ONE trace, not N records.
+            if doc and all(isinstance(x, dict) and "role" in x for x in doc):
+                return [doc], []
+            return doc, []
+        return [doc], []
+    except json.JSONDecodeError:
+        pass
+    records: list[Any] = []
+    errors: list[str] = []
+    for i, line in enumerate(text.splitlines(), 1):
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            records.append(json.loads(line))
+        except json.JSONDecodeError as e:
+            errors.append(f"line {i}: {e.msg}")
+    return records, errors
+
+
+def read_upload(path: str) -> tuple[list[Any], list[str]]:
+    """Read an uploaded file (JSON or JSONL) into records."""
+    if not path or not os.path.exists(path):
+        return [], [f"file not found: {path}"]
+    with open(path, "r", encoding="utf-8", errors="replace") as f:
+        return parse_records(f.read())
+
+
+def iter_traces(records: Iterable[Any], *, source: str = "space-upload"
+                ) -> Iterator[tuple[Any, Trace]]:
+    """Yield `(record, trace)`; a bad record raises with its index."""
+    for i, rec in enumerate(records):
+        yield rec, trace_from_record(rec, fallback_id=f"upload-{i}",
+                                     source=source)

traceforge/space/jobs.py

@@ -0,0 +1,118 @@
+"""Background worker: drain the store's queue without blocking the UI.
+
+A Space request must return in seconds, but one trace takes minutes. The
+worker is a single daemon thread (configurable) that claims the oldest
+queued hash, runs the pipeline, and records the outcome. It is safe to
+start twice — the second call is a no-op — and it survives a Gradio
+reload because the queue lives in the store, not in memory.
+"""
+from __future__ import annotations
+
+import os
+import threading
+import time
+from typing import Optional, Protocol
+
+from traceforge.space.pipeline import SpacePipeline
+from traceforge.space.store import TraceStore
+
+
+class _Pipeline(Protocol):
+    """What the worker needs: a store and a process(hash) -> dict."""
+    store: TraceStore
+
+    def process(self, h: str) -> dict: ...
+
+
+class Worker:
+    def __init__(self, pipeline: _Pipeline, *, poll_s: float = 2.0,
+                 name: str = "traceforge-space-worker"):
+        self.pipeline = pipeline
+        self.store: TraceStore = pipeline.store
+        self.poll_s = poll_s
+        self.name = name
+        self._stop = threading.Event()
+        self._thread: Optional[threading.Thread] = None
+        self._current: Optional[str] = None
+        self._last: Optional[dict] = None
+        self._started_at: Optional[float] = None
+        self._lock = threading.Lock()
+
+    # ------------------------------------------------------------- control
+
+    def start(self) -> "Worker":
+        with self._lock:
+            if self._thread and self._thread.is_alive():
+                return self
+            self._stop.clear()
+            self._thread = threading.Thread(target=self._run, name=self.name,
+                                            daemon=True)
+            self._started_at = time.time()
+            self._thread.start()
+        return self
+
+    def stop(self, timeout: float = 5.0) -> None:
+        self._stop.set()
+        t = self._thread
+        if t and t.is_alive():
+            t.join(timeout=timeout)
+
+    @property
+    def alive(self) -> bool:
+        return bool(self._thread and self._thread.is_alive())
+
+    def status(self) -> dict:
+        return {
+            "alive": self.alive,
+            "current": self._current,
+            "last": self._last,
+            "uptime_s": (round(time.time() - self._started_at, 1)
+                         if self._started_at else 0.0),
+            "queued": self.store.queue_depth(),
+        }
+
+    # ---------------------------------------------------------------- loop
+
+    def _run(self) -> None:
+        while not self._stop.is_set():
+            job = None
+            try:
+                job = self.store.claim_next()
+            except Exception:
+                job = None
+            if not job:
+                self._stop.wait(self.poll_s)
+                continue
+            h = job["hash"]
+            self._current = h
+            try:
+                self._last = self.pipeline.process(h)
+            except Exception as e:  # process() is meant not to raise; belt+braces
+                self._last = self.store.update(
+                    h, status="failed", error=f"worker crashed: {e}")
+            finally:
+                self._current = None
+
+
+# --------------------------------------------------------------- singleton
+
+_WORKER: Optional[Worker] = None
+_WORKER_LOCK = threading.Lock()
+
+
+def get_worker(pipeline: SpacePipeline, *, autostart: bool = True) -> Worker:
+    """One worker per process, started on first use."""
+    global _WORKER
+    with _WORKER_LOCK:
+        if _WORKER is None:
+            _WORKER = Worker(pipeline)
+        if autostart and not _WORKER.alive:
+            _WORKER.start()
+        return _WORKER
+
+
+def worker_count() -> int:
+    try:
+        return max(1, int(os.environ.get("TRACEFORGE_SPACE_WORKERS", "1")))
+    except ValueError:
+        return 1

traceforge/space/pipeline.py

@@ -0,0 +1,239 @@
+"""The pipeline behind the web UI.
+
+`SpacePipeline` is the only thing the Gradio app talks to. It wraps the
+windowed splice engine (`improve_trace_splice`) so a trace that arrives as
+JSON — not as a session id in a local store — can be processed:
+
+    submit(record)      hash -> dedupe -> enqueue (idempotent)
+    process(hash)       run triage + rewrite + gates, store the output
+
+Configuration is environment-only, because a Space has no `~/.hermes`:
+
+    TRACEFORGE_BASE_URL      OpenAI-compatible endpoint (…/v1)
+    TRACEFORGE_LLM_KEY       bearer token
+    TRACEFORGE_MODEL         writer/judge model id
+    TRACEFORGE_TRIAGE_MODEL  cheap labeler (default: TRACEFORGE_MODEL)
+    TRACEFORGE_JUDGE_MODEL   judge override (default: TRACEFORGE_MODEL)
+    TRACEFORGE_DATA_DIR      store root (default: <repo>/data/space)
+
+`split_long` is deliberately OFF: splitting one submitted seed into N
+segments would produce N outputs for one hash, which is exactly what the
+one-seed-one-output rule forbids. The splice engine already windows a long
+trace internally.
+"""
+from __future__ import annotations
+
+import os
+import shutil
+import time
+from typing import Any, Optional
+
+from traceforge.config import Config, repo_root_from_pkg
+from traceforge.space.hashing import fingerprint, output_hash, seed_hash
+from traceforge.space.ingest import trace_from_record
+from traceforge.space.store import (DONE, FAILED, QUEUED, REJECTED, RUNNING,
+                                    TraceStore)
+from traceforge.trace import Trace
+
+
+# ------------------------------------------------------------------- config
+
+def _env_int(name: str, default: int) -> int:
+    try:
+        return int(os.environ.get(name, "") or default)
+    except ValueError:
+        return default
+
+
+def space_config(data_dir: Optional[str] = None, **overrides: Any) -> Config:
+    """A Config that needs no Hermes profile — env vars only."""
+    root = repo_root_from_pkg()
+    cfg = Config()
+    cfg.data_dir = (data_dir or os.environ.get("TRACEFORGE_DATA_DIR")
+                    or os.path.join(root, "data", "space"))
+    cfg.guidelines_path = os.path.join(root, "guidelines",
+                                       "IDEAL_TRACE_GUIDELINES.md")
+    cfg.provider = os.environ.get("TRACEFORGE_PROVIDER") or None
+    cfg.model = os.environ.get("TRACEFORGE_MODEL") or "qwen-fable-selfhosted"
+    cfg.triage_model = (os.environ.get("TRACEFORGE_TRIAGE_MODEL")
+                        or cfg.model)
+    cfg.judge_model = os.environ.get("TRACEFORGE_JUDGE_MODEL") or ""
+    cfg.triage_base_url = (os.environ.get("TRACEFORGE_BASE_URL") or "").rstrip("/")
+    cfg.triage = True
+    cfg.splice = True
+    # The Space processes one submitted trace as one unit.
+    cfg.split_long = False
+    cfg.raw_cache = False
+    cfg.verify = False
+    cfg.review = False
+    cfg.splice_workers = _env_int("TRACEFORGE_SPLICE_WORKERS", 4)
+    cfg.triage_workers = _env_int("TRACEFORGE_TRIAGE_WORKERS", 4)
+    cfg.rewrite_timeout_s = _env_int("TRACEFORGE_REWRITE_TIMEOUT_S", 300)
+    cfg.judge_timeout_s = _env_int("TRACEFORGE_JUDGE_TIMEOUT_S", 300)
+    for k, v in overrides.items():
+        setattr(cfg, k, v)
+    return cfg.resolve(root)
+
+
+def pipeline_ready(cfg: Config) -> tuple[bool, str]:
+    """(ready, reason). A Space without an endpoint must say so, not hang."""
+    try:
+        from traceforge.triage import _resolve_endpoint
+        base, _key, model = _resolve_endpoint(cfg)
+    except Exception as e:  # RuntimeError from the resolver, or a bad config
+        return False, f"no LLM endpoint: {e}"
+    if not base:
+        return False, "no LLM endpoint configured (TRACEFORGE_BASE_URL)"
+    if not model:
+        return False, "no model configured (TRACEFORGE_MODEL)"
+    return True, f"{model} @ {base}"
+
+
+# ----------------------------------------------------------------- pipeline
+
+class SpacePipeline:
+    """Submit / process / report, with the store as the only state."""
+
+    def __init__(self, store: TraceStore, cfg: Optional[Config] = None,
+                 *, max_attempts: int = 2):
+        self.store = store
+        self.cfg = cfg or space_config()
+        self.max_attempts = max_attempts
+
+    # ------------------------------------------------------------- submit
+
+    def submit(self, record: Any, *, sponsor: str = "",
+               source: str = "space-upload") -> dict:
+        return self.store.submit(record, sponsor=sponsor, source=source)
+
+    def submit_many(self, records: list[Any], *, sponsor: str = "",
+                    source: str = "space-sponsor") -> list[dict]:
+        out = []
+        for rec in records:
+            try:
+                out.append(self.submit(rec, sponsor=sponsor, source=source))
+            except Exception as e:
+                out.append({"hash": "", "status": "error",
+                            "error": str(e), "duplicate": False})
+        return out
+
+    # ------------------------------------------------------------ process
+
+    def process(self, h: str) -> dict:
+        """Run the pipeline for one queued hash. Never raises."""
+        rec = self.store.get(h)
+        if not rec:
+            return {"hash": h, "status": FAILED, "error": "unknown hash"}
+        if rec.get("status") == DONE:
+            return {**rec, "skipped": True}
+
+        sid = f"space-{h[:16]}"
+        started = time.time()
+        self.store.update(h, status=RUNNING, started_at=started,
+                          attempts=int(rec.get("attempts") or 0) + 1)
+        try:
+            record = self.store.load_seed(h)
+            trace = trace_from_record(record, fallback_id=sid,
+                                      source="space-upload")
+            trace.session_id = sid
+            trace.source = "space-upload"
+            trace.improved = False
+            trace.parent_session_id = None
+        except Exception as e:
+            return self._fail(h, [f"unparseable seed: {e}"], started)
+
+        # A fresh attempt must not regate a stale artifact from a previous
+        # run (the splice engine's stage-0 fast path would promote it as-is).
+        workdir = os.path.join(self.cfg.improver_workdir, sid)
+        shutil.rmtree(workdir, ignore_errors=True)
+
+        try:
+            from traceforge.splice import improve_trace_splice
+            res = improve_trace_splice(
+                self.cfg, trace, max_attempts=self.max_attempts,
+            )
+        except Exception as e:
+            return self._fail(h, [f"pipeline crashed: {e}"], started)
+
+        if not res.get("ok") or res.get("improved") is None:
+            return self._fail(h, list(res.get("errors") or ["no output"]),
+                              started, route=res.get("route"),
+                              attempts=res.get("attempts"))
+
+        improved: Trace = res["improved"]
+        improved.session_id = sid
+        improved.parent_session_id = sid
+        improved.improved = True
+        improved.source = "traceforge-space"
+
+        # The splice engine gates before returning, but a structural check
+        # here is cheap and makes the Space's own contract explicit.
+        try:
+            from traceforge.validate import validate_structural
+            structural = validate_structural(improved)
+        except Exception as e:
+            structural = [f"validator crashed: {e}"]
+        if structural:
+            return self._fail(h, structural, started, route=res.get("route"),
+                              attempts=res.get("attempts"))
+
+        # Task bucket: heuristic only — the Space must not spend an LLM call
+        # on a label the dashboard can derive for free.
+        try:
+            from traceforge.classify import classify_heuristic
+            improved.task = classify_heuristic(improved).get("task")
+        except Exception:
+            pass
+
+        # Time sensitivity: measured on the RAW record (it may carry the
+        # `tools` array, which is what decides anchored vs sensitive) and on
+        # the ideal rewrite (did it actually call the retrieval tool?).
+        ts_raw = self._time_sensitivity(record)
+        ts_out = self._time_sensitivity(improved)
+
+        self.store.put_output(h, improved)
+        duration = time.time() - started
+        row = self.store.update(
+            h, status=DONE, finished_at=time.time(), duration_s=round(duration, 1),
+            output_hash=output_hash(improved),
+            route=res.get("route") or "",
+            attempts=res.get("attempts") or 0,
+            n_messages_out=len(improved.messages),
+            task=improved.task or rec.get("task") or "",
+            time_sensitivity=ts_raw,
+            time_sensitivity_improved=ts_out,
+            error="",
+        )
+        return row
+
+    def _fail(self, h: str, errors: list[str], started: float, *,
+              route: Optional[str] = None, attempts: Any = None) -> dict:
+        msg = "; ".join(str(e) for e in errors)[:1200]
+        return self.store.update(
+            h, status=FAILED, finished_at=time.time(),
+            duration_s=round(time.time() - started, 1),
+            route=route or "", attempts=attempts or 0, error=msg,
+        )
+
+    @staticmethod
+    def _time_sensitivity(obj: Any) -> dict:
+        try:
+            from traceforge.timesensitivity import detect
+            d = detect(obj)
+            return {k: d[k] for k in ("class", "label", "temporal_facts",
+                                      "retrieval_available", "retrieval_called",
+                                      "needs_rewrite", "score")}
+        except Exception as e:
+            return {"class": "", "label": f"unavailable: {e}"}
+
+    # -------------------------------------------------------------- report
+
+    def result(self, h: str) -> dict:
+        """Everything the UI needs about one hash."""
+        rec = self.store.get(h) or {"hash": h, "status": "unknown"}
+        out = self.store.load_output(h)
+        rec = dict(rec)
+        rec["has_output"] = out is not None
+        if out is not None:
+            rec["output_messages"] = len(out.messages)
+        return rec