research/deepread/13-synthesis-architecture.md

# Synthesis: Critical-Review Verdict + The Dataset-Generation Pipeline Architecture

> **Date:** 2026-06-09. **Inputs:** the 8 deep-reads (`01`–`08`), grounding map (`00`),
> verified findings (`12`), design-critic minimal pipeline (`11` §MINIMAL).
> **Provenance:** 8 source-cluster readers re-fetched and re-read every primary source
> (Composer 2.5 blog verbatim, Composer 2 techreport arXiv:2603.24477 full HTML,
> SWE-smith/SWE-Gym/R2E-Gym/SWE-bench full texts, SDPO/OPSD, Dr.GRPO/DAPO/GSPO/CISPO +
> Comedy-of-Estimators, DiLoCo/Streaming-DiLoCo, MuZero/Dreamer/CWM + the anti-evidence
> cluster, LATS/ToT/rStar/Tree-GRPO/SWE-Search/Symphony/Socratic-SWE, TRL/verl/SkyRL live
> docs, SWE-MiniSandbox); 2 adversarial critics' findings were then independently
> verified line-by-line — **0 refuted**.

---

## Part A — The critical-review verdict in one page

**The recipe-replication layer (Channels 1+2 mechanics, env, safeguards) is solid and
mostly faithful. The *story we tell about it* has confirmed fidelity defects, and the
*envisioned dataset pipeline* has four structural breaks that would have made it
unbuildable as drawn.** The good news: every break has a cheap, evidence-backed fix, and
the biggest fix is a *buy* (SWE-smith), not a build.

### What survived adversarial review intact
- Feature-deletion-by-gold-patch-reversion as the task mechanic — **independently
  validated** by SWE-smith's ablation (its "PR Mirror" strategy *is* our mechanic and
  produces the **best** training data of its five strategies, Table 5).
- The execution-oracle reward (`_grade()` masked pass-fraction), the 4-gate validator,
  `scrub_tree` as primary anti-hack control, fractional-credit curriculum (SELECT half).
- The k1-in-reward KL fix (Composer-2 §4.1 verbatim confirms k1 = −log r in reward,
  citing the same variance argument) and the behavior-rewards bank (§4.2 verbatim).
- The DiLoCo-over-S3 substrate (math verified against torchft; live-S3 validated).
- The honest-provenance discipline itself: Channel 3 + tree are OUR additions, not
  Cursor's — re-confirmed.

### The four structural breaks in the envisioned pipeline (all CONFIRMED)
1. **Seed-trace/oracle disjointness.** The tree was drawn growing off Claude Code traces,
   but those traces have no executable environment (no `broken_image`, no
   `fail_to_pass`) — `FeatureDeletionEnv` literally cannot `reset()` on them. The tree
   must seed from **env-grounded rollouts**, which don't exist yet because…
2. **No rollout harness.** Nothing in the repo runs an agent loop against
   `FeatureDeletionEnv` to completion. The SFT corpus has **no producer**. This is the
   single highest-priority build item.
3. **Divergence gate uncomputable.** `_normalize_action` is a whitespace-collapse stub;
   teachers return free text; there is no tool-call action algebra to compare. Ungated,
   the tree is O(N^D).
4. **No `Sandbox.fork()`.** Branching from a mid-trajectory state requires state
   cloning the Sandbox protocol doesn't have.

### The five worst fidelity defects (all CONFIRMED, now to be corrected)
1. "SDPO is mathematically the same as Composer's mechanism" — **wrong**; Cursor cites
   SDPO/OPSD only as *background*; SDPO's published loss is full-rollout with
   feedback-in-prefix + EMA-regularized teacher; ours is a turn-localized hint-splice
   with a live teacher. It is a *third design* (blog-inspired), and must be labeled so.
2. Fabricated numbers circulating as Cursor-stated: 69.3% CursorBench / Terminal-Bench
   parity (in no primary source), "24 other generators" (back-formed from "25x"),
   "85% post-training compute" (community speculation).
3. CWM misread: it trains-on-all in a separate **mid-training stage**, not as an aux
   head riding RL gradients; its 65.8% requires test-time scaling; Chain-of-World is a
   **robotics** paper. The world-model head has **zero** direct published evidence for
   the exact proposed configuration — it stays an ablation arm.
4. Streaming DiLoCo citation doubly wrong (2501.18512 = Douillard et al. "Streaming
   DiLoCo"; "Eager Updates" = 2502.12996 Kale et al.).
5. Cost figures mislabeled: "$0.98/trace" is a 50-state synthetic benchmark (real
   sessions ≈ $70–80 flat); "$64 ungated tree" is a flat 8×1000 extrapolation, not a
   tree.

### Missing pieces no design mentioned (all now in the architecture)
**Benchmark decontamination** (zero mentions anywhere — training substrates overlap
SWE-bench eval repos), **secrets/PII scrub at trace ingest**, **SPDX license detection
at repo ingest**, **canonical trajectory IR**, **cross-generation dedup**,
**`golden_diff` serialization leak** (`repr=False` does not survive `asdict()`),
**corpus acceptance probe**, **two unreconciled S3 contracts**.

---

## Part B — The architecture: "point at a repo → enhanced dataset"

**The committed answer to the user's question: yes — point at an open-source repo and
build the dataset. The engine is SWE-smith (buy), the trajectories come from a rollout
harness (build), and our vision enhancements (oracle-graded multi-candidate divergence,
typed signal routing) layer on top as Stage 1+ — strictly after Stage 0 produces one
real corpus end-to-end locally.**

### Why SWE-smith is the engine (the buy-vs-build verdict)
- `pip install swesmith` (MIT) ships: env construction from arbitrary GitHub repos
  (one Docker image **per repo**, 500× more storage-efficient than per-task), five
  bug-synthesis strategies (LM Modify 56% yield / LM Rewrite 35% / Procedural-AST 40%
  at $0 / Combine 96.9% at $0 / PR Mirror 33.8%), issue-text generation, and
  validation-by-test-execution. 50k tasks for **$1,360 + 20 human-hours** total.
- Its **PR Mirror ≡ our gold-patch reversion** — and its ablation shows PR Mirror
  trajectories train the best models. The repo's core mechanic is *independently
  validated*; what we were about to hand-build is exactly what the toolkit ships.
- Its **Combine-Bugs** (96.9% yield, $0) is the cheapest implementation of the blog's
  "create harder tasks" CREATE-half — multi-bug escalation for free.
- R2E-Gym's SweGen covers the "commits without tests" case (LLM-synthesized F2P tests,
  27.8% vs 28.0% — indistinguishable from real) — the fallback when a pointed-at repo
  has thin test coverage. Adopt as data (R2E-Gym-Subset), not code, for now.
- Composer-2 reward nuance (from the techreport): reward = correctness + succinctness +
  SE-principles. Our pass-fraction is the correctness core; `behavior_rewards.py`
  (Wave 20) already carries the succinctness/style components.

### The Stage-0 pipeline (local, no new AWS services)

```
            point at repo URL (or HF substrate, or trace dir)
                              │
   ┌──────────────────────────▼───────────────────────────┐
   │ 1. INGEST GATE  (datagen/repo_gate.py)                │
   │    SPDX license detect → trainable/redistributable    │
   │    tier; BENCHMARK DECONTAMINATION (repo ∉ SWE-bench/ │
   │    Verified/Lite/Multimodal eval repo lists)          │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 2. TASK SYNTHESIS (buy: swesmith)                     │
   │    swesmith profile → env image → PR-Mirror first,    │
   │    Combine-Bugs for escalation (CREATE-half), 13      │
   │    procedural AST transforms for volume               │
   │    → SwesmithAdapter.to_task() → FeatureDeletionTask  │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 3. VALIDATE (have): 4-gate validate_task() in         │
   │    DockerSandbox + scrub_tree                         │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 4. ROLLOUT HARNESS (build — the critical missing      │
   │    component): agent loop over FeatureDeletionEnv     │
   │    (prompt → act → env.step → … → submit → _grade()), │
   │    pluggable policy (frontier API / local model),     │
   │    $cap per task. Output: CanonicalTrajectory.        │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 5. ADMIT + TYPE (have + build): _grade()==1.0 +       │
   │    HackMonitor-clean + PASS_TO_PASS → sft/;           │
   │    near-misses → dpo candidates; failures → withheld  │
   │    (wm_tuples only when the P4 ablation is scheduled) │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 6. CORPUS (build): CanonicalTrajectory → messages-    │
   │    schema SFT rows via to_policy_row() (golden_diff   │
   │    PROVABLY absent — unit-tested); MinHash dedup      │
   │    (cross-generation aware); HeldoutSplit; secrets    │
   │    scrub; ONE reconciled S3/local layout + manifest   │
   │    + dataset card  (pipeline/s3_contract.py)          │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   7. ACCEPTANCE PROBE: small-model SFT (LoRA) on each corpus
      generation; require measurable holdout delta before
      promote-to-accepted.  (the dataset analogue of HeldOutGuard)
```

**Free byproduct:** step 4's trajectories are *env-grounded* — exactly the seed nodes
the tree-of-work needs, fixing structural break #1 without extra work. Claude Code
traces are demoted to flat Channel-3 / SFT-style uses (their honest capability).

### What the vision enhancements add, strictly in order
- **Stage 1 — DPO channel + tool-call algebra:** parse tool calls into a canonical
  action form (`ToolCall(name, normalized_args)`), fix `_normalize_action`, extract
  DPO pairs from env-grounded multi-candidate rollouts (N samples per state, each
  env-stepped + graded — depth-1, no fork needed). Bedrock batch / AWS Batch only when
  volume justifies.
- **Stage 2 — depth-1 tree → divergence gate measurement:** N candidates per decision
  point, one env-step each, oracle-graded. Measure the divergence gate's firing rate on
  real traces BEFORE building depth>1. `Sandbox.fork()` spike gates depth>1.
- **Stage 3 — curriculum CREATE-half (Combine-Bugs escalation), flywheel with
  cross-generation dedup, wm_tuples emission only when P4 ablation scheduled.**
- **Stage 4 — Step Functions/Argo orchestration once runs are routine.**

### Build manifest (Stage 0, ~900 LOC + 1 adopted dep)

| # | Module | What | ~LOC |
|---|---|---|---|
| 1 | `datagen/repo_gate.py` | SPDX license detection (LICENSE/classifier heuristics) + trainable-vs-redistributable tiers + benchmark-decontamination list (SWE-bench{,-Lite,-Verified,-Multimodal}+SWE-Gym eval repos) + gate verdict dataclass | ~150 |
| 2 | `datagen/swesmith_adapter.py` | swesmith task instance → `FeatureDeletionTask` (mirror of SweBenchAdapter; handles swesmith's image naming, F2P/P2P, strategy provenance field) + optional thin synthesis driver behind `[swesmith]` extra | ~120 |
| 3 | `datagen/trajectory.py` | `CanonicalTrajectory` IR: steps of (obs, ToolCall-or-text action, result, error flag), provenance, grade; adapters: ClaudeCode TraceState→IR; IR→SFT messages; IR→`to_policy_row()` (golden_diff/deleted_symbols PROVABLY dropped) | ~180 |
| 4 | `datagen/rollout_harness.py` | `RolloutPolicy` protocol (pluggable: API model / local / scripted-fake); `collect_trajectory(env, task, policy, max_turns, budget)` loop; admission filter (`_grade()==1.0` + monitor-clean + guard) | ~220 |
| 5 | `pipeline/s3_contract.py` | THE single reconciled layout (file:// + s3:// via fsspec): `runs/<run_id>/{tasks,traj,corpus_sft,corpus_dpo,holdout,quarantine}/` + run manifest (counts, cost, lineage, schema_version, parent_run_id) + dataset card writer | ~200 |
| 6 | `pipeline/dedup.py` | MinHash near-dup detection over SFT rows; cross-generation aware (accepts prior-run signature file) | ~120 |
| 7 | `pipeline/build_corpus.py` | the local stage-driver wiring 1→6: `build_corpus(source, out, policy, budget)`; source = swesmith repo / SWE-* substrate rows / trace dir | ~180 |
| 8 | Fidelity-fix batch | research/01 + COMPOSER_RECIPE_MAPPING strikes/tags; opsd.py β + "same loss" claims; diloco citation; kl_in_reward wording; teacher_replay cost docstring; ADR-016 records all of Part A | docs |

Everything testable CPU-only with fakes (swesmith synthesis itself needs Docker+Linux —
the adapter and driver are tested on fixture instances, the live path is
`skipif`-gated like the existing Docker e2e).

### Pre-registered falsifiers (kept from the original vision, now evidence-bounded)
- Heterogeneous-N-models vs single-model-N-samples at equal compute (Symphony vs
  data-processing-inequality — genuinely two-sided, SWE-specific answer unrun).
- World-model aux head: build only if the P0–P6 ladder's P4/P6 beat P0–P3 on
  foresight+calibration (CWM supports mid-training train-on-all, NOT an RL-time aux
  head — the proposed configuration is in a null-evidence zone).
- Tree depth>1: build only if the measured divergence-gate firing rate makes
  O(N·decision-points) real, and only after `Sandbox.fork()` exists.