research/deepread/13-synthesis-architecture.md

Synthesis: Critical-Review Verdict + The Dataset-Generation Pipeline Architecture

Date: 2026-06-09. Inputs: the 8 deep-reads (01–08), grounding map (00), verified findings (12), design-critic minimal pipeline (11 §MINIMAL). Provenance: 8 source-cluster readers re-fetched and re-read every primary source (Composer 2.5 blog verbatim, Composer 2 techreport arXiv:2603.24477 full HTML, SWE-smith/SWE-Gym/R2E-Gym/SWE-bench full texts, SDPO/OPSD, Dr.GRPO/DAPO/GSPO/CISPO + Comedy-of-Estimators, DiLoCo/Streaming-DiLoCo, MuZero/Dreamer/CWM + the anti-evidence cluster, LATS/ToT/rStar/Tree-GRPO/SWE-Search/Symphony/Socratic-SWE, TRL/verl/SkyRL live docs, SWE-MiniSandbox); 2 adversarial critics' findings were then independently verified line-by-line — 0 refuted.

---

Part A — The critical-review verdict in one page

The recipe-replication layer (Channels 1+2 mechanics, env, safeguards) is solid and mostly faithful. The story we tell about it has confirmed fidelity defects, and the envisioned dataset pipeline has four structural breaks that would have made it unbuildable as drawn. The good news: every break has a cheap, evidence-backed fix, and the biggest fix is a buy (SWE-smith), not a build.

What survived adversarial review intact

• Feature-deletion-by-gold-patch-reversion as the task mechanic — independently validated by SWE-smith's ablation (its "PR Mirror" strategy is our mechanic and produces the best training data of its five strategies, Table 5).
• The execution-oracle reward (_grade() masked pass-fraction), the 4-gate validator, scrub_tree as primary anti-hack control, fractional-credit curriculum (SELECT half).
• The k1-in-reward KL fix (Composer-2 §4.1 verbatim confirms k1 = −log r in reward, citing the same variance argument) and the behavior-rewards bank (§4.2 verbatim).
• The DiLoCo-over-S3 substrate (math verified against torchft; live-S3 validated).
• The honest-provenance discipline itself: Channel 3 + tree are OUR additions, not Cursor's — re-confirmed.

The four structural breaks in the envisioned pipeline (all CONFIRMED)

1. Seed-trace/oracle disjointness. The tree was drawn growing off Claude Code traces, but those traces have no executable environment (no broken_image, no fail_to_pass) — FeatureDeletionEnv literally cannot reset() on them. The tree must seed from env-grounded rollouts, which don't exist yet because…
2. No rollout harness. Nothing in the repo runs an agent loop against FeatureDeletionEnv to completion. The SFT corpus has no producer. This is the single highest-priority build item.
3. Divergence gate uncomputable. _normalize_action is a whitespace-collapse stub; teachers return free text; there is no tool-call action algebra to compare. Ungated, the tree is O(N^D).
4. No Sandbox.fork(). Branching from a mid-trajectory state requires state cloning the Sandbox protocol doesn't have.

The five worst fidelity defects (all CONFIRMED, now to be corrected)

1. "SDPO is mathematically the same as Composer's mechanism" — wrong; Cursor cites SDPO/OPSD only as background; SDPO's published loss is full-rollout with feedback-in-prefix + EMA-regularized teacher; ours is a turn-localized hint-splice with a live teacher. It is a third design (blog-inspired), and must be labeled so.
2. Fabricated numbers circulating as Cursor-stated: 69.3% CursorBench / Terminal-Bench parity (in no primary source), "24 other generators" (back-formed from "25x"), "85% post-training compute" (community speculation).
3. CWM misread: it trains-on-all in a separate mid-training stage, not as an aux head riding RL gradients; its 65.8% requires test-time scaling; Chain-of-World is a robotics paper. The world-model head has zero direct published evidence for the exact proposed configuration — it stays an ablation arm.
4. Streaming DiLoCo citation doubly wrong (2501.18512 = Douillard et al. "Streaming DiLoCo"; "Eager Updates" = 2502.12996 Kale et al.).
5. Cost figures mislabeled: "$0.98/trace" is a 50-state synthetic benchmark (real sessions ≈ $70–80 flat); "$64 ungated tree" is a flat 8×1000 extrapolation, not a tree.

Missing pieces no design mentioned (all now in the architecture)

Benchmark decontamination (zero mentions anywhere — training substrates overlap SWE-bench eval repos), secrets/PII scrub at trace ingest, SPDX license detection at repo ingest, canonical trajectory IR, cross-generation dedup, golden_diff serialization leak (repr=False does not survive asdict()), corpus acceptance probe, two unreconciled S3 contracts.

---

Part B — The architecture: "point at a repo → enhanced dataset"

The committed answer to the user's question: yes — point at an open-source repo and build the dataset. The engine is SWE-smith (buy), the trajectories come from a rollout harness (build), and our vision enhancements (oracle-graded multi-candidate divergence, typed signal routing) layer on top as Stage 1+ — strictly after Stage 0 produces one real corpus end-to-end locally.

Why SWE-smith is the engine (the buy-vs-build verdict)

• pip install swesmith (MIT) ships: env construction from arbitrary GitHub repos (one Docker image per repo, 500× more storage-efficient than per-task), five bug-synthesis strategies (LM Modify 56% yield / LM Rewrite 35% / Procedural-AST 40% at $0 / Combine 96.9% at $0 / PR Mirror 33.8%), issue-text generation, and validation-by-test-execution. 50k tasks for $1,360 + 20 human-hours total.
• Its PR Mirror ≡ our gold-patch reversion — and its ablation shows PR Mirror trajectories train the best models. The repo's core mechanic is independently validated; what we were about to hand-build is exactly what the toolkit ships.
• Its Combine-Bugs (96.9% yield, $0) is the cheapest implementation of the blog's "create harder tasks" CREATE-half — multi-bug escalation for free.
• R2E-Gym's SweGen covers the "commits without tests" case (LLM-synthesized F2P tests, 27.8% vs 28.0% — indistinguishable from real) — the fallback when a pointed-at repo has thin test coverage. Adopt as data (R2E-Gym-Subset), not code, for now.
• Composer-2 reward nuance (from the techreport): reward = correctness + succinctness + SE-principles. Our pass-fraction is the correctness core; behavior_rewards.py (Wave 20) already carries the succinctness/style components.

The Stage-0 pipeline (local, no new AWS services)

            point at repo URL (or HF substrate, or trace dir)
                              │
   ┌──────────────────────────▼───────────────────────────┐
   │ 1. INGEST GATE  (datagen/repo_gate.py)                │
   │    SPDX license detect → trainable/redistributable    │
   │    tier; BENCHMARK DECONTAMINATION (repo ∉ SWE-bench/ │
   │    Verified/Lite/Multimodal eval repo lists)          │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 2. TASK SYNTHESIS (buy: swesmith)                     │
   │    swesmith profile → env image → PR-Mirror first,    │
   │    Combine-Bugs for escalation (CREATE-half), 13      │
   │    procedural AST transforms for volume               │
   │    → SwesmithAdapter.to_task() → FeatureDeletionTask  │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 3. VALIDATE (have): 4-gate validate_task() in         │
   │    DockerSandbox + scrub_tree                         │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 4. ROLLOUT HARNESS (build — the critical missing      │
   │    component): agent loop over FeatureDeletionEnv     │
   │    (prompt → act → env.step → … → submit → _grade()), │
   │    pluggable policy (frontier API / local model),     │
   │    $cap per task. Output: CanonicalTrajectory.        │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 5. ADMIT + TYPE (have + build): _grade()==1.0 +       │
   │    HackMonitor-clean + PASS_TO_PASS → sft/;           │
   │    near-misses → dpo candidates; failures → withheld  │
   │    (wm_tuples only when the P4 ablation is scheduled) │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   ┌──────────────────────────────────────────────────────┐
   │ 6. CORPUS (build): CanonicalTrajectory → messages-    │
   │    schema SFT rows via to_policy_row() (golden_diff   │
   │    PROVABLY absent — unit-tested); MinHash dedup      │
   │    (cross-generation aware); HeldoutSplit; secrets    │
   │    scrub; ONE reconciled S3/local layout + manifest   │
   │    + dataset card  (pipeline/s3_contract.py)          │
   └──────────────────────────┬───────────────────────────┘
                              ▼
   7. ACCEPTANCE PROBE: small-model SFT (LoRA) on each corpus
      generation; require measurable holdout delta before
      promote-to-accepted.  (the dataset analogue of HeldOutGuard)

Free byproduct: step 4's trajectories are env-grounded — exactly the seed nodes the tree-of-work needs, fixing structural break #1 without extra work. Claude Code traces are demoted to flat Channel-3 / SFT-style uses (their honest capability).

What the vision enhancements add, strictly in order

• Stage 1 — DPO channel + tool-call algebra: parse tool calls into a canonical action form (ToolCall(name, normalized_args)), fix _normalize_action, extract DPO pairs from env-grounded multi-candidate rollouts (N samples per state, each env-stepped + graded — depth-1, no fork needed). Bedrock batch / AWS Batch only when volume justifies.
• Stage 2 — depth-1 tree → divergence gate measurement: N candidates per decision point, one env-step each, oracle-graded. Measure the divergence gate's firing rate on real traces BEFORE building depth>1. Sandbox.fork() spike gates depth>1.
• Stage 3 — curriculum CREATE-half (Combine-Bugs escalation), flywheel with cross-generation dedup, wm_tuples emission only when P4 ablation scheduled.
• Stage 4 — Step Functions/Argo orchestration once runs are routine.

Build manifest (Stage 0, ~900 LOC + 1 adopted dep)

#	Module	What	~LOC
1	datagen/repo_gate.py	SPDX license detection (LICENSE/classifier heuristics) + trainable-vs-redistributable tiers + benchmark-decontamination list (SWE-bench{,-Lite,-Verified,-Multimodal}+SWE-Gym eval repos) + gate verdict dataclass	~150
2	datagen/swesmith_adapter.py	swesmith task instance → FeatureDeletionTask (mirror of SweBenchAdapter; handles swesmith's image naming, F2P/P2P, strategy provenance field) + optional thin synthesis driver behind [swesmith] extra	~120
3	datagen/trajectory.py	CanonicalTrajectory IR: steps of (obs, ToolCall-or-text action, result, error flag), provenance, grade; adapters: ClaudeCode TraceState→IR; IR→SFT messages; IR→to_policy_row() (golden_diff/deleted_symbols PROVABLY dropped)	~180
4	datagen/rollout_harness.py	RolloutPolicy protocol (pluggable: API model / local / scripted-fake); collect_trajectory(env, task, policy, max_turns, budget) loop; admission filter (_grade()==1.0 + monitor-clean + guard)	~220
5	pipeline/s3_contract.py	THE single reconciled layout (file:// + s3:// via fsspec): runs/<run_id>/{tasks,traj,corpus_sft,corpus_dpo,holdout,quarantine}/ + run manifest (counts, cost, lineage, schema_version, parent_run_id) + dataset card writer	~200
6	pipeline/dedup.py	MinHash near-dup detection over SFT rows; cross-generation aware (accepts prior-run signature file)	~120
7	pipeline/build_corpus.py	the local stage-driver wiring 1→6: build_corpus(source, out, policy, budget); source = swesmith repo / SWE-* substrate rows / trace dir	~180
8	Fidelity-fix batch	research/01 + COMPOSER_RECIPE_MAPPING strikes/tags; opsd.py β + "same loss" claims; diloco citation; kl_in_reward wording; teacher_replay cost docstring; ADR-016 records all of Part A	docs

Everything testable CPU-only with fakes (swesmith synthesis itself needs Docker+Linux — the adapter and driver are tested on fixture instances, the live path is skipif-gated like the existing Docker e2e).

Pre-registered falsifiers (kept from the original vision, now evidence-bounded)

• Heterogeneous-N-models vs single-model-N-samples at equal compute (Symphony vs data-processing-inequality — genuinely two-sided, SWE-specific answer unrun).
• World-model aux head: build only if the P0–P6 ladder's P4/P6 beat P0–P3 on foresight+calibration (CWM supports mid-training train-on-all, NOT an RL-time aux head — the proposed configuration is in a null-evidence zone).
• Tree depth>1: build only if the measured divergence-gate firing rate makes O(N·decision-points) real, and only after Sandbox.fork() exists.