Baladithya Balamurugan

Wave 21: deep-read critical review — 8 source clusters re-read, findings verified

2a16b30 27 days ago

33.2 kB

Deep-Read Critical Review: Trace-Replay / Multi-Teacher Distillation / MCTS-for-LLM-Agents

Cluster 07 — Channel 3 + Tree-of-Work

Reviewer: agent (Claude Sonnet 4.6) Date: 2026-06-10 Scope: composer_replication/teacher_replay.py, research/05-trace-replay-distillation.md, research/notes/final_report_socratic-mcts-swe-worldmodel-8f6dea.md §§1,3,6,7,10, docs/adrs/ADR-002-trace-source.md, docs/adrs/ADR-013-lma-integration-channel-ladder.md

0. TL;DR

Channel 3 (teacher_replay.py) and the tree-of-work design are neither pure novelty nor pure reinvention. The specific combination — frozen-trace replay at every decision step across N heterogeneous frontier models, consensus-disagreement-as-DPO-signal, divergence-gated recursive branching with an execution oracle, and world-model auxiliary loss — is genuinely novel synthesis. But the repo's research note (research/05) overclaims novelty in three specific ways, the final report's cost estimates are internally inconsistent, the "closest precedent" identification (rStar) is partially wrong, and there is a missing piece (agent-trace distillation at code-repo scale already exists in published form at direct overlap) that neither research/05 nor the final report adequately cites. This review quotes sources verbatim, identifies every misread, and gives a verdict on each cost figure.

1. What Was Fetched / Read

Primary sources fetched and read:

LATS (arXiv:2310.04406, Zhou et al. 2023): Language Agent Tree Search — MCTS over agent reasoning/acting/planning steps with external env feedback and LM-valued nodes.
rStar (arXiv:2408.06195, Qi et al. 2024): mutual self-play two-SLM MCTS for math reasoning; generator+discriminator role split.
rStar-Math (arXiv:2501.04519, Guan et al. 2025): MCTS-driven step-verified data synthesis for math SLMs; trains PPM (process preference model) from tree rollouts.
Tree-of-Thoughts (arXiv:2305.10601, Yao et al. 2023): foundational ToT framework, BFS/DFS over thought trees, LM self-evaluation.
SWE-Search (arXiv:2410.20285, Antoniades et al. 2024): MCTS over SWE-bench tasks; hybrid LM value function, Value Agent, Discriminator Agent; 23% relative gain over no-MCTS; scales with inference-time compute.
Tree-GRPO (arXiv:2509.21240, Ji et al. 2025/ICLR 2026): tree-structured group-relative policy optimization; shared-prefix branching; proves intra-tree group-relative advantage == step-level DPO.
Socratic-SWE (arXiv:2606.07412, Xiao et al. 2026): closed-loop self-evolving SWE agent; trace-derived Agent Skill Registry; 4-gate verifier; solver-gradient alignment reward; 50.40% SWE-bench Verified after 3 iters.
Symphony (arXiv:2601.22623, Zhu et al. 2026/NeurIPS 2025): heterogeneous LM pool in MCTS planning; argues single-agent MCTS has insufficient branch diversity; heterogeneous pool improves rollout diversity.
SWE-Gym (arXiv:2412.21139, Pan et al. 2024/ICML 2025): 2,438 real-world Python task environments; inference-time scaling via trained verifiers; 32.0%/26.0% on SWE-bench Verified/Lite.
Code World Model MCTS (arXiv:2405.15383, Dainese et al. 2024): MCTS-guided code world model generation; GIF-MCTS; NeurIPS 2024.
Tree Search for LM Agents (arXiv:2407.01476, Koh et al. 2024): best-first tree search for web agents; 39.7% relative gain on VisualWebArena with GPT-4o; effective at realistic tasks.

Repo files read in full:

composer_replication/teacher_replay.py (281 LOC)
research/05-trace-replay-distillation.md
research/notes/final_report_socratic-mcts-swe-worldmodel-8f6dea.md (§§1–10, full)
docs/adrs/ADR-002-trace-source.md
docs/adrs/ADR-013-lma-integration-channel-ladder.md

2. What the Literature Actually Says

2a. Replaying a trace prefix across N heterogeneous models then branching — has anyone done it?

Short answer: The exact frozen-trace multi-teacher replay mechanism is not directly published. But the components are deeply covered and combinatorially close work exists.

The relevant lineage:

Tree-of-Thoughts / LATS / SWE-Search all expand nodes via a policy (one model, sometimes self-evaluated) and do NOT fix a human-collected trace then replay each step with multiple heterogeneous external models. The branching is forward-generative, not retrospective-replay.
rStar (the closest analogue cited in research/05) uses two SLMs in fixed roles (generator + discriminator) at trajectory-level evaluation. The discriminator evaluates the full trajectory, not each step position independently. Quote from the abstract: "another SLM, with capabilities similar to the target SLM, acts as a discriminator to verify each trajectory generated by the target SLM." This is trajectory-level verdict, not per-step replay with N teachers.
rStar-Math does per-step MCTS rollouts, but these are single-policy forward rollouts (not fixed-trace, not multi-teacher). The PPM then scores steps. No external teacher heterogeneity.
SWE-Gym trains verifiers on agent trajectories and scales inference via them, but the training data collection does not use multi-teacher per-step replay.
AgentTrek (cited in research/05) uses "guided replay demonstrations" from web tutorials — semantically similar name but operationally different: it is single-teacher demonstration collection, not multi-teacher disagreement harvesting.

Verdict on (a): Nobody has published the exact frozen-trace-prefix → N-teacher parallel replay → disagreement-as-DPO mechanism at the agentic-step level. The combination is genuinely novel. However, research/05's claim that "No published work systematically freezes a trace at step t, replays that exact state with N different teachers, harvests variance as per-step supervision" is accurate — but research/05 then goes on to undercite the work that comes closest, which is more than rStar.

2b. Execution-oracle-graded tree search over real repos at scale — what does the literature actually say and what does it cost?

Published execution-oracle tree search at repo scale:

SWE-Search (arXiv:2410.20285): MCTS over SWE-bench using a hybrid value function (LLM-estimated numerical + qualitative). Critical nuance: the value function is LLM-estimated, NOT a true execution oracle. It uses a "Value Agent" that provides qualitative feedback. Full pytest runs are expensive; SWE-Search's value is LM-evaluated fitness, not test-suite-pass-fraction. The 23% relative gain is impressive but it is LM-valued MCTS, not execution-oracle MCTS.
SWE-Gym verifiers (arXiv:2412.21139): trains a verifier (a model that learns to score trajectory quality) on trajectories, then uses it for inference-time selection. The verifier learns execution outcome but is not the oracle itself during search.
Socratic-SWE (arXiv:2606.07412): uses a 4-gate verifier — Format/Grounding/Execution/Semantics — for task validation, NOT as the tree search value function. The training reward is Valid(τ)·cos(g_τ, G_val) — solver-gradient alignment, NOT test-suite pass-fraction. The execution gate is a solvability filter, not a step-level oracle.

Only the repo's own FeatureDeletionEnv._grade() (and CWM arXiv:2510.02387, which trains a model on execution trajectories) use raw test-suite pass-fraction as the MCTS/RL reward at training time. The final report correctly identifies this as the critical differentiator.

Cost data for execution-oracle tree search:

The literature does NOT provide a clean "$X per trace" figure for execution-oracle MCTS at real repo scale. The closest:

DeepSWE (cited as [43] in the final report) ran 64 H100s for six days on 0/1 sparse reward outcome-RL with no branching tree — making the per-trace cost of a flat outcome-RL run tractable but providing no per-branch cost.
SWE-rebench infrastructure (nebius.com, [54]) evaluates thousands of SWE instances/hour via distributed container orchestration — providing a throughput floor but not a per-branch dollar figure.

2c. Cross-model disagreement as a branch gate — is it a useful signal?

Tree-GRPO (arXiv:2509.21240, ICLR 2026) provides the strongest theoretical backing: it proves that intra-tree group-relative advantage estimation is equivalent to step-level direct preference learning. This directly validates the repo's claim that "sibling divergence → DPO signal" is principled. Quote from abstract: "we demonstrate that the objective of intra-tree level group relative policy optimization is equivalent to that of step-level direct preference learning."

However, this is about same-model intra-tree branching (shared prefixes, different continuations), not heterogeneous-model branching. The equivalence proof does not cover cross-family disagreement.

Symphony (arXiv:2601.22623, NeurIPS 2025) argues explicitly that "single-agent MCTS yields insufficient branch diversity and a heterogeneous LM pool improves rollout diversity and exploration." This endorses cross-model disagreement as a diversity mechanism — but note: Symphony tests this in planning tasks, not repo-level SWE.

Counterpoint: arXiv:2604.02460 (Single-Agent LLMs Outperform Multi-Agent, Tran & Kiela 2026) makes the data-processing inequality argument that under equal compute, single-agent is information-theoretically superior. The SWE-specific result is unrun (as the final report correctly acknowledges).

2d. DPO-pair extraction from sibling branches

Tree-GRPO (arXiv:2509.21240) provides the clearest theoretical backing for this being well-founded. The repo's extract_dpo_pairs implements a simpler heuristic (Counter over normalized actions, break-on-first-pair per state) — this is a sound but weaker version of what Tree-GRPO formalizes. The repo's implementation does not implement the intra-tree advantage weighting Tree-GRPO derives.

3. Critical Review of `research/05-trace-replay-distillation.md`

Finding 05-R1: Overclaim on rStar as "closest precedent" (MISREAD)

research/05 Section "The Closest Published Precedent" frames rStar as the closest precedent and describes it as having "multi-model step-level evaluation." This is partially wrong:

rStar's discriminator evaluates full trajectories, not individual steps. Quote from rStar abstract: "acts as a discriminator to verify each trajectory." The per-step MCTS action expansion in rStar uses the generator SLM alone; the discriminator only votes on completed trajectories.
The correct characterization: rStar is close because it uses two models with agreement as signal, but the granularity is trajectory-level, not step-level.

More accurate closest precedent: Tree-GRPO (arXiv:2509.21240) is closer to the DPO-extraction mechanism. research/05 does not cite Tree-GRPO at all — this paper was published September 2025 and is directly relevant.

Finding 05-R2: Missing citation — Tree-GRPO (MISS)

research/05 does not cite Tree-GRPO (arXiv:2509.21240), which:

Proves intra-tree group-relative advantage == step-level DPO (exactly what extract_dpo_pairs is trying to compute informally).
Shows that shared-prefix branching is a principled way to generate process supervision from outcome reward alone.
Is published at ICLR 2026, directly relevant.

research/05 was presumably written before Tree-GRPO's acceptance was known, but this is now a material gap. The final report (§7, footnote [44]) cites Tree-GRPO correctly — so this was caught downstream but never propagated back to research/05.

Finding 05-R3: Missing citation — SWE-Search (MISS)

research/05 does not cite SWE-Search (arXiv:2410.20285), which applies MCTS specifically to SWE-bench tasks (the exact domain), achieves 23% relative improvement, and demonstrates that the value signal scales with inference-time compute. This is the most directly relevant published tree-search SWE result. The final report cites it as [51] — again, a gap that exists only in research/05.

Finding 05-R4: AgentTrek description is misleading (MISREAD)

research/05 describes AgentTrek as "Large-scale multimodal trajectory dataset from web tutorials. Guided replay demonstrations." The characterization "Guided replay demonstrations" is misleading — it implies similarity to multi-teacher replay. AgentTrek generates demonstrations by following web tutorial steps with a single agent; there is no replay of a frozen trace across multiple models. The "connection" note ("Demonstrates feasibility of guided/counterfactual replay") overstates the analogy.

Finding 05-R5: "Trace-Freezing + Multi-Teacher Replay: Novel" claim is directionally right but mis-attributed (PARTIAL OVERCLAIM)

Section "What IS Novel" states: "No published work systematically freezes a trace at step t, replays that exact state with N different teachers, harvests variance as per-step supervision." This specific combination IS genuinely novel. But the claim "Step-Level Multi-Teacher Preference Data: Traditional multi-teacher: response-level preferences; PRMs: single-teacher step evaluation; Gap: No multi-teacher per-step comparison" understates what Tree-GRPO achieves: Tree-GRPO generates multiple continuations from a shared prefix (a frozen state) and extracts step-level preference signal — which is mechanically very close to what Channel 3 does (though with a single model rather than heterogeneous teachers).

Finding 05-R6: Cost estimates are internally consistent in `research/05` but inconsistent with the final report (FLAG)

research/05 states the "Baseline" cost as:

$0.008/step × 1000 × 8 = $64 per trace

This is "8 teachers × 1000 steps" ungated. The final report's footnote [6] says "~$0.98 flat vs ~$64 ungated" — the $0.98 figure comes from Spike 001's actual measurement on 50 synthetic states with 3 teachers (reported in teacher_replay.py docstring: "Verified economic floor (✅ spike 001): $0.98 mean per-trace cost ungated, $0.30/trace projected with VOI gating.").

The inconsistency: research/05 constructs the $64 figure from 8 teachers × 1000 steps, while teacher_replay.py uses DEFAULT_TEACHERS with only 3 teachers. The final report uses $0.98 (3 teachers, real trace) and $64 (8 teachers, 1000 steps) in the same sentence, implying $64 is the cost of "the tree" not "flat Channel 3 with 8 teachers." This is confusing:

$0.98 per trace = real Spike 001 measurement, N=3 teachers, ~50 synthetic states
$64 per trace = constructed estimate from research/05, N=8 teachers, 1000 steps, flat (no branching)
A true branching tree at depth D with N branches is O(N^D) which is worse than $64

The final report says "a true branching tree is O(N^D), strictly worse than either flat figure" (§10), which is correct — but the $64 figure is NOT a "branching tree" cost, it is a flat all-steps N=8 estimate. Calling it "ungated tree" in the final report (§10: "~$64 ungated tree") is a misnomer. It should be "$64 flat 8-teacher 1000-step, vs O(N^D) for a true tree."

4. Critical Review of the Final Report (§§1,3,6,7,10 of `final_report_socratic-mcts-swe-worldmodel-8f6dea.md`)

Finding FR-R1: The "flat (depth-1)" characterization of Channel 3 is accurate (CORRECT)

§1: "It is flat (depth-1): the teachers never take their candidate action or continue, so the 'tree' is a set of depth-1 stars hung off a pre-existing linear human trace." This is exactly right — confirmed by reading replay_trace() (lines 178-188) and extract_dpo_pairs() (lines 206-262).

Finding FR-R2: "Fitness is teacher plurality, not execution" characterization is accurate (CORRECT)

§1: "its fitness is teacher plurality, not execution: selection is a Counter over normalized actions, nothing runs against a test suite." Confirmed by reading _normalize_action() and extract_dpo_pairs() — the pairing logic is purely textual counter-based with whitespace normalization. There is no execution oracle in Channel 3's current form.

Critical gap: The normalization in _normalize_action() (lines 196-203) is very weak — it only normalizes whitespace and lowercases. For real agentic traces with structured tool calls, this will produce near-zero agreement (two tool calls that are semantically identical but differently formatted will count as disagreements). The docstring acknowledges this: "For real agentic traces, this should parse the tool call (name + args) and return a canonical form." This is a known unimplemented critical path for the channel to produce any real signal.

Finding FR-R3: The Socratic-SWE description has a materially wrong claim (MISREAD, HIGH SEVERITY)

§1 describes Socratic-SWE: "Socratic-SWE (2606.07412) is the closest published analogue — a closed-loop self-evolving SWE trainer that distills traces into skills, generates targeted repair tasks, gates them through a four-stage execution validator, scores by solver-gradient alignment, and reaches 50.40% on SWE-bench Verified after three iterations. But it generates repair tasks; it does NOT inject bugs."

This is accurate as far as it goes. However, the final report then says: "Bug injection — manufacturing a broken state by reverting a gold patch and rewarding re-derivation — is the repo's own FeatureDeletionEnv, the analogue of Cursor's 'feature deletion'."

The implicit contrast — "they do repair tasks, we do feature deletion / bug injection" — is valid. But the report slightly overstates the contrast by not noting that Socratic-SWE's task generation pipeline is also execution-validated (Format/Grounding/Execution/Semantics gate) and also targets agent-specific weaknesses (via skill distillation). The gap is real (no bug injection, no per-step branching) but smaller than the report implies.

Bigger issue: The claim "it does NOT inject bugs" deserves a citation to the Socratic-SWE paper to confirm this is an architectural choice and not an oversight. Quote from the Socratic-SWE abstract: "Existing synthetic data methods typically create tasks through fixed mutation or bug-injection procedures, making the resulting distributions largely independent of the agent's own weaknesses and training progress. We introduce Socratic-SWE, a closed-loop self-evolution framework that reuses the agent's historical solving traces as a source of training signal." So Socratic-SWE explicitly positions against bug injection. The final report correctly notes this but doesn't quote it, which weakens the citation.

Finding FR-R4: "heterogeneous-per-turn branching + execution-oracle fitness + divergence-derived textual feedback + world-model aux loss — is the novel synthesis" claim is correctly hedged (CORRECT)

§1: "The multi-model per-turn tree is a recombination whose ingredients all exist (SWE-Search expands nodes with one policy [51]; Symphony does heterogeneous-LM planning [52]; Channel 3 queries N teachers, flat) but whose specific combination [...] is the novel synthesis. Claim the synthesis, not the parts." This is well-calibrated. The component-level citations are verified correct.

One gap: The paper arXiv:2407.01476 ("Tree Search for Language Model Agents," Koh et al. 2024) is in the vault and applies best-first tree search to realistic web agent tasks. This is another component-level precedent that should be in the synthesis inventory but is not cited in §1 or §7.

Finding FR-R5: The cost estimate claim "$0.98 mean per-trace cost ungated" needs qualification (POTENTIAL OVERCLAIM)

The $0.98 figure appears both in teacher_replay.py's docstring ("Verified economic floor (✅ spike 001): $0.98 mean per-trace cost ungated") and in the final report (§10, [6]).

However, Spike 001 used 50 hand-crafted synthetic states (ADR-002: "Spike 001 used 50 hand-crafted synthetic states for the cost-floor measurement"). Real Claude Code sessions have 125–2,830 tool_use messages per session (ADR-002). If a "trace" is a full session with ~1,400 states (median), the actual cost of flat N=3 replay would be roughly:

1,400 steps × 3 teachers × (prompt_tokens × cost_in + completion_tokens × cost_out)

The DEFAULT_TEACHERS pricing in teacher_replay.py at 200 max_tokens:

claude-opus-4.7: $15/M in + $75/M out → at 2k prompt tokens + 200 completion: ~$0.045/call
gpt-5: $1.25/M in + $10/M out → ~$0.004/call
deepseek-v4-pro: $1.10/M in + $4.40/M out → ~$0.003/call

Total per-step: $0.052 For 1,400 steps: **$73 per session** (not $0.98).

The $0.98 figure was correct for 50 synthetic states with short messages. At real session scale, the cost is one to two orders of magnitude higher. The docstring's "Verified economic floor" language suggests the $0.98 is the per-trace figure for the spike's trace definition (50 steps), not per full-session. The final report's §10 is careful about this distinction but teacher_replay.py's docstring is misleading: it should say "per 50-step trace (spike 001 benchmark)," not just "per-trace."

Finding FR-R6: The $64 "ungated tree" terminology is a misnomer (FLAG, see 05-R6 above)

§10: "flat Channel-3 replay is ~$0.98/trace at N=3 and ~$64/trace at the eight-teacher × thousand-step scale, both flat O(N·T); a true branching tree is O(N^D), strictly worse than either flat figure."

The statement is mathematically correct but the label "~$64 ungated tree" (used in the final report's in-text references) is misleading because $64 is a flat (non-branching) extrapolation from research/05. It is not a measured figure, and the "eight teachers" scenario (N=8) does not correspond to any actual configuration in the codebase (DEFAULT_TEACHERS has N=3). The label should be "$64 flat-replay extrapolation at N=8, T=1000, unmeasured."

Finding FR-R7: The heterogeneity pushback is correctly represented (CORRECT)

§7 cites arXiv:2604.02460 (Tran & Kiela 2026) and arXiv:2601.12307 (Rethinking multi-agent) correctly. Both papers are in the vault with correct titles/years. The data-processing-inequality framing for single-agent efficiency is correctly attributed.

One gap: The final report does not note that arXiv:2604.02460 studies multi-hop reasoning tasks (QA), not SWE tasks. The domain transfer is uncertain. This is the kind of caveat that belongs in §7's pushback discussion.

Finding FR-R8: The _normalize_action issue is a silent failure mode not surfaced in the final report (MISS)

The final report discusses the $0.98→$64 cost gap and the O(N^D) blowup, but does not flag that extract_dpo_pairs will produce near-zero pairs on real agentic traces because _normalize_action only normalizes whitespace. On real Claude Code traces with structured tool_use blocks (JSON-formatted), two semantically identical "run bash command X" steps formatted slightly differently will count as disagreements, and two structurally different calls with identical normalized strings will count as agreements. The pair-extraction logic is likely to produce mostly noise on real traces without a proper tool-call parser.

This is acknowledged in the code comment ("For real agentic traces, this should parse the tool call (name + args) and return a canonical form") but is not surfaced as a risk in the final report's §10 failure modes section.

Finding FR-R9: The world-model aux loss discussion correctly cites the key interference and causal-gap papers (CORRECT)

§2 cites arXiv:2602.00994 (Reasoning and Tool-use Compete in Agentic RL) and arXiv:2605.05029 (The Predictive-Causal Gap) correctly. Both are in the vault with correct titles. The conclusion — parameter-isolated head, ablation-gated — is appropriate given the evidence.

Finding FR-R10: The "strip_thinking=False" critical path claim is not verified by any cited paper (UNVERIFIED)

§6: "strip_thinking must be False: ~67% of real Claude Code error-recovery turns are pure thinking." The 67% figure appears to come from internal repo analysis (ADR-002 / claude_code.py) rather than any external source. This is likely accurate (it's an empirical measurement from real sessions) but should be marked as an internal finding, not stated as if it has external validation.

5. Is the Channel-3/Tree Design Novel or Reinventing Something That Exists?

Novel (good):

Frozen-trace per-step replay across N heterogeneous frontier models with consensus-DPO harvesting: No published paper does exactly this. The closest are Tree-GRPO (shared-prefix same-model branching) and rStar (two-model trajectory verification). The heterogeneous-teacher-consensus mechanism applied to a frozen captured trace is original.
Combining execution-oracle fitness with multi-model disagreement for divergence-gated branching: SWE-Search uses LM-estimated value (not execution oracle). Socratic-SWE uses execution validation as a filter, not as the MCTS fitness. Using test-suite pass-fraction as the tree's value function is the most meaningful differentiator.
Two-loop architecture (outer MCTS/datagen + inner RL trainer) with the world-model head as the bridge: The specific coupling where failed branches become world-model training targets (not policy gradient targets) is a design synthesis not found in any single paper.

Reinventing (needs citation):

MCTS over real code repositories: SWE-Search (arXiv:2410.20285) does exactly this and is 18 months old. The repo cites it correctly as [51] in the final report but research/05 doesn't mention it. The "23% relative improvement" from SWE-Search should inform the expected gain from adding search (whether at training or inference time).
Tree-structured DPO signal from shared prefix branches: Tree-GRPO (arXiv:2509.21240) formalizes exactly this. The repo's extract_dpo_pairs is a weaker/informal version. The repo should acknowledge that it is implementing a version of what Tree-GRPO proves, and should consider whether the formal Tree-GRPO objective would be stronger than the current Counter-based heuristic.
Per-step process supervision from MCTS: rStar-Math (arXiv:2501.04519) and Math-Shepherd do this for math. The adaptation to code/SWE is novel, but the core mechanism is not. research/05 does acknowledge this in the PRM section, but understates how directly rStar-Math applies (it performs MCTS rollouts to generate step-verified training data, which is structurally similar to what the tree-of-work aims to do).
World-model head trained on agent trajectories: CWM (arXiv:2510.02387) trains a 32B model on observation-action trajectories for next-state prediction and achieves 65.8% on SWE-bench Verified. This is directly cited in the final report as [13] and correctly described. No reinvention issue.

6. Cost Estimate Plausibility

Estimate	Source	Plausibility	Issues
$0.98/trace ungated	Spike 001 measurement, 50 synthetic states, N=3	Accurate for 50-step traces. Misleading at full-session scale (~1400 steps → ~$73 with same model prices).	`teacher_replay.py` docstring should specify the scope.
$64/trace flat	`research/05` constructed estimate, N=8 teachers, T=1000 steps	Plausible as an extrapolation for a specific 8-teacher 1000-step scenario. Not measured, not a codebase config.	Calling it "ungated tree" is a misnomer. DEFAULT_TEACHERS has N=3, not N=8.
O(N·decision-points) with divergence gating	Final report §3, citing research/05	Plausible as a theoretical claim if gating is very aggressive (VOI/entropy > τ fires only at a small fraction of steps). The 60-80% reduction estimate from "PRM literature" is from `research/05`'s analysis, not a specific paper citation.	No empirical measurement of what fraction of real agentic trace steps are "high-uncertainty." The claim needs a specific PRM citation.
$0.30/trace with VOI gating	`teacher_replay.py` docstring, "projected"	Speculative projection from $0.98 × ~0.3 reduction. Not measured.	The 70% reduction rate should cite a specific paper or be labeled "estimated."

Overall cost plausibility verdict: The $0.98 and $64 figures are internally consistent as upper/lower bounds for specific parameterizations, but neither reflects the actual cost of running Channel 3 on real full-session Claude Code traces (which would be far higher). The divergence-gating reduction estimate (60-80%) is reasonable but unverified for agentic traces specifically. The final report correctly identifies the O(N^D) combinatorial blowup as the dominant cost concern for a true branching tree — this framing is sound.

7. Summary of Findings

ID	Severity	Type	Location	Finding
05-R1	Medium	MISREAD	research/05	rStar is trajectory-level verdict, not per-step replay — overclaims closeness
05-R2	High	MISS	research/05	Tree-GRPO (arXiv:2509.21240) not cited; directly formalizes the DPO-from-branching mechanism
05-R3	High	MISS	research/05	SWE-Search (arXiv:2410.20285) not cited; most direct execution-oracle-adjacent MCTS-on-SWE precedent
05-R4	Low	MISREAD	research/05	AgentTrek "guided/counterfactual replay" connection overstated
05-R5	Low	PARTIAL OVERCLAIM	research/05	Tree-GRPO's shared-prefix branching is closer to the step-level preference extraction than rStar
05-R6	Medium	INCONSISTENCY	research/05 + teacher_replay.py	$64 "8-teacher flat" extrapolation mislabeled as "ungated tree"; N=8 not a codebase config
FR-R1	—	CORRECT	final_report §1	Depth-1 / flat characterization of Channel 3 is accurate
FR-R2	—	CORRECT	final_report §1	Teacher-plurality vs execution-oracle distinction is accurate
FR-R3	Medium	MISREAD (mild)	final_report §1	Socratic-SWE contrast slightly overstated; execution-gating detail understated
FR-R4	—	CORRECT	final_report §1	Novel synthesis vs ingredient-level precedents is well-calibrated
FR-R5	High	POTENTIAL OVERCLAIM	final_report §10 + teacher_replay.py	$0.98/trace docstring applies to 50-step synthetic traces; at real full-session scale (~1400 steps), cost would be ~$73/session
FR-R6	Medium	FLAG	final_report §10	"$64 ungated tree" label is misleading; it is a flat extrapolation with N=8, T=1000
FR-R7	—	CORRECT	final_report §7	Heterogeneity pushback cited correctly
FR-R8	High	MISS	final_report §10	`_normalize_action` whitespace-only normalization is a critical silent failure path on real agentic traces; not in §10 failure modes
FR-R9	—	CORRECT	final_report §2	World-model interference and causal-gap papers cited correctly
FR-R10	Low	UNVERIFIED	final_report §6	67% of error-recovery turns are pure thinking: internal empirical claim, should not be stated as externally validated

8. Actionable Recommendations

Fix _normalize_action before any real-trace replay: The whitespace-only normalization is the most likely silent failure path. Implement tool-call parsing that canonicalizes (tool_name, args) tuples. This is acknowledged in the code comment but not in any ADR or the final report's failure mode list. Add to ADR-002 consequences or open a new ADR.
Correct teacher_replay.py docstring: Change "Verified economic floor (✅ spike 001): $0.98 mean per-trace cost ungated" to "Verified economic floor for 50-step synthetic-state benchmark traces (Spike 001): $0.98 mean per-trace. Full Claude Code sessions (~1,400 tool-use steps) would cost ~$70–80 flat ungated at DEFAULT_TEACHERS pricing."
Cite Tree-GRPO in research/05 and consider adopting its formal objective in extract_dpo_pairs rather than the Counter heuristic. The Tree-GRPO advantage estimator (intra-tree group-relative) is a stronger principled basis for the DPO pair extraction.
Add SWE-Search to research/05 and note that its 23% relative gain represents the expected floor for adding any tree search to a SWE agent. The tree-of-work must beat this at equal compute on a per-training-dollar basis, not just at inference time.
Standardize the $64 figure's label: In the final report's in-text citations and in research/05, consistently label it "flat N=8 T=1000 extrapolation (not measured, not a codebase config)" rather than "ungated tree."
Add the _normalize_action failure mode to §10's failure modes list in any future revision of the final report.
The tree-of-work core claim stands: The combination of execution-oracle fitness + frozen-trace multi-teacher replay + divergence-gated recursive branching + typed signal routing (world-model head for failed branches, contrastive DPO for near-misses) is a genuine novel synthesis. The final report's pre-registered ablation design (P0–P6 axis + heterogeneity control) is the right scientific vehicle — do not abandon it based on the misreads identified here, which are correction-level, not foundation-level problems.