think-decide/REPORT.md

Can diffusion steps replace token thinking in a decision model?

Think-then-Decide, 6 November 2026. One A100 80GB, ~8 hours. Code, logs, predictions and checkpoints are in the bucket Mercity/Desicion-Model-Custom under think-decide/. Lab notebook: NOTEBOOK.md; every change to the plan: DEVIATIONS.md.

TL;DR

• No latent "thinking" design worked. Six designs that iterate on latent vectors were tried, from the original plan's thought slots to a separate latent reasoner read by a decision head. All six score the same with 32 steps as with 1 step (gains between -0.04 and +0.24 points, every 95% CI includes 0).
• The one design that gains from extra passes is answer-level diffusion (MQ). All of a task's sub-questions go on one canvas, and the model commits its most confident answer each pass, so earlier answers help later ones. Result: +0.48 points [+0.21, +0.76] on the same questions, rising to +1.5 on the questions answered last. The ceiling is +1.75 (all other answers given), so MQ captures about 27% of it.
• The gains are small next to token thinking. In the bucket's previous project, the 4B model gained about 48 points on the same procedural tasks by writing reasoning tokens. Here, iteration buys about half a point.
• Diagnosis. Iteration only helps when the positions being refined are supervised (answers or sub-answers). Unsupervised latent vectors get ignored, whether they sit inside the backbone, in a separate latent reasoner, or behind a gate. Both literature reviews reached the same conclusion.

Figure 1. Accuracy gain from extra passes vs one pass, same questions, paired bootstrap over items (95% CI). The two Core rows use Core's reasoning suites; all other rows use the procedural testbed. *MQ-soft's gain is relative to its own one-pass accuracy, which is 1.5 points below MQ's; in absolute terms MQ is better (see section 3.3).

1. Question and setup

The user's question: can the token-by-token thinking of an LLM be replaced by diffusion steps? The test was a small diffusion decision model doing a few extra denoising passes "in its head" before deciding, compared with deciding in one pass.

• Backbone. S1 (SEU-ZZH/Shared-YesNo-Qwen3-0.6B-S1, from jev-dllm) is Qwen3-0.6B converted to a masked diffusion LM with bidirectional attention, fine-tuned to make decisions in one pass. Each option gets a [MASK]; its score is logit(Yes) - logit(No) at that mask, followed by a softmax over the options.
• Benchmarks.
  • Core: jev-dllm's 17-suite benchmark, 6,744 decisions. All items were scored; S1's own protocol drops 221 long ContractNLI items.
  • Procedural testbed: tasksource/procedural-typed-decisions, 18 multi-step task families × 5 difficulty levels. Not in S1's training data. Token thinking helped the previous project's 4B model by about 48 points on this distribution.
• Statistics. Every gain is measured within one model (K passes vs 1 pass) on the same items, with a paired bootstrap (10,000 resamples) stratified by suite or task family.

2. Everything that was run

ID	What	Data	Result
E0	Harness parity: jev-dllm's evaluator and ours, on S1	Core	reproduces published S1; group scores equal
E1	S1 + decision head, 1 epoch (control)	S1's 40k train	Core reasoning 64.99 (S1 64.07)
E2	Plan: 32 thought slots, gated latent loop, K in {1,2,4,8}	40k	flat (+0.08 [-0.27, +0.43]); gate stuck at 0.002
E3	E2 + CODI-style distillation from S1 reading notes	40k + 5.4k teacher targets	flat (-0.04 [-0.58, +0.59])
P0	S1 zero-shot	procedural	42.4%
P1	S1 + head, 1 question per state, K=1	32k procedural	88.7%
P2o	E2's loop, gate opened at 0.014	32k	killed at step 520: 4 passes = 1 pass on dev (83.97 = 83.97)
P2s	Soft-token feedback into the slots, no gate	32k	killed at step 310: 4 passes = 1 pass on dev (71.80 = 71.80)
MQ	All sub-questions on one canvas; diffusion over answers	32k states (105k questions)	sequential +0.48 [+0.21, +0.76]; 1 pass 90.7 on target questions
MQ-soft	MQ + soft-belief self-conditioning	32k states	multi-pass +0.8 over its own 1 pass, but below MQ overall
LDR A-D	Separate latent reasoner (32 vectors, K steps) read by a decision head	32k procedural	flat in all four variants

Not run, by decision: E5 (notes written as tokens by diffusion; dropped at the user's request), general-mix E2o/E2s (superseded), E4 latency (not measured), and the full-data MQ run (the user called time).

3. Results

3.1 The pre-registered plan on Core (E0-E3)

Model	K	Reasoning	Lookup	Other	Core17	Core14
S1	1	64.07	73.96	61.49	66.07	64.05
E1 (head, no thinking)	1	64.99	74.10	60.50	66.08	64.15
E2 (latent loop)	1 / 8	64.67 / 64.76	74.45 / 74.35	59.82 / 59.94	65.83 / 65.88	63.93 / 63.97
E3 (+ distillation)	1 / 8	64.51 / 64.47	75.31 / 75.34	59.56 / 59.63	65.94 / 65.96	64.02 / 64.01

Core17 is the equal-weight mean accuracy over all 17 suites. Core14 drops three suites that S1 saw in training.

Pre-registered hypothesis	Test	Result [95% CI]	Verdict
H1: accuracy rises with K	E2, K=8 vs K=1, reasoning	+0.08 [-0.27, +0.43]	not supported
H2: the gain comes from thinking, not the head or extra epoch	E2 (best K) vs E1	-0.31 [-1.51, +0.91]	not supported
H3: a teacher target helps	E3 (best K) vs E2 (best K)	-0.59 [-1.84, +0.60]	not supported

Why it failed: the gate deadlocks. The feedback is x = e_mask + g·W·LN(h), with g starting at 0.

• W's gradient is proportional to g, so W cannot learn while the gate is shut.
• g's gradient points along W·LN(h): a last-layer state pushed into input-embedding space, which is meaningless until W has learned something.

So neither can move first. The gate ended at 0.002 in E2 and 0.001 in E3, while about 0.014 is needed for the feedback to match the mask embedding's size. Opening the gate (P2o) or removing it (P2s) did not help either: the model left the channel unused.

Why E3's teacher could not help. Frozen S1 barely uses notes, even notes written by Qwen3.5-9B with the gold answer in view: 93.6% accuracy with a note vs 93.7% without, over 9,370 training items. E3's distillation target was therefore close to S1's own states.

3.2 The procedural testbed (P0-P2)

Model	All	L0 (easy)	L1	L2	L3	L4 (hard)
P0: S1 zero-shot	42.37	53.25	47.52	40.65	36.21	34.21
P1: trained, 1 question, 1 pass	88.68	96.28	91.93	87.94	84.01	83.20

Levels are difficulty: level 0 is short and few-step, level 4 long and many-step. One epoch of training on 32k items (9% of the dataset) takes S1 from 42% to 89%. The errors left over are concentrated at levels 3-4.

3.3 Answer diffusion (MQ): the only positive result

Design. Every question of a state goes on one canvas, in S1's own format, each with its answer masks:

State: ...
Question 1: Which policy governs?   P1: [MASK]   P2: [MASK]
Question 2: Is access allowed?      Answer: [MASK]
Question 3: How risky is it?        level 0: [MASK]  level 1: [MASK]  level 2: [MASK]

• Training is masked diffusion over answers: each question's gold Yes/No tokens are revealed with probability 1-t (t ~ U(0,1)), and the loss is computed on the hidden ones.
• Inference has three modes, all on the same model and questions:
  • parallel: all answers in 1 pass;
  • sequential: each pass commits the most confident open question, then re-reads;
  • oracle: all other answers are given correctly.
• There are no new parameters and no gate. What carries information between steps is committed answer tokens, which are supervised.

MQ readout (11,582 test questions)	All	Target q	L3	L4	vs parallel [95% CI]
parallel (1 pass)	88.45	90.67	83.07	82.75	-
sequential (1 answer per pass)	88.98	91.10	83.82	83.25	+0.48 [+0.21, +0.76]
oracle (others given)	90.84	92.28	87.29	86.81	+1.75 [+1.45, +2.06]

• The gain grows with commit order. Questions answered later have more committed answers to use. Against parallel they gain +0.6 at rank 1, +0.8 at rank 2 and +1.5 at rank 3.
• The clearest gains come from families whose questions chain explicitly. Policy applicability goes 97.5 to 99.0 and belief tracking 98.8 to 100.0. The largest untapped headroom is record eligibility: 78.1 sequential vs 87.5 oracle.
• Training on all the questions helps one-pass accuracy too. On the same target questions MQ scores 90.67 in one pass vs P1's 88.68.

Does accuracy grow with the number of steps? Yes, up to about 3, then it plateaus. These are the same questions with only the step budget changed; each pass commits ceil(open/passes left) answers.

MQ-soft (continuous diffusion). Nothing is committed: each pass feeds back soft Yes/No beliefs as blended embeddings.

• Continuous soft passes gain about as much as discrete commits, relative to the same model's one-pass score: +0.81 [+0.50, +1.14] for soft passes vs +0.80 for commits.
• But soft self-conditioning training lowered one-pass accuracy by 1.5 points. MQ-soft's best result (87.86 on all questions) therefore stays below plain MQ's (88.98).
• My automatic rule for the large run compared gains instead of absolute accuracy, so it picked MQ-soft. That run was stopped once this was noticed.

3.4 Latent-vector reasoners (LDR): the user's target architecture

Design. The backbone runs once and its token states are cached. A small weight-tied reasoner (38M parameters: self-attention over 32 latents and the answer-belief slots, cross-attention to the cached states, MLP) refines the latents for K steps. A decision head then reads them and adds a correction to the option states before the Yes/No readout.

• Zero-initialised output, so K=0 equals the encoder alone.
• Random depth K ~ U{1..12}, truncated backprop through the last 3 steps, EMA weights.
• Noise in the initial latents, averaged over 4 seeds at evaluation.
• Recipe from TRM, HRM and Huginn, per the third literature review.

Variant	K=0	K=1	K=8	K=32	K=32 - K=0 [95% CI]
A: frozen P1 encoder	88.68	88.71	88.71	88.71	+0.12 [-0.26, +0.55]
B: frozen MQ encoder, all sub-questions	88.03	88.00	87.98	87.98	-0.01 [-0.19, +0.18]
C: encoder trained jointly (from S1)	85.05	84.96	85.05	85.05	0.00 [-0.39, +0.39]
D: frozen raw S1 (huge headroom)	42.43	42.62	42.49	42.46	+0.24 [-0.27, +0.80]

D is the telling case. With 46 points of headroom on the table, the latent reasoner learned essentially nothing in 550 steps: its gradient norm stayed around 0.01-0.3 and the loss did not fall. In C, joint training also left the encoder worse than P1 (85.1 vs 88.7). At this model size and training budget (500 steps per run), latent iteration did not learn any useful computation.

4. Diagnosis: why latent thinking did not work here

1. Unsupervised latents get ignored. This held in every placement tried (inside the backbone, behind a gate, without a gate, in a separate reasoner). The literature agrees: Coconut fails without its curriculum, filler tokens need dense supervision, latents collapse in SIM-CoT, and He, Welleck and Fried (2602.03769) find that extra unsupervised positions hurt.
2. One pass already solves most of each decision. P1 gets 89% in one pass. The remaining errors need information that a re-read of the same states cannot create; the supervised sub-answers can (MQ's oracle, +1.75).
3. Retrofitting recurrence takes far more training than we had. Published retrofits use billions of tokens (McLeish et al. 2511.07384: ~50B; ETD: 50B; LoopUS: 3B). Each run here saw ~10-30M. Loopholing reports that fine-tuning-only retrofits failed.
4. The feedback space matters. Last-layer states fed back into the input deadlock under a zero gate. Token-space feedback (soft beliefs, committed answers) works immediately; it is what MQ and MQ-soft use.

5. Cost of the one thing that worked

• MQ's sequential mode costs one backbone pass per committed question: n passes for n questions (3.25 on average), because a bidirectional model cannot cache earlier work. On a 700-token state with 3 questions, that is about 2,100 token-passes of a 0.6B model in 3 sequential rounds.
• Token thinking in the previous project's 4B model wrote about 700-1,000 reasoning tokens per question, in as many sequential decode rounds of a model about 7x larger.
• So MQ is far cheaper per decision but buys about +0.5 points, against about +48 points for token thinking on the same task family.
• The latent reasoners' steps are cheap (32 latents through two small blocks), but they bought nothing.
• Wall-clock latency was not measured (E4 not run).

6. Caveats

• Single seed per arm. Uncertainty comes from bootstrapping over test items only.
• The plan's decision head was nearly inert. It added its correction to the pre-norm option state, which has norm ~1,700 vs ~46 after the norm. E1-E3's differences from S1 therefore come from the extra backbone epoch, not the head. Found at 20:00; the latent reasoners use a post-norm head.
• Harness parity. Group scores match jev-dllm's evaluator exactly; 1.4% of argmaxes still differ (bf16 numerics), the same for every arm.
• P2o and P2s were stopped early (steps 520 and 310 of 1,000), with dev K=4 = K=1 at every check.
• The large MQ run was never completed. A 216k-state MQ-soft run was stopped at 18% (wrong pick rule, section 3.3); the full-data MQ run was not started.
• E3's notes were generated with the gold answer visible, then leak-filtered (93.7% kept). Frozen S1 could not use them, which limits what E3 could show.

7. What I would do next

1. Scale MQ to the full 360k states (about 4 h on one A100). This answers whether the sequential gain grows (better use of committed answers) or shrinks (one-pass accuracy catches up). Train-time noise on the answer channel and loss-weighting hard questions (MGDM-style) are cheap additions.
2. Give the latent reasoner a fair budget before ruling it out. Cache the frozen encoder states once, then train the reasoner for 30+ epochs at a higher learning rate. That takes seconds per epoch instead of minutes. If K still does not matter, call latent iteration a no-go at 0.6B.
3. Supervise the latents. Decode each latent step to a sub-answer or a short reasoning step (SIM-CoT, TRM's y channel). This is where all of the positive evidence points.

8. Engineering notes

• Speed. torch.compile on the decoder layers gave 1.6x. A padding-aware micro-batch split cut padded compute from 1.3-3.0x to 1.17x of real tokens. fp32 master weights with bf16 autocast (S1's own trainer kept pure bf16 weights, which loses small updates).
• Memory. Training peaked at 74.9 GiB of 80 (94%), with an out-of-memory fallback that restores gradients and retries with checkpointing; 0 retries were needed.
• No truncation anywhere. Every Core item was scored. Procedural training used max length 8,192, so no item was dropped; a 2,048 cap would have dropped 2.5%, nearly all level 3-4. Note generation flagged capped outputs (none occurred) rather than cutting them.
• Data. Core 6,744 items. S1's training set: 40k. Procedural: 78.5k prepared (32k used per arm), 1,751 dev, 3,484 test states (11,582 questions). MQ used the 32k states' 105k questions; the dataset has 360k training states.

9. Files

• src/: common.py (renderer, loaders), model.py (ThinkDecide), train.py, eval.py, distill.py, teacher_cache.py, gen_notes.py, prep_procedural.py, mq.py (answer diffusion), ldr.py / ldr2.py (latent reasoners), bootstrap.py, analyze*.py
• runs/<arm>/: config, training log, predictions, report, weights. bf16 checkpoints: e1, e2, e3, p1, mq, mqsoft; ldr2_c/encoder.safetensors and the reasoner_ema.pt files.
• results/: results.md (Core), results_proc.md, results_mq.md, summary_stats.json, figures
• logs/: every training and eval log; NOTEBOOK.md, DEVIATIONS.md, TODO.md