Buckets:

Mercity
/

Desicion-Model-Custom

Files

xet

Mercity/Desicion-Model-Custom / think-decide /NOTEBOOK.md

Pranav2748

2 days ago

preview code

download

raw

12.9 kB

# Lab notebook: think-then-decide (latent denoising passes as thinking)

Times UTC. Observations as they happened; numbers from the files named.

## 2026-11-06

**12:54** Box: A100-SXM4 80GB (RunPod), 255 CPUs, 1 TB RAM. Bucket `Mercity/Desicion-Model-Custom` (previous
project: Qwen3.5-4B decision model with token thinking) reviewed; its 76.5 GB of checkpoints deleted locally (they
stay in the bucket) to keep disk free.

**13:00** jev-dllm cloned (3b8c01b), S1 downloaded at the pinned revision. S1's model code reviewed (27-line
bidirectional Qwen3 override; `last_hidden_state` is post final-norm; accepts `inputs_embeds`). Training data:
40,000 train / 1,000 dev, lengths with 32 thought slots median 246, p99 846, max 1,395 tokens (none over 2,048).
Core: 6,744 decisions in 17 suites; 221 ContractNLI items exceed 4,096 tokens (S1's protocol drops them).

**13:03 E0 step 1** (`runs/e0/core-s1-theirs`): jev-dllm's own evaluator on S1, this GPU, bf16, batch 32:
Jabr 65.94, MASSIVE en-US 77.87, Civil Comments 82.67 = published; JevBench original 84.72 (published 86.11,
one item), VitaminC 67.61 (67.95), PubMedQA 57.20 (57.60): bf16 cross-GPU flips. 118 s of forward time.

**13:20 E0 step 2** (`runs/e0/core-s1-ours*`): our harness, same 6,523 items: suites within 1-2 items of
jev-dllm's run (JevBench original 86.11 = published), reasoning group 63.96 in both. 1.4% argmax flips remain
with jev-dllm's exact batches, max |dp| 0.13 (DEVIATIONS #3).

**13:25** Speed: profile showed a memory-bound small model (GEMMs ~25% of GPU time; unfused RMSNorm/RoPE/SwiGLU/
casts and attention backward dominate). torch.compile on the decoder layers: 1.6x (22.8k token-passes/s at K=4).
Checkpointing: 1.5-2x slower. Padding: K=1 micro-batches were ~3x padded; a padding-aware split brings every K to
~1.17x (adopted after E1; E1 ran with the budget-only split, same maths).

**13:33** Is the training set memorised by S1 (3 epochs on it)? No: on 5,000 train items KL 0.18, acc 93.5%,
23% with p(gold) < 0.9; held-out new_dev KL 0.31, acc 88%. Enough signal; plan's data kept.
**13:40 E1** started (S1 + head, no thinking, 1 epoch). Dev at step 250 (peak LR): KL 0.260 -> 0.327, acc 90.3 ->
87.6%. S1's released checkpoint was selected on this dev set (min KL), so dev is biased toward S1; Core decides.
**13:55 E1 done** (`runs/e1`): 1,250 steps, 14.4 min, peak 61.4 GiB (budget-only split; K=1 steps were one
~3x-padded micro-batch), no OOM retries. Dev: KL 0.260 -> 0.303, acc 90.3 -> 91.5%. Core (all 6,744):
reasoning 64.99, lookup 74.10, other 60.50, Core17 macro 66.08. (S1 at 4,096 protocol: 63.96 / 73.96.)
**14:02 E2 started** (32 slots, K ~ U{1,2,4,8} per step, budget 42,000): peak 74.8 GiB (94%), 22.5k
token-passes/s, ~1.4 s/step. Gate stays at |g| ~ 1e-4 (needs ~0.014 for the feedback to match the mask
embedding's norm 0.46); W drifts from identity by Adam noise (||W - I|| 1.28 at step 380; its gradient is
proportional to g ~ 0, but Adam rescales it to lr-sized steps). Dev at 250: K1 89.9 / K4 89.8%.
**14:16** Notes pilot failed: vLLM 0.33 needs `ninja` on PATH (installed in its venv). Saved weights were
2x size (ThinkDecide registers base.model twice as `bb`); fixed, E1 file deduplicated (310 keys, identical).
First bucket sync: `think-decide/` (token scan before every sync; .env never in the tree).
**14:38 E2 done** (`runs/e2`): 1,250 steps, 29.4 min, peak 74.9 GiB (94%), 0 OOM retries. Gate 0 -> 0.0019 (needs
~0.014 to matter); dev K1 = K4 = 91.37%. Core (all 6,744; K=8 eval took 27 min): reasoning 64.67 / 64.65 / 64.82 /
64.76 at K = 1 / 2 / 4 / 8; lookup 74.45 -> 74.35. Paired bootstrap: K8 - K1 reasoning +0.08 [-0.27, +0.43];
E2@1 - E1 reasoning -0.31 [-1.51, +0.87]. **H1 and H2 not supported for E2.**
Diagnosis: the feedback x = e_mask + g W LN(h) deadlocks at g = 0. h is a last-layer state (not input-embedding
space), so W must learn a mapping before feedback helps; W's gradient is proportional to g, and g's gradient points
along W LN(h), which is meaningless until W has learned. Under Adam both move by noise (W drifted ||W - I|| ~1.3
by step 380). So E2's null speaks to this initialisation, not to latent thinking as such.
S1's predictions at the Notes slots are generic (entropy ~5.6 nats/slot, greedy "The1 is is the the,,,,"), so a
soft-token feedback would start nearly content-free too.

**14:50** Exploratory arms added (labelled post hoc, after E2's diagnosis; one change each vs E2):
E2o = gate init 0.014 (feedback norm = mask-embedding norm, so W gets gradient from step 0);
E2s = soft-token self-conditioning (slot input = E^T softmax(own token logits), mask excluded; gate fixed at 1):
the continuous-diffusion version, no learned mapping needed.

**15:05 Notes** (`runs/e3/notes.jsonl`, Qwen3.5-9B, greedy): the first pilot showed concluding bullets that name
the answer ("The highest milestone reached is Release") and world-knowledge notes contradicting labels; prompt
tightened (facts/rules only, no conclusion, never name an option) and the leak filter extended to the gold
option's short name and stem. 10,000 notes, 9,370 pass (dbpedia 271 and clinc 112 name the gold class); median
76 tokens, max 175 (cap 256: none capped).

**15:10 Teacher** (frozen S1 reads the note; keep a note only if it lowers CE to the target): placement probe on
1,500 notes (first in file order: arc, clinc, sgd, snli only): after-options keep 70.7% vs before 64.8% ->
"after" (also position-aligned with the student's slots). Full cache: 5,409 / 9,370 kept (57.7%). Teacher
accuracy with vs without note: 93.6 vs 93.7% (ARC 96.9 vs 91.8, BoolQ 89.9 vs 88.8, GoEmotions 65.0 vs 68.2).
Frozen S1 barely uses notes, so E3's target sits close to S1's own states.

**15:12 S1 all 6,744 items** (`runs/s1`): reasoning 64.07, lookup 73.96, other 61.49, Core17 66.07.
E1 - S1: reasoning +0.92.

**15:52 E3 trained** (`runs/e3`; 5,409 items with teacher targets): 29.6 min, peak 74.9 GiB, 0 OOM retries. Gate
0 -> 0.00097 (smaller than E2's 0.0019): the distillation terms did not open the channel. Dev K1 91.47 / K4 91.26%.
In a direct test at init the distill losses gave the gate a gradient of 1.45 at K=2, but the target is close to
S1's own states (frozen S1 barely uses notes), so the consistent signal stayed small.

**16:00 Re-think (user asked for something that would actually show diffusion can replace thinking).** On S1's
data even gold-aware notes do not help frozen S1 (93.6 vs 93.7%), so these benchmarks have little headroom for any
thinking. A convincing test needs a task where token thinking is known to help a lot: the previous project's
procedural decisions (4B: 46.8% readout vs 95.1% after thinking). New testbed E6 = tasksource/procedural-typed-
decisions (Apache-2.0; 18 families x 5 levels; not in S1's training data): 78,548 train states (one question
each, stratified), 1,751 dev (validation split), 3,484 test (test split). Lengths: median 550 tokens (level 0:
330, level 4: ~1,000), max 6,334 -> train max length 8,192 (no drops; a 2,048 cap would have dropped 2.5%, nearly
all level 3-4). Arms (32k items, 1 epoch, backbone lr 2.5e-5 = S1's own, the same for all): P1 control (head,
K=1), P2o (plan's loop, gate opened at 0.014), P2s (soft-token feedback). Evals compiled (all procedural arms).
**P0: S1 zero-shot** test 42.37% (L0 53.3 -> L4 34.2), dev 41.73%.

**16:10** Procedural CoT notes for E5 (token thinking by diffusion) started beside E3's eval: only 5 prompts/s while
sharing the GPU (~1.5 h for 32k) and holding 40 GB that P1 needs; stopped (nothing written). Will regenerate for
8,000 items when the GPU has room.

**16:20 E3 Core** (all 6,744): reasoning 64.51 / 64.08 / 64.39 / 64.47 at K = 1/2/4/8. Pre-registered verdicts
(results/results.md): H1 not met (E2 K8-K1 +0.08 [-0.27, +0.43]; E3 -0.04 [-0.58, +0.59]), H2 not supported
(E2 best - E1 -0.31 [-1.51, +0.91]), H3 not supported (-0.59 [-1.84, +0.60]). Only significant difference: E3 lookup
+1.17 [+0.31, +2.04] over E1. One more epoch (E1/E2/E3 alike) trades suites: Civil Comments +6-7, ContractNLI +2-4,
PubMedQA -5.6 to -7.

**16:45 P1** (procedural control, 32k items, K=1): 1,000 steps, 18 min. Test 88.68% (L0 96.3, L1 91.9, L2 87.9,
L3 84.0, L4 83.2); dev 89.08%.

**17:05 Two research agents** (literature, independent briefs) converged: (1) latent thinking positions need dense
supervision (Coconut without curriculum, filler tokens, SIM-CoT collapse, He/Welleck/Fried 2602.03769: extra
unsupervised positions hurt); (2) feedback should be in a space the model understands (soft token mixtures, not
last-layer states; LAW: hidden-state feedback < 1.5%); (3) two-pass stop-gradient self-conditioning beats backprop
through all passes (Loopholing, LAW), and Loopholing reports fine-tuning-only retrofit failed; (4) noise during
training is a curriculum (clean-only training: 0% in 2609.01449); (5) the procedural states carry 3.25 questions
each, often intermediate steps of one another (governing_policy -> access_allowed -> review_risk; world_location ->
belief -> belief_matches_world; shortest_cost -> edge_essential), which my prep had dropped.

**17:35 P2o killed at step ~520** (user's call; plan's loop, gate opened at 0.014): dev K1 = K4 at step 250 (72.36%)
and 500 (83.97%); gate drifted 0.0140 -> 0.0139. The open channel was ignored.

**17:40 MQ (answer diffusion)**: all questions of a state on one canvas (S1's block format); masked diffusion over
answers (reveal each question's gold Yes/No tokens with prob 1-t, t ~ U(0,1), loss on masked questions); readout
= S1's native Yes/No, no new parameters, no gate. Eval modes on all questions of the 3,484 test states: parallel
(1 pass), sequential (commit the most confident question per pass), oracle (others revealed). MQ-soft adds two-pass
stop-gradient self-conditioning with soft Yes/No beliefs (p=0.8) and an eval of K soft passes.

**18:00 MQ results** (`runs/mq`, 32k train states = 105,286 questions, 1 epoch, 21 min; test = all 11,582 questions of
the 3,484 test states). Parallel 88.45 / sequential 88.98 / oracle 90.84. Paired bootstrap (macro over families):
**sequential - parallel +0.48 [+0.21, +0.76]** (first positive result of the project); multi-question states +0.52
[+0.22, +0.82]; L3-4 +0.51 [-0.01, +1.02]; target questions +0.42 [-0.17, +0.97]. Oracle - parallel +1.75 [+1.45,
+2.06] (L3-4 +3.02): sequential recovers ~27% of the headroom. Gain grows with commit rank (rank 1 +0.6, rank 2
+0.8, rank 3 +1.5): later answers profit from committed earlier ones. Families: policy_applicability 97.5 -> 99.0
(oracle 100), entity_belief_tracking 98.8 -> 100, record_eligibility 76.8 -> 78.1 (oracle 87.5).
MQ parallel on the target questions 90.67 vs P1 88.68: training on all questions helps one-pass accuracy too.
Under-training check (user): P1 dev was still rising at the end (87.1 -> 89.0% over the last 250 steps); we used 9%
of the 360k train states. Queued MQ-large (128k states, soft self-conditioning) after P2s and MQ-soft.

**18:05 P2s killed at step ~310** (soft-token slots, no gate; user asked for a no-gate test): learning the task (loss
1.23 -> 0.70, dev 71.8% at step 250) but dev K1 = K4 exactly; grad norm spikes to 171 through the vocabulary softmax.

**18:40 MQ step test** (same questions, B passes total, ceil(open/passes left) commits per pass): all questions 88.45 /
88.72 / 89.01 / 88.97 / 89.01 at 1/2/3/4/6 steps; states with 6+ questions 80.48 / 80.30 / 81.73 / 81.43 / 81.67.
More steps help up to ~3, then plateau.

**19:15 MQ-soft** (two-pass soft-belief self-conditioning): parallel 86.95, sequential 87.86, soft2-4 87.64-87.70,
oracle 89.93 (all questions). Gains vs its own parallel: sequential +0.80 [+0.54, +1.06], soft4 +0.81 [+0.50, +1.14]:
continuous soft passes match discrete commits, but soft training costs 1.5 points of one-pass accuracy, so MQ is better
in absolute terms. My pick rule compared gains, not accuracy, and started a 216k-state MQ-soft run; stopped at 18%.

**19:45 User redirected** to the intended architecture: a set of latent vectors iterated K times, read once by a
decision head. Third literature review (TRM/HRM/Huginn/LaDiR/2609.01449): random depth, truncated BPTT, belief
feedback in separate slots, same init noise at train and eval, EMA. Found that the plan's decision head added its
correction to pre-norm option states (norm ~1,700 vs ~46 post-norm): nearly inert in E1-E3 (DEVIATIONS #14).

**20:00-21:05 Latent reasoners (LDR v2, 38M params, 32 latents, K ~ U{1..12})**: A frozen P1 88.68 -> 88.71 (K=0 ->
32); B frozen MQ encoder, all sub-questions 88.03 -> 87.98; C encoder trained jointly from S1 85.05 -> 85.05 (and
below P1); D frozen raw S1 42.43 -> 42.46 (learned nothing; grad norm 0.01-0.3). All flat (CIs include 0).

**21:10 Wrap-up** (user's call): REPORT.md + results/report.pdf, figures fig1-fig6, summary_stats.json; bucket sync.

Xet Storage Details

Size:: 12.9 kB
Xet hash:: 1bb75e51d9116967c3b04db87a5a91a9047fa7a052cace97eb0c5a4c1db0220f

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# Lab notebook: think-then-decide (latent denoising passes as thinking)

	Times UTC. Observations as they happened; numbers from the files named.

	## 2026-11-06

	12:54 Box: A100-SXM4 80GB (RunPod), 255 CPUs, 1 TB RAM. Bucket `Mercity/Desicion-Model-Custom` (previous
	project: Qwen3.5-4B decision model with token thinking) reviewed; its 76.5 GB of checkpoints deleted locally (they
	stay in the bucket) to keep disk free.

	13:00 jev-dllm cloned (3b8c01b), S1 downloaded at the pinned revision. S1's model code reviewed (27-line
	bidirectional Qwen3 override; `last_hidden_state` is post final-norm; accepts `inputs_embeds`). Training data:
	40,000 train / 1,000 dev, lengths with 32 thought slots median 246, p99 846, max 1,395 tokens (none over 2,048).
	Core: 6,744 decisions in 17 suites; 221 ContractNLI items exceed 4,096 tokens (S1's protocol drops them).

	13:03 E0 step 1 (`runs/e0/core-s1-theirs`): jev-dllm's own evaluator on S1, this GPU, bf16, batch 32:
	Jabr 65.94, MASSIVE en-US 77.87, Civil Comments 82.67 = published; JevBench original 84.72 (published 86.11,
	one item), VitaminC 67.61 (67.95), PubMedQA 57.20 (57.60): bf16 cross-GPU flips. 118 s of forward time.

	13:20 E0 step 2 (`runs/e0/core-s1-ours*`): our harness, same 6,523 items: suites within 1-2 items of
	jev-dllm's run (JevBench original 86.11 = published), reasoning group 63.96 in both. 1.4% argmax flips remain
	with jev-dllm's exact batches, max \|dp\| 0.13 (DEVIATIONS #3).

	13:25 Speed: profile showed a memory-bound small model (GEMMs ~25% of GPU time; unfused RMSNorm/RoPE/SwiGLU/
	casts and attention backward dominate). torch.compile on the decoder layers: 1.6x (22.8k token-passes/s at K=4).
	Checkpointing: 1.5-2x slower. Padding: K=1 micro-batches were ~3x padded; a padding-aware split brings every K to
	~1.17x (adopted after E1; E1 ran with the budget-only split, same maths).

	13:33 Is the training set memorised by S1 (3 epochs on it)? No: on 5,000 train items KL 0.18, acc 93.5%,
	23% with p(gold) < 0.9; held-out new_dev KL 0.31, acc 88%. Enough signal; plan's data kept.

	13:40 E1 started (S1 + head, no thinking, 1 epoch). Dev at step 250 (peak LR): KL 0.260 -> 0.327, acc 90.3 ->
	87.6%. S1's released checkpoint was selected on this dev set (min KL), so dev is biased toward S1; Core decides.

	13:55 E1 done (`runs/e1`): 1,250 steps, 14.4 min, peak 61.4 GiB (budget-only split; K=1 steps were one
	~3x-padded micro-batch), no OOM retries. Dev: KL 0.260 -> 0.303, acc 90.3 -> 91.5%. Core (all 6,744):
	reasoning 64.99, lookup 74.10, other 60.50, Core17 macro 66.08. (S1 at 4,096 protocol: 63.96 / 73.96.)

	14:02 E2 started (32 slots, K ~ U{1,2,4,8} per step, budget 42,000): peak 74.8 GiB (94%), 22.5k
	token-passes/s, ~1.4 s/step. Gate stays at \|g\| ~ 1e-4 (needs ~0.014 for the feedback to match the mask
	embedding's norm 0.46); W drifts from identity by Adam noise (\|\|W - I\|\| 1.28 at step 380; its gradient is
	proportional to g ~ 0, but Adam rescales it to lr-sized steps). Dev at 250: K1 89.9 / K4 89.8%.

	14:16 Notes pilot failed: vLLM 0.33 needs `ninja` on PATH (installed in its venv). Saved weights were
	2x size (ThinkDecide registers base.model twice as `bb`); fixed, E1 file deduplicated (310 keys, identical).
	First bucket sync: `think-decide/` (token scan before every sync; .env never in the tree).

	14:38 E2 done (`runs/e2`): 1,250 steps, 29.4 min, peak 74.9 GiB (94%), 0 OOM retries. Gate 0 -> 0.0019 (needs
	~0.014 to matter); dev K1 = K4 = 91.37%. Core (all 6,744; K=8 eval took 27 min): reasoning 64.67 / 64.65 / 64.82 /
	64.76 at K = 1 / 2 / 4 / 8; lookup 74.45 -> 74.35. Paired bootstrap: K8 - K1 reasoning +0.08 [-0.27, +0.43];
	E2@1 - E1 reasoning -0.31 [-1.51, +0.87]. H1 and H2 not supported for E2.
	Diagnosis: the feedback x = e_mask + g W LN(h) deadlocks at g = 0. h is a last-layer state (not input-embedding
	space), so W must learn a mapping before feedback helps; W's gradient is proportional to g, and g's gradient points
	along W LN(h), which is meaningless until W has learned. Under Adam both move by noise (W drifted \|\|W - I\|\| ~1.3
	by step 380). So E2's null speaks to this initialisation, not to latent thinking as such.
	S1's predictions at the Notes slots are generic (entropy ~5.6 nats/slot, greedy "The1 is is the the,,,,"), so a
	soft-token feedback would start nearly content-free too.

	14:50 Exploratory arms added (labelled post hoc, after E2's diagnosis; one change each vs E2):
	E2o = gate init 0.014 (feedback norm = mask-embedding norm, so W gets gradient from step 0);
	E2s = soft-token self-conditioning (slot input = E^T softmax(own token logits), mask excluded; gate fixed at 1):
	the continuous-diffusion version, no learned mapping needed.

	15:05 Notes (`runs/e3/notes.jsonl`, Qwen3.5-9B, greedy): the first pilot showed concluding bullets that name
	the answer ("The highest milestone reached is Release") and world-knowledge notes contradicting labels; prompt
	tightened (facts/rules only, no conclusion, never name an option) and the leak filter extended to the gold
	option's short name and stem. 10,000 notes, 9,370 pass (dbpedia 271 and clinc 112 name the gold class); median
	76 tokens, max 175 (cap 256: none capped).

	15:10 Teacher (frozen S1 reads the note; keep a note only if it lowers CE to the target): placement probe on
	1,500 notes (first in file order: arc, clinc, sgd, snli only): after-options keep 70.7% vs before 64.8% ->
	"after" (also position-aligned with the student's slots). Full cache: 5,409 / 9,370 kept (57.7%). Teacher
	accuracy with vs without note: 93.6 vs 93.7% (ARC 96.9 vs 91.8, BoolQ 89.9 vs 88.8, GoEmotions 65.0 vs 68.2).
	Frozen S1 barely uses notes, so E3's target sits close to S1's own states.

	15:12 S1 all 6,744 items (`runs/s1`): reasoning 64.07, lookup 73.96, other 61.49, Core17 66.07.
	E1 - S1: reasoning +0.92.

	15:52 E3 trained (`runs/e3`; 5,409 items with teacher targets): 29.6 min, peak 74.9 GiB, 0 OOM retries. Gate
	0 -> 0.00097 (smaller than E2's 0.0019): the distillation terms did not open the channel. Dev K1 91.47 / K4 91.26%.
	In a direct test at init the distill losses gave the gate a gradient of 1.45 at K=2, but the target is close to
	S1's own states (frozen S1 barely uses notes), so the consistent signal stayed small.

	16:00 Re-think (user asked for something that would actually show diffusion can replace thinking). On S1's
	data even gold-aware notes do not help frozen S1 (93.6 vs 93.7%), so these benchmarks have little headroom for any
	thinking. A convincing test needs a task where token thinking is known to help a lot: the previous project's
	procedural decisions (4B: 46.8% readout vs 95.1% after thinking). New testbed E6 = tasksource/procedural-typed-
	decisions (Apache-2.0; 18 families x 5 levels; not in S1's training data): 78,548 train states (one question
	each, stratified), 1,751 dev (validation split), 3,484 test (test split). Lengths: median 550 tokens (level 0:
	330, level 4: ~1,000), max 6,334 -> train max length 8,192 (no drops; a 2,048 cap would have dropped 2.5%, nearly
	all level 3-4). Arms (32k items, 1 epoch, backbone lr 2.5e-5 = S1's own, the same for all): P1 control (head,
	K=1), P2o (plan's loop, gate opened at 0.014), P2s (soft-token feedback). Evals compiled (all procedural arms).
	P0: S1 zero-shot test 42.37% (L0 53.3 -> L4 34.2), dev 41.73%.

	16:10 Procedural CoT notes for E5 (token thinking by diffusion) started beside E3's eval: only 5 prompts/s while
	sharing the GPU (~1.5 h for 32k) and holding 40 GB that P1 needs; stopped (nothing written). Will regenerate for
	8,000 items when the GPU has room.

	16:20 E3 Core (all 6,744): reasoning 64.51 / 64.08 / 64.39 / 64.47 at K = 1/2/4/8. Pre-registered verdicts
	(results/results.md): H1 not met (E2 K8-K1 +0.08 [-0.27, +0.43]; E3 -0.04 [-0.58, +0.59]), H2 not supported
	(E2 best - E1 -0.31 [-1.51, +0.91]), H3 not supported (-0.59 [-1.84, +0.60]). Only significant difference: E3 lookup
	+1.17 [+0.31, +2.04] over E1. One more epoch (E1/E2/E3 alike) trades suites: Civil Comments +6-7, ContractNLI +2-4,
	PubMedQA -5.6 to -7.

	16:45 P1 (procedural control, 32k items, K=1): 1,000 steps, 18 min. Test 88.68% (L0 96.3, L1 91.9, L2 87.9,
	L3 84.0, L4 83.2); dev 89.08%.

	17:05 Two research agents (literature, independent briefs) converged: (1) latent thinking positions need dense
	supervision (Coconut without curriculum, filler tokens, SIM-CoT collapse, He/Welleck/Fried 2602.03769: extra
	unsupervised positions hurt); (2) feedback should be in a space the model understands (soft token mixtures, not
	last-layer states; LAW: hidden-state feedback < 1.5%); (3) two-pass stop-gradient self-conditioning beats backprop
	through all passes (Loopholing, LAW), and Loopholing reports fine-tuning-only retrofit failed; (4) noise during
	training is a curriculum (clean-only training: 0% in 2609.01449); (5) the procedural states carry 3.25 questions
	each, often intermediate steps of one another (governing_policy -> access_allowed -> review_risk; world_location ->
	belief -> belief_matches_world; shortest_cost -> edge_essential), which my prep had dropped.

	17:35 P2o killed at step ~520 (user's call; plan's loop, gate opened at 0.014): dev K1 = K4 at step 250 (72.36%)
	and 500 (83.97%); gate drifted 0.0140 -> 0.0139. The open channel was ignored.

	17:40 MQ (answer diffusion): all questions of a state on one canvas (S1's block format); masked diffusion over
	answers (reveal each question's gold Yes/No tokens with prob 1-t, t ~ U(0,1), loss on masked questions); readout
	= S1's native Yes/No, no new parameters, no gate. Eval modes on all questions of the 3,484 test states: parallel
	(1 pass), sequential (commit the most confident question per pass), oracle (others revealed). MQ-soft adds two-pass
	stop-gradient self-conditioning with soft Yes/No beliefs (p=0.8) and an eval of K soft passes.

	18:00 MQ results (`runs/mq`, 32k train states = 105,286 questions, 1 epoch, 21 min; test = all 11,582 questions of
	the 3,484 test states). Parallel 88.45 / sequential 88.98 / oracle 90.84. Paired bootstrap (macro over families):
	sequential - parallel +0.48 [+0.21, +0.76] (first positive result of the project); multi-question states +0.52
	[+0.22, +0.82]; L3-4 +0.51 [-0.01, +1.02]; target questions +0.42 [-0.17, +0.97]. Oracle - parallel +1.75 [+1.45,
	+2.06] (L3-4 +3.02): sequential recovers ~27% of the headroom. Gain grows with commit rank (rank 1 +0.6, rank 2
	+0.8, rank 3 +1.5): later answers profit from committed earlier ones. Families: policy_applicability 97.5 -> 99.0
	(oracle 100), entity_belief_tracking 98.8 -> 100, record_eligibility 76.8 -> 78.1 (oracle 87.5).
	MQ parallel on the target questions 90.67 vs P1 88.68: training on all questions helps one-pass accuracy too.
	Under-training check (user): P1 dev was still rising at the end (87.1 -> 89.0% over the last 250 steps); we used 9%
	of the 360k train states. Queued MQ-large (128k states, soft self-conditioning) after P2s and MQ-soft.

	18:05 P2s killed at step ~310 (soft-token slots, no gate; user asked for a no-gate test): learning the task (loss
	1.23 -> 0.70, dev 71.8% at step 250) but dev K1 = K4 exactly; grad norm spikes to 171 through the vocabulary softmax.

	18:40 MQ step test (same questions, B passes total, ceil(open/passes left) commits per pass): all questions 88.45 /
	88.72 / 89.01 / 88.97 / 89.01 at 1/2/3/4/6 steps; states with 6+ questions 80.48 / 80.30 / 81.73 / 81.43 / 81.67.
	More steps help up to ~3, then plateau.

	19:15 MQ-soft (two-pass soft-belief self-conditioning): parallel 86.95, sequential 87.86, soft2-4 87.64-87.70,
	oracle 89.93 (all questions). Gains vs its own parallel: sequential +0.80 [+0.54, +1.06], soft4 +0.81 [+0.50, +1.14]:
	continuous soft passes match discrete commits, but soft training costs 1.5 points of one-pass accuracy, so MQ is better
	in absolute terms. My pick rule compared gains, not accuracy, and started a 216k-state MQ-soft run; stopped at 18%.

	19:45 User redirected to the intended architecture: a set of latent vectors iterated K times, read once by a
	decision head. Third literature review (TRM/HRM/Huginn/LaDiR/2609.01449): random depth, truncated BPTT, belief
	feedback in separate slots, same init noise at train and eval, EMA. Found that the plan's decision head added its
	correction to pre-norm option states (norm ~1,700 vs ~46 post-norm): nearly inert in E1-E3 (DEVIATIONS #14).

	20:00-21:05 Latent reasoners (LDR v2, 38M params, 32 latents, K ~ U{1..12}): A frozen P1 88.68 -> 88.71 (K=0 ->
	32); B frozen MQ encoder, all sub-questions 88.03 -> 87.98; C encoder trained jointly from S1 85.05 -> 85.05 (and
	below P1); D frozen raw S1 42.43 -> 42.46 (learned nothing; grad norm 0.01-0.3). All flat (CIs include 0).

	21:10 Wrap-up (user's call): REPORT.md + results/report.pdf, figures fig1-fig6, summary_stats.json; bucket sync.