training_code/latent_threads/FINDINGS.md

# latent_threads — results (batches lt1/lt2/lt3, 2026-06-12)

Five delayed-selector latent-reasoning organisms on Qwen3-8B (design: HYPOTHESES.md;
mechanism: bottleneck mask Y↛X + filler-dot latent block + query revealed AFTER the
latents). Checkpoints: `/workspace-vast/jbauer/exp/models/latent_threads/<batch>/<task>/final`.
Plots: `/workspace-vast/jbauer/exp/latent_threads/results/`. wandb groups lt1/lt2/lt3
(MATS10-CS-JB/cot-oracle).

## Headline table

| organism | batch | M | organism acc | ablated (Z↛X) | gap | ensemble completeness (per query) |
|---|---|---|---|---|---|---|
| parallel_select | lt1 | 4 | 1.000 | 0.090 | **0.91** | 1.00 / 0.99 / 1.00 / 1.00 |
| chase_select | lt1 | 12 | 1.000 | 0.093 | **0.91** | a 1.00 / b 1.00 / c 1.00 |
| step_select | lt2 (resume of lt1 stall_k07, gate 0.88) | 8 | 0.820 | 0.107 | **0.71** | steps 1-4: 0.98-0.99; steps 5-8: 0.71/0.77/0.68/0.67 |
| coin_track | lt1 | 8 | 0.937 | ~0.14 | **0.80** | Anna 0.99 / Ben 0.89 / Cara 0.85 |
| agg_select | lt3g (v2 grind) | 6 | 0.893 | 0.073 | **0.82** | values 1.00/0.99/1.00; sum10 0.813 |

All ablated readouts ≈ chance ⇒ every organism's answer information flows X → dots → Y
only: the latents are load-bearing by construction, verified. Completeness = accuracy per
possible query on the SAME instances; since Z cannot see the query, high completeness on
all slots = one fixed latent computation serves the whole query family (the ensemble claim).

## Probe grids (ridge linear probes on dot residual streams, operating mask)

**parallel_select (best L36).** H1a (full ensemble decodable) CONFIRMED: every register's
result hits ~1.0 at several dots; inputs (starts) are carried too. H1b (one-result-per-dot
diagonal) REJECTED: the layout is prefix-accumulating and redundant (dot1 already holds
res_a+res_b at 1.0; by dots 3-4 nearly everything decodes). Logit lens P(surface digit)
≤ 0.2 everywhere — the code is NOT vocab-aligned (non-verbal carry, not "hidden digits").

**chase_select (best L36).** H2a (lockstep stripes dot_{3(t-1)+b} ↔ s_b(t)) REJECTED;
instead chain-major bursts with run-ahead: dot1 decodes chain a's ENTIRE trajectory
(s_a2-s_a4 ≈ 0.96-1.00 — one dot's depth covers ~4 lookups, cf. pn1), dots 2-4 hold chain
b (0.97-1.00), chain c materializes over dots 5-9 (c4 → 1.00 from dot7), late dots refresh
a. H2b (ensemble maintenance) CONFIRMED, with a twist: finals live at DIFFERENT positions
(b4 fades to ~0.5 by dot12 but reads 0.97-1.0 at dots 2-6) — the block is a spatially
distributed key-value buffer and the answer position fetches content-addressed; behavioral
completeness is 1.00 for all three chains.

**step_select (best L36).** Run-ahead staircase: dot p decodes s_1..s_{p+1ish} (dot1: s1,s2
at 1.0, s3 0.88). H3a retention CONFIRMED for computed states: s1/s2 still 0.95-0.99 at
dot8. Deep states s5-s8 cap ~0.6-0.8 everywhere — matching the behavioral per-step
gradient (0.99 for t≤4 vs ~0.7 for t≥5): the organism's errors are deep-chain COMPUTATION
errors (serial cap ~6-8, cf. pm2/pm3), not retention/readout failures. Lens peaks 0.45 at
dots 3-4 then fades — partial early vocab-surfacing, opaque later.

**coin_track (re-probed at forced L36; the auto-picked L20 grid was an artifact).**
Layer-selection pitfall first: who-moved-at-event-i (prompt-derivable, 3-class) saturates
1.00 at every dot at L4-L20 and dominated the best-layer mean — lesson: exclude
prompt-syntactic targets from layer selection (probe.py grew --layer). At L36 the real
structure: per-event running counts e1-e8 ≈ CHANCE at every dot — the trajectory is NOT
retained — while the three FINAL counts decode strongly at scattered parking dots
(final_Anna 0.97 @dot1, final_Ben 0.90-0.95 @dots 2,5-8, final_Cara 0.85-0.89 @dots 1,4),
matching behavioral completeness (0.99/0.89/0.85). H4a (event-driven update train)
REJECTED at the readout layer; H4b (world-state ensemble) CONFIRMED as a 3-slot register
file updated in place. The step_select contrast is the headline: queries there cover every
intermediate → every state retained; here only finals are queryable → only finals kept.
**The latent ensemble retains exactly what the query distribution demands.**

## agg_select: the aggregation-over-latents story (3 attempts)

- v1 (value/argmax/argmin/sum, lt1): STALL at k=0 (surface!), 0.672. Diagnosis at k=0:
  value 1.00, sum 1.00, argmax 0.39, argmin 0.43 (chance 0.33) — 3-way COMPARISON over
  spread digits doesn't train in the masked suffix even with surface operands.
- v2 (value+sum, lt2): mastered k=0 instantly, STALL at k=1, 0.516. Diagnosis at k=1:
  value-via-dot 1.000 but sum10 0.078 (below chance = systematically stale) — the dot
  demonstrably carries the digit (value query reads it), but the suffix ADDITION circuit
  keeps consuming the surface-token pathway and cannot use the latent operand. Clean
  negative: reading latents transfers; arithmetic INTEGRATION of a latent operand does not
  (at LoRA r=32, lr 1e-4, 800 steps).
- v3 (lt3, sum-in-body): STALL at k=1 (0.539 ≈ (value 1.0 + sum 0.08)/2) — at k=0 the
  model answered sum queries by COPYING the teacher-forced surface sum digit; dotting it
  removed the copy source. Design lesson: a teacher-forced aggregate in the surface
  curriculum invites a copy shortcut.
- **v2g (lt3g): SUCCESS — the organism.** Resume of v2 stall_k01 with 2400 steps/stage +
  smoothing 0.25: k=1 (the wall) cleared, k=2-6 fell quickly (the integration circuit,
  once formed, generalizes across operands). Final: organism 0.893 / ablated 0.073 (gap
  0.82); completeness value 1.00/0.99/1.00, sum10 0.813. So suffix-side aggregation over
  latent operands is TRAINABLE but ~3x slower than value-reads — a trainability-cost
  finding, not an impossibility.
- **H5a CONFIRMED (raw workspace, not answer cache), probe grid @L28:** finals v1/v2/v3
  decode at ~1.0 across the block (prefix-staircase: v1 from dot1, v2 from dot2, v3 from
  dot3 — run-ahead again) and intermediates are carried too, but **sum10 ≈ chance
  (0.06-0.16) at EVERY dot** while behavioral sum accuracy is 0.813 ⇒ the sum is
  assembled downstream in the masked suffix from raw latent operands. Caveats: argmax/
  argmin probe at 0.7-0.91 despite never being trained queries in this lineage (partially
  derivable from value codes by a linear readout; not a clean precompute claim). Lens:
  uniquely here the LAST dot is vocab-aligned (P(surface)=0.85 for v3); other positions
  dark (≤0.18).
- v3b (lt4, bonus): STALL at k=1 even with 2400 steps (acc oscillating 0.51-0.62 ≈ value
  1.0 + sum near-chance) — **a single dot position cannot learn 3-operand mod-10 addition
  under this regime, even with all operands surface-visible**, while the multi-position
  suffix learned the same function in v2g on the same budget. Depth is not the obstacle
  (one position composes ~4-5 serial lookups, cf. run-ahead/pn1); the plausible culprit is
  gradient indirectness — the answer position receives the loss directly, a dot's hidden
  state only via attention. Net dissociation: latent positions trained this way COMPUTE
  (chains, lookups) and CARRY, but new multi-operand circuits form preferentially at
  loss-bearing positions.

## Cross-task picture (the "latent trace ensemble")

1. **Load-bearing is robust and easy** under the mask — 5/5 attempted blocks carry their
   task (incl. v3 pending), ablations ≈ chance.
2. **Delayed-selector pressure works**: every organism keeps ALL threads readable (high
   completeness on unselected threads/steps/entities), confirming the structural
   superposition argument — Z can't know the query, so it must keep the ensemble.
3. **The layout is the model's own economy, not the curriculum's surface order**: bursts +
   run-ahead + content-addressed parking, NOT lockstep per-position carries. The 8B's
   depth (~4-5 serial lookups per position) makes single dots powerful; the block
   organizes as a buffer of partial trajectories.
4. **Computation in the suffix is the bottleneck**: reading latents is easy (value queries
   1.0 through dots), comparisons/arithmetic over latent operands are hard to SFT in.
5. **Logit lens is mostly dark** (P(surface digit) ≤ 0.2-0.45, fading late): linear-probe-
   decodable but not vocab-aligned — a good AO target that a text monitor cannot read.

## Reproduction

- train: `python -m latent_threads.train --config latent_threads/configs/<task>.json --batch-id <id>`
- verify: `python -m latent_threads.eval_masked --adapter .../final`
- probe: `python -m latent_threads.probe --adapter .../final --lens [--layer L]`
- per-type agg diagnosis: `python -m latent_threads.diag_agg --adapter ... --k <stage>`
- batch logs: `/workspace-vast/jbauer/exp/latent_threads/logs/`; launchers (not committed):
  `/workspace-vast/jbauer/exp/latent_threads/{smoke_all,lt1_all,lt2_all,lt3_all}.sh`.
- Stalls and retries: step_select stalled at k=7/0.852 with gate 0.9 (lt1) and finished
  under gate 0.88 after resume; node-13 gave cudaErrorDevicesUnavailable (excluded).

## ★★ SOLVED: journeys — several cohesive NATURAL-LANGUAGE load-bearing trains (lj1, 2026-06-13)

The NL extension of the Markov organism. K travelers each walk a contiguous m-room PATH = a
cohesive natural-language "train of thought" extending over m latent positions (e.g. study ->
garage -> library -> bedroom); several run in parallel; each position load-bearing.

Architecture (markov_nl.py / train_markov_nl.py): THREAD-MAJOR layout (thread b's m rooms are a
contiguous span) + within-thread MARKOV mask (position t attends only t-1; pos 0 reads prompt) +
a CONTIGUOUS answer-slot row (K slots gather each thread's final room into adjacent positions so
the delayed-selector answer reads positionally — the readout fix; spread thread-lasts capped
TF-readout ~0.5, the slot row broke it). Vocab-constrained feedback over a 10 single-token ROOM
WORD alphabet; teacher-forcing anneal. Fixed transition rule (no per-instance table -> trainable
under the load-bearing mask, unlike a pointer chase). Delayed POSITIONAL query (person number j).

Verified (lj1k3m5, K=3 m=5, free-running): organism 0.970; ablate-thread-start 0.075 (chance);
per-room-step corruption -> 0.12/0.35/0.22/0.27 (EVERY room position load-bearing, drop 0.62-0.85).
k3m4 and k3m5 both reach ~1.0. Recipe is the digit Markov recipe restructured thread-major with NL
symbols. Code latent_threads/{markov_nl,train_markov_nl,verify_nl}.py; configs journeys_k3m{4,5};
ckpts lj1k3m4/lj1k3m5. Diagnostic note: aux (room-classifier CE) hit ~0 early = the trains were
always perfectly computed; the only difficulty was the delayed-selector READOUT (3 readout
iterations: name-query -> positional-query -> contiguous-slot row, which solved it).

## ★ SOLVED: Markov latent-CoT organism — genuine PER-STEP load-bearing (lmv, 2026-06-13)

The 4/4 organism the whole investigation chased. dot diffuse got 3/4 (per-step necessity failed:
recompute). The Markov organism gets ALL FOUR.

**Organism: `lmvk3m4` (and lmvk3m5), diffuse K=3 M=4/5.** Embedding-space latent CoT: K positions
per step (one per cell), input(t+1,i) = vocab-constrained feedback from h(t,i). Recipe (each piece
fixes a diagnosed failure):
1. MARKOV step-windowed mask (markov.py build_markov_mask, UNIT-TESTED): step1 attends prompt;
   step t (t>1) attends ONLY step t-1's K positions; Y attends ONLY step M. The only path is
   X->step1->...->stepM->Y -> per-step necessity BY CONSTRUCTION, immune to recompute/depth shortcut.
2. VOCAB-CONSTRAINED feedback: feedback = softmax(digit_head(h)/temp) @ E[digits] (soft mixture
   over digit embeddings), aux = CE(digit_head, gt). Readable by construction; CE drops where the
   plain-MSE-to-embedding plateaued at 0.5 (which left free-running at chance — lm1).
3. SCHEDULED-SAMPLING anneal: teacher-force GT digit embeddings with prob tf annealed 1->0 over
   2500 steps -> bootstrap from the trainable surface-digit regime, hand off to self-generated.

**Verification (verify_markov.py, k3m4 best, free-running eval = self-generated latents):**
- organism 0.930; ablate step1->prompt 0.090 (chance: chain's only input gone);
- per-STEP corruption (noise into step t's feedback): step1->0.109, step2->0.078, step3->0.105 —
  EVERY step drops to chance (drop ~0.82-0.85). No recompute escape (step t+1 sees only step t).
- k3m5 free-running 0.977; k3m4 0.93-0.96.

**Grades 4/4:** (1) multi-step, EACH step load-bearing (corrupt any -> chance) — the criterion that
defeated dots (recompute) and soft (vestigial); (2) parallel (K=3 cells/step); (3) parallelism
NECESSARY (light-cone proof); (4) load-bearing (ablate -> chance). Code: latent_threads/markov.py,
train_markov.py, verify_markov.py; configs markov_k3m{4,5}_vocab.json; ckpts lmvk3m4,lmvk3m5.

## TARGET-CRITERION RESULT: "multi-step latent tokens truly bearing PARALLEL load, parallelism NECESSARY"

Decomposed the criterion into 4 tests and chased an organism that passes all 4. Result: 3/4 are
achievable and DELIVERED; the 4th (per-step necessity) hits a fundamental trilemma.

**Delivered organisms (ls8, dot recipe): dot diffuse** — coupled ring CA x_i'=(x_{i-1}+x_{i+1})%10,
K=3, M=4/5 (12/15 dots), FULL-ROW surface CoT (each step writes all K cells -> dots carry the whole
evolution). Trained via the proven bottleneck-mask dot recipe (compute-from-prompt, NOT recurrence).
  - k3m4: organism 0.947 / ablated 0.117 / gap 0.830; completeness c1/c2/c3 = 0.98/0.99/0.83.
  - k3m5: organism 0.970 / ablated 0.097 / gap 0.873.
  Ckpts ls8k3m4/ls8k3m5 under exp/models/latent_threads/.

**Grading vs the 4 criteria:**
1. LOAD-BEARING (block) -- PASS: ablation Z!->X -> chance (gap ~0.85); the dot block is the
   necessary conduit prompt->answer.
2. PARALLEL -- PASS: per-cell completeness 0.83-0.99; one fixed dot computation serves every cell
   query.
3. PARALLELISM NECESSARY TO SOLVE -- PASS by construction: light-cone proof (K=3,M>=2: each final
   cell depends on ALL initial cells; cannot solve tracking one thread). This is the criterion chase
   could NOT meet (independent chains).
4. MULTI-STEP, each step truly load-bearing -- NOT achieved. leave-out-per-dot ~0.92-0.98 (no single
   dot necessary), cross-patch follows the ORIGINAL not the donor (0.94 vs 0.07) -- the SAME
   recompute signature as the vestigial soft finding: under Y!->X the dots recompute the CA from the
   prompt (Z->X allowed), so no individual step/position is load-bearing, only the block.

**The trilemma (why criterion 4 is fundamentally hard on the 8B):**
- Allow Z->X (bottleneck mask): trains, but latents RECOMPUTE from the prompt -> per-step not
  load-bearing (block-only).
- Forbid Z->X (tight/tight_first recurrence): per-step would be load-bearing by construction, but
  DOES NOT TRAIN (chase: lookup table unreachable; diffuse-soft: mod-10 arithmetic unlearnable in
  latent space; ls5/ls6 flat at chance through all warm-starts incl GT + mask-curriculum).
- Force single-pass to fail (large M): dot-chain caps ~7-16 (k3m6/M=18 STALLS at k=16) and the 8B
  single-passes shallow reach -> no clean window where multi-step is required AND achievable
  (the pointer_chase depth-wall, reconfirmed).
So: load-bearing-block + parallel + parallelism-necessary is achievable; adding genuine per-step
necessity is blocked by recompute-vs-trainability-vs-depth. This is the central negative result.

**Soft-token detour (ls1-ls6) summary:** moved to CODI-style recurrence to get per-step load-bearing;
default-mask soft organisms train but are VESTIGIAL (controls: zero-all inputs 1.000, cross-patch
follows original) -- they too recompute from the prompt. Tight/tight_first recurrence (the fix) does
not train (mod-10 arithmetic in latent space; flat at chance across cold/GT/GT+mask-curriculum, m4-8,
K3-5). GT-warmstart, mask-curriculum, multi-GPU bigger batch all tried. Honest conclusion: genuine
load-bearing in-latent recurrence is very hard to induce on an 8B at this scale -- matching 2508.03440
(single-threaded/collapse), 2602.00449 (shortcut on long), 2604.04902 (latents often unnecessary).

## Compression frontier (lt7, train_shrink.py) — the dot-count question answered

The D-sweep above mixes dots with surface digits (every point is a trained curriculum stage
gated at >=0.9), so it cannot test whether FEWER latent positions would suffice. The shrink
curriculum does (resume the organism, reduce the all-dot block length L stepwise,
mastery-gated, no surface scaffold at any L):

- **chase_select: frontier L=1.** All 11 shrink stages mastered; at a SINGLE dot the
  organism reads every chain's final at 1.000 (ablated 0.000). The L=1 probe: the one dot
  decodes the three finals at 0.88-1.00 plus most of the late trajectory (early
  intermediates fade) — 12 task states through one residual stream, retaining the queryable
  subset. The M=12 layout was convention, not necessity; 3 parallel 4-hop chains fit in one
  position's depth x width.
- **step_select: frontier L=2 — and compression IMPROVES accuracy.** L=2 organism = 1.000
  (vs the M=8 organism's 0.820); the L=1 attempt plateaus at exactly 0.820 (single-position
  depth limit ~the original's level). Mechanism: 8 serial hops through 8 dot positions
  accumulate errors; two positions of ~4-hop run-ahead each sit inside the model's reliable
  single-pass depth. Probe at L=2: dot1 = early trajectory (s1-s2 1.0, fading), dot2 = a
  broad window (s1-s4 ~0.9, deep 0.7-0.8) — yet behavioral per-step readout is 1.00, so
  **linear probe decodability is a lower bound, not the ceiling, on what the (nonlinear,
  attention-based) answer head extracts.**

Implication for the published M-length organisms: their many-dot layouts are
curriculum-induced redundancy; capacity was never binding. The "necessity" property that
survives compression is the ABLATION (latents are the only channel), plus now a measured
positions-vs-accuracy tradeoff (1 dot = 0.82, 2 dots = 1.00 for the 8-step trajectory).

## Soft-token organisms + the VESTIGIAL-RECURRENCE finding (ls1/ls3, soft.py)

Moved into the CODI/SoftCoT substrate: L soft positions z_t, input(z_{t+1}) = proj(last-layer
hidden(z_t)), answer-CE BPTT (train_soft.py, multi-GPU manual all-reduce). Three arms on
chase_select (L=4 steps, 3 chains superposed per token): warm (LoRA from dot organism) hit gap
1.000 in ~150 steps but INHERITED the dot burst layout; GT-supervised (Latent-SFT targets z*_t =
sum_b R_b E[digit s_b(t)]/sqrt(B)) hit gap 1.000 in ~500 steps.

**BUT the {shuffle, cross-patch, leave-out} CoT controls (eval_controls.py) reveal the soft
recurrence is VESTIGIAL** for the GT-chase organism (default ORGANISM mask, Y!->X only):
- baseline 0.996; **ZERO-ALL latent inputs 1.000**; **RANDOM latent inputs 0.879**;
  cross-patch a different instance's whole latent block -> answer follows the ORIGINAL 0.996
  (donor 0.109 = chance); shuffle 0.992; leave-out 1.0 every position.
- Interpretation: because Z->X is allowed, each latent POSITION recomputes its content from the
  prompt via attention; the fed-back embeddings are ignored. It is the DOT mechanism with
  GT-shaped activations, NOT genuine continuous-thought recurrence. This single fact explains:
  depth-gen 0.000 (no recurrence to extend), width-gen 1.000 (extra slot just computes from
  prompt), shuffle no-drop, cross-patch follows original, and homogeneity cos 0.99 (last layer).
- Matches the literature's cautions: 2604.04902 (latents often unnecessary/decodable),
  2508.03440 (single-threaded/collapse), 2602.00449 (shortcut on longer tasks).
- Held-out at trained length is still 1.000/1.000 (no memorization) — the organism is correct,
  just not via the recurrence we intended.

**FIX = TIGHT recurrence (soft.py TIGHT = forbid Z->X too).** The prompt then reaches the latents
ONLY through z_1's projected input, so the recurrence is the sole prompt->answer path -> the input
feedback is load-bearing by construction (zero-all/random/cross-patch should then collapse to
chance, and depth-gen has a chance of working). train_soft `tight:true`; config
soft_chase_tight_gt.json; ls3t run. The default-mask soft organisms are kept as the
"compute-from-prompt" baseline that the controls expose.

## CoT controls (every organism, eval_controls.py) — report in model cards
- shuffle (permute latent positions), cross-patch (splice a different instance's latent block;
  donor-following = sole-path test), leave-out (per-position necessity), + zero-all / random
  inputs (soft: recurrence load-bearing?). Dot organisms use a layer-1 residual-stream hook.

## Train vs test (seen-vs-fresh, eval_seen_vs_fresh.py)

Training data is an infinite procedural stream; "train" = bit-exact replay of the first
training stage's instances (provably seen once), "test" = disjoint seed. n=400 each:
parallel 1.000/1.000, chase 1.000/1.000, step 0.845/0.823, coin 0.900/0.915, agg
0.922/0.920 — max |delta| 0.022: no instance-level memorization, the organisms run the
algorithm.

## Published (2026-06-12)

HF: cds-jb/qwen3-8b-latent-threads-{parallel-select,chase-select,step-select,coin-track,
agg-select} — adapter + model card (training details, train/test acc, plots) + examples.md
+ training_code/ snapshot (per the AGENTS.md HF-upload imperative). Collection:
huggingface.co/collections/cds-jb/latent-threads-delayed-selector-latent-reasoning-qwen3-8b-6a2c3eee19d4300f6186394a

## Probing across layers+positions & shuffle/cross-patch controls (Markov organisms, 2026-06-13)

`probe_markov.py` — one pass per organism: (a) a ridge linear probe decoding EACH latent position's
own task value from its residual stream at every layer → a [layer × position] decodability grid; (b)
behavioral shuffle/cross-patch controls on the free-running latents. Results:
- **Probe grid**: every position's parallel-thread state is linearly decodable. Diffuse (coupled CA)
  peaks **late, layer 36** (mean per-position decodability m4=0.898, m5=0.997) — the digit states are
  computed deep. Journeys (room trains) is decodable **early, layer 4** (mean 1.000) — fed-back room
  tokens are present from the start. (figs: `results/probe_markov_<tag>.png`)
- **Controls** (acc vs chance 0.10): intact ≈1.0; **shuffle** (permute latent positions) and
  **cross-patch** (swap in another instance's latents) BOTH collapse to chance —
  diffuse-m4 1.000→0.106/0.113, diffuse-m5 0.988→0.087/0.106, journeys-m4 1.000→0.131/0.119. The
  answer needs the specific content at each position in the right order (not a bag, not the prompt).
  This is the live opposite of the vestigial SOFT organism (zero-all=1.0, cross-patch followed the
  original) — here the recurrence is the only path. Added to every Markov model card.

## Tales — the "statements" organism (3rd Markov variant, in training 2026-06-13)

Same requirements as journeys but the train of thought is **actual statements, not a logical token
sequence**. Each latent STEP is a 9-token NL clause *"{name} entered the {room} and found the
{item}."* — template tokens are fixed word embeddings (so the latent span literally reads as a
statement), with TWO computed slots (room@+3, item@+7) carrying TWO INDEPENDENT load-bearing chains
(room (7r+3)%10, item (3i+1)%10). Each statement bears parallel load in >1 token; the delayed query
selects one (character × attribute) of 2K chains → all maintained. Statement-windowed Markov mask:
each computed slot attends ONLY its same-attribute predecessor (+ prompt at step 1), never the
template tokens → per-statement load-bearing, no prompt-recompute past step 1. Code:
`latent_threads/{tasks.py::Tales, markov_tales.py, train_markov_tales.py, configs/tales_k3m4.json}`.
Mask CPU-unit-tested (each slot ↔ its predecessor only; Y reads only the 2K slots).

## AVBench wiring (DONE)

Suite `latent_threads`, 7 tasks, 1240 token-exact rows (scripts/build_latent_threads_eval.py
→ build_avbench.py::build_latent_threads_rows; narrow/broad tiers via the NARROW registry).
Rows are GPU-filtered to queries the organism answers correctly (pass rates: par 320/320,
chase_final 240/240, step 67/80, coin 222/240, agg 311/320 — matching behavioral accuracy);
chase_state rows (intermediates) are probe-supported pass-throughs, mirroring the
pointer_state precedent. `row_metadata.selected=False` rows (25-100% per task) ask about
threads never named anywhere in the transcript — content with no textual trace at all.
Pushed to cds-jb/AVBench (2887 rows, 44 tasks), items re-ingested, viewer restarted.