EXPERIMENT BACKLOG — SONAR TAE interp follow-ups (2026-06-11)
Generated against INTERP_RESULTS.md §10–12 + AUTOPILOT_INTERP.md. Conventions:
- All scripts live in
parascopes/layerwise/src/(synced to boxes; run from project venv). - Runtime = SONAR load (~5 min) + job, single A40 unless noted. Encoding dominates most jobs.
- "NEW FLAG
--x" = small addition to an existing script (≤20 lines); "NEW SCRIPT" = ≤80-line sketch given. - Every item: (1) sub-question + which finding it deepens/falsifies, (2) how, (3) runtime, (4) what would be surprising.
TIER 1 — VERY HIGH VALUE, SHORT (≤ ~1.5 h each; saturate free boxes with these first)
1.1 Joiner-operators: is each connective its own linear operator?
- Q: §12 found ONE order-equivariant high-rank map for " " concatenation. Is "However,"/"because"/"Meanwhile," each a different rotation, and is W_however − W_space low-rank (a "discourse-relation delta")? Deepens role-rotation anatomy; falsifies "one universal concat operator" if joiners need disjoint maps.
- How (existing flag):
python tae_composition.py --shards 0-3 --n_pairs 12000 --joiner " However, " --out tae_comp_joiner_however.jsonrepeat with" because "," Meanwhile, "," Then "; then compare cross-joiner: apply W_space to however-pairs (NEW FLAG--eval_map path.ptto load a saved map; also save maps via NEW FLAG--save_map). - Runtime: 5 min load + ~35 min per joiner (4 jobs, parallelize).
- Surprising: cross-joiner transfer cos ≈ same-joiner (semantics of the connective lives nowhere in the map — pure surface concat); or ΔW rank ≤ 8 (discourse relation = low-rank additive code, contradicting "order is a global rotation").
1.2 Unbinding from NOISED z_AB — is invertible superposition robust?
- Q: Unbind recovers both slots at cos .72. VSA predicts graceful degradation under noise; if recovery cliffs at small σ, "invertible superposition" is a ridge artifact fit to clean vectors. Deepens/falsifies §12 round-2 invertibility.
- How: NEW FLAG
--noise_sigma 0,0.1,0.2,0.4,0.8(×RMS, applied to z_AB at TEST time only) intae_comp_unbind.py:python tae_comp_unbind.py --shards 0-3 --n_pairs 15000 --noise_sigma 0,0.1,0.2,0.4,0.8 --out tae_comp_unbind_noise.json - Runtime: 5 min + ~40 min (encoding reused across σ).
- Surprising: recovery FLAT to σ=0.4 (slots stored far more redundantly than content, which dies at σ≈0.4 per rate–distortion); or asymmetric collapse (slot B dies first → slot-A dominance is also a robustness asymmetry).
1.3 Serial-position: does the middle-slot deficit grow with depth?
- Q: 3-slot unbinding gave .63/.58/.62 (middle worst). Does depth 4–5 give a deeper U-curve (capacity limit) or flat-middle (fixed crosstalk)? Deepens §12 unbind3; connects to human serial-position literature.
- How (existing flag):
python tae_comp_unbind3.py --shards 0-5 --n_chains 8000 --depth 4 --out tae_comp_unbind3_d4.jsonpython tae_comp_unbind3.py --shards 0-5 --n_chains 8000 --depth 5 --out tae_comp_unbind3_d5.json - Runtime: 5 min + ~50 min each (two boxes).
- Surprising: recovery for ALL slots collapses at depth 5 (hard capacity ≈ 4 ≈ ID/6?); or middle deficit vanishes at depth 4 (3-slot result was noise).
1.4 Serial-position: position or LENGTH?
- Q: Is the middle-slot deficit indexed by slot ordinal or by char-offset? Falsifies "serial position" framing if it's just "stuff far from both ends in characters".
- How: NEW FLAG
--len_bucket short|longintae_comp_unbind3.py(filter constituent sentences to <60 chars vs >120 chars; same depth 3). Compare deficit depth at matched slot count, 2× different absolute offsets. - Runtime: 5 min + ~40 min ×2.
- Surprising: deficit scales with chars not slot index → it's an attention/length effect of the encoder, not a binding-capacity effect.
1.5 Belief simplex under emission-noise dose
- Q: Does the 3-vertex barycentric model keep beating the 1024-d probe as emissions get noisier (surface baseline already drops to .92 at emit .55)? The cleanest "geometry is real" stress test; falsifies the simplex upgrade if vertex-inversion L1 degrades faster than the probe.
- How (existing flag):
for e in 0.45 0.55 0.7 0.9; do python tae_belief_barycentric.py --n_seqs 4000 --len 6 --emit_own $e --out tae_belief_bary_e$e.json; done - Runtime: 5 min + ~25 min per dose (one box, sequential ~2 h, or split).
- Surprising: simplex advantage GROWS with noise (geometry is what generalizes); or vertices drift with emit_own (vertices are emission-statistics artifacts, not state anchors).
1.6 4-state HMM → tetrahedron?
- Q: Comp-mech predicts a 3-simplex (tetrahedron) for 4 states. Does the barycentric win persist with 4 vertices, and are the fitted vertices affinely independent? Deepens §12 simplex claim from triangle to family.
- How: NEW FLAG
--n_states 4intae_beliefstate.py/tae_belief_barycentric.py(add a 4th topic bank — sports clauses — toTOPICS; transition matrix extends symmetrically; ~15 lines).python tae_belief_barycentric.py --n_states 4 --n_seqs 5000 --len 6 --out tae_belief_bary_4state.json - Runtime: 5 min + ~35 min.
- Surprising: 4-state vertex inversion FAILS while 3-state worked (simplex was a coincidence of 3 ≤ visible-topic dims); or vertices land in a 2-plane (rank deficit → posterior is compressed).
1.7 Attribute adders beyond length: digit-count adder, question-ness OR-gate
- Q: PC0(z_AB)=0.546·PC0(A)+0.545·PC0(B) is a literal size-adder. Are other extensive attrs (digit count, quote count) also adders, and are BOOLEAN attrs (is_question, first_person) OR-gates (max) rather than sums? Deepens §12 size-adder into an "attribute arithmetic table"; falsifies "PC0 is special" or "everything is a weighted average".
- How: NEW FLAG
--attr_addersintae_comp_anatomy.py(~25 lines: for each attr probe direction d, fit d·z_AB ≈ f(d·z_A, d·z_B) with f ∈ {sum, max, mean, logistic-OR}; report best model + R² per attr).python tae_comp_anatomy.py --shards 0-3 --n_pairs 20000 --attr_adders --out tae_comp_attr_adders.json - Runtime: 5 min + ~45 min.
- Surprising: is_question composes as a SUM (graded "questionness mass") rather than OR; or digit-count is NOT additive while length is (length axis privileged by SONAR's training objective).
1.8 Language fingerprint: MACHINE vs HUMAN text
- Q: The 84% language-ID fingerprint was built from TAE round-trip translations (machine text). Does it hold for HUMAN-authored text in each language, or is it a translationese/round-trip artifact? Could falsify the entire language-axis interpretation.
- How: NEW SCRIPT
tae_lang_human.py(~70 lines): load FLORES+ dev (datasets.load_dataset("openlanguagedata/flores_plus"), 6 langs matching tae_comp_language), encode human sentences with correct source_lang, apply the ALREADY-TRAINED 6-way classifier from tae_comp_language (NEW FLAG there:--save_clf); also train fresh on human, test on machine (both directions). Report acc + mean-offset cos(human_μ_L, machine_μ_L). - Runtime: 5 min + ~20 min (few thousand short sentences).
- Surprising: machine→human transfer ≈ chance (fingerprint = round-trip artifact, §12 row needs a caveat); or human fingerprint STRONGER (round-trip partially launders it).
1.9 Does the language fingerprint survive composition?
- Q: Compose two French-origin embeddings with the (English-trained) composer: is W(z_fr(A), z_fr(B)) still classified French at 84%, attenuated, or English? Connects the two §12 families; tests whether the composer's rotation preserves the 4.4-dim lang subspace.
- How: NEW FLAG
--comp_same_lang fra_Latn,deu_Latn,spa_Latnintae_comp_language.py(~20 lines: reuse its existing composer + classifier; compose same-lang pairs, classify the output).python tae_comp_language.py --shards 0 --n_texts 1500 --comp_same_lang fra_Latn,deu_Latn,spa_Latn --out tae_comp_lang_survive.json - Runtime: 5 min + ~50 min (translation decodes dominate).
- Surprising: composed embeddings classify as ENGLISH (composer trained on English pairs imprints its own fingerprint → the fingerprint is partly an operator artifact, not content provenance).
1.10 Is the fingerprint the cause of the cross-lingual alignment gap?
- Q: cos(z_en, z_xx) ≈ 0.71–0.76 for same content. The lang component is decode-inert and LEACE-erasable at 8.8% edit. If LEACE-ing language raises cross-lingual cos toward ~0.95, the fingerprint IS the misalignment — and erasure gives a free "language-neutral SONAR". Deepens decode-inert finding into a usable tool; falsifies "fingerprint is small" if alignment barely moves.
- How: NEW FLAG
--xlingual_alignintae_lang_inert.py(~15 lines: pair z_en with z_xx of same content — both already computed — report cos before/after LEACE, per language).python tae_lang_inert.py --shards 0 --n_texts 600 --xlingual_align --out tae_lang_align.json - Runtime: 5 min + ~30 min.
- Surprising: cos jumps to >0.9 (the entire cross-lingual gap is the vestigial fingerprint — headline: a 4-dim LEACE makes SONAR language-neutral); equally surprising: no change (the .25 gap is content/rendering, fingerprint truly tiny).
1.11 Inverse-order semantics: "B. Before that, A." — surface order or event order?
- Q: z encodes surface order (cos(z_AB,z_BA)=.52, order = global rotation). For text whose SURFACE order is B,A but whose SEMANTIC order is A-then-B ("B. Before that, A."), is z closer to z("A then B") or to z("B then A")? Tests whether SONAR's order code is token-order or event-order — sharpens what the rotation encodes.
- How: NEW FLAG
--template "{B} Before that, {A}"intae_composition.py(generalizes--joiner; ~10 lines). Then: cos of encoded template text to z_AB and z_BA; and does W applied as (A,B) or (B,A) predict it better?python tae_composition.py --shards 0-3 --n_pairs 10000 --template "{B} Before that, {A}" --out tae_comp_beforethat.json - Runtime: 5 min + ~35 min.
- Surprising: the embedding sits near z("A then B") (SONAR normalizes to EVENT order — a semantic canonicalization nobody has reported); expected/boring: pure surface order.
1.12 Aligned core = high-frequency features?
- Q: L2-reg created an aligned core (PW-MCC .095→.289, 19% > .5). Are the aligned features simply the HIGH-FREQUENCY ones (which any objective pins), with the rare tail still arbitrary? Deepens canonicity fix; predicts whether scaling L2 can ever canonicalize the tail. Connects to "mid-frequency = labelability sweet spot".
- How: NEW SCRIPT
tae_canon_core_anatomy.py(~60 lines, CPU-only on saved fixoff ckpts + cached activations): per matched feature pair, scatter PW-MCC vs log firing freq, vs mean activation, vs labelability proxy (decode-direction coherence); report Spearman. - Runtime: no SONAR needed if activations cached, else 5 min + ~20 min.
- Surprising: alignment is UNCORRELATED with frequency (L2 pins a semantic core, not a frequency core — much stronger claim); or perfectly rank-correlated (canonicity fix is trivially "frequent features converge").
TIER 2 — HIGH VALUE, LONG (multi-hour; one box each, launch early)
2.1 Behavior⊥geometry instance #4: UNBINDING map trained through decoder CE
- Q: Three times now CE-through-decoder improved behavior while cosine dropped (SAE, composer, textloss-v2). Does the unbinding map show the same trade — decode-recovery of slots up, cos down? If yes, behavior⊥geometry is a LAW of this space, not an SAE quirk.
- How: NEW SCRIPT
tae_unbind_ce.py(~80 lines): warm-start the ridge unbind map fromtae_comp_unbind.py(add--save_mapthere), fine-tune withsonar_grad.SonarDecoderCEloss on decoded ẑ_A, ẑ_B vs true A, B texts; eval cos + decode-SBERT per slot, before/after. - Runtime: 5 min + 2–4 h (CE grads through decoder are the cost).
- Surprising: CE training FAILS to improve decode here (the trade is specific to dictionary-like objects); or middle-slot/B-slot deficit closes under CE (deficit was geometric, not informational — big reframe of 1.3).
2.2 Canonicity: L2 + Matryoshka combined
- Q: Matryoshka features are more compositional (additivity R² .60 vs .38) and the coarse prefix carries big chunks; L2 aligns a core. Do nesting + L2 stack to push PW-MCC past .289 — i.e., is hierarchy itself a canonicalizing pressure?
- How: NEW VARIANT
--variant l2_matryoshkaintae_canonicity_fixoff.py(~30 lines: import the matryoshka prefix-loss fromtrain_sae_matryoshka.py, add encoder L2 1e-4; two seeds as the harness already does).python tae_canonicity_fixoff.py --variant l2_matryoshka --shards 0-15 --epochs 12 --l2 1e-4 --out tae_canon_l2matry.json - Runtime: 5 min + ~4–6 h (two trainings).
- Surprising: prefix features reach PW-MCC > .6 while tail stays .1 (canonicity is purely a coarse-scale property — clean scale-dependence statement).
2.3 From-scratch TEXT-LOSS canonicity (the open question)
- Q: Warm-start text-loss = reproducible micro-adjustment; variance-SAE seeds = disjoint. The untested cell: two FROM-SCRATCH SAEs trained with decoder-CE in the loss. If text loss canonicalizes from scratch, "wrong objective" explains non-canonicity; if not, non-canonicity is intrinsic to overcomplete dictionaries on this manifold.
- How: extend
train_sae_textloss.pywith--from_scratch --seed {0,1}(init random instead of warm ckpt), mixed lossMSE + λ·CEfor stability; then score with the fixoff matcher (NEW FLAG--match_ckpts a.pt,b.ptintae_canonicity_fixoff.py). - Runtime: 5 min + 6–10 h (CE training is slow; budget overnight, 2 boxes).
- Surprising: PW-MCC ≈ .4+ (objective was the whole problem — major, publishable); PW-MCC ≈ .15 (canonical-atoms hope dead even with the right objective).
2.4 SAE-code-space composer: does the right basis make composition LOW-rank?
- Q: Composition is high-rank in ambient space (r256 → .52). If the SAE/Matryoshka code is closer to the "true variables", the composition map in CODE space (sparse 16384-d → 16384-d) might be low-rank/sparse/near-diagonal. Deepens both composition anatomy and the dictionaries-as-frames claim; a positive result = first evidence the dictionary basis simplifies a known computation.
- How: NEW SCRIPT
tae_comp_saespace.py(~80 lines): encode pairs, get pre-activation codes c_A, c_B, c_AB (k128 + matryoshka), fit ridge [c_A;c_B]→c_AB with rank sweep + per-feature sparsity of the map; decode predicted codes through SAE decoder + SONAR for behavioral check. - Runtime: 5 min + ~2 h.
- Surprising: near-diagonal map in matryoshka space (features really are compositional variables — contradicts the 17%-survival finding in a productive way); or WORSE rank curve than ambient (dictionary actively scrambles composition).
2.5 Canonicity dose completion + archetypal-v2 sweep
- Q: Where does the L2 dose-response peak (1e-3 pending — does alignment keep rising or does FVU collapse first)? Can archetypal work with relaxed constraints (v1 collapsed, FVU 1.95)?
- How (existing flags):
python tae_canonicity_fixoff.py --variant l2reg --l2 1e-3 --shards 0-15 --epochs 12python tae_canonicity_fixoff.py --variant archetypal --relax 1.0 --p_scale 1.0 --shards 0-15 --epochs 12(collect the in-flight runs first; relaunch only if dead.) - Runtime: ~4 h each (training ×2 seeds).
- Surprising: l2 1e-3 reaches PW-MCC > .5 at FVU < .5 (canonicity is just regularization-starved); archetypal v2 works AND beats l2 (anchor-based identifiability transfers to SONAR).
2.6 Shift-SSAE canonicity with L2 (identifiability sidestep, quantified)
- Q: SSAE-LITE (shift dictionaries on paraphrase pairs) launched on iron — are shift-atoms more seed-canonical than state-atoms (theory says yes: differences strip content), and does L2 stack on top?
- How: collect
tae_ssae.json×2 seeds; NEW FLAG--l2 1e-4intae_ssae.py; match with fixoff matcher.python tae_ssae.py --shards 0-7 --n_texts 30000 --k 32 --l2 1e-4 --out tae_ssae_l2_seed1.json - Runtime: 5 min + ~3 h per seed.
- Surprising: shift-atoms ALSO disjoint (.15) — even removing content doesn't canonicalize; the frame story is total.
2.7 Transcoder feature circuits: L6 → L12 weight-based wiring
- Q: With per-layer transcoders at FVU .32, can we read feature→feature connections (virtual weights through the decoder block) and find a 2-layer circuit (e.g., "determiner feature → noun-phrase feature")? First circuit-level claim inside a TAE decoder; deepens §10 transcoder row.
- How: NEW SCRIPT
tae_tc_circuit.py(~80 lines): W_virtual = enc_L12 · (block path) · dec_L6 restricted to top-active features; rank edges by |weight|·act-frequency; decode-name endpoints with existing tc feature naming. - Runtime: 5 min + ~2 h (mostly CPU matmuls + naming decodes).
- Surprising: edge weights are dense/unstructured (transcoder features don't compose across layers — dictionary-circuit program fails at step 1).
2.8 Does the composer implement Bayes? (belief-update composition)
- Q: Wildcard-grade but mechanistically sharp: for HMM clause-sequences, z(seq1+seq2) has a posterior. Does W(z_seq1, z_seq2) predict an embedding whose decoded posterior matches the BAYES UPDATE of the two halves (vs just slot storage)? Connects composition and belief-state families.
- How: NEW SCRIPT
tae_comp_belief.py(~80 lines): generate HMM sequences, split in half, encode halves + whole; apply the trained composer; probe posterior (existing beliefstate probes) on true vs composed embedding; compare to exact filtered posterior. - Runtime: 5 min + ~1.5 h.
- Surprising: composed embedding's posterior matches the Bayes filter through the boundary (the linear composer carries belief-state updating — strong "algebra over meanings" result).
2.9 DAS with CE objective (behavior⊥geometry #5 + binding retest)
- Q: Multiclass DAS collapsed on held-out objects under cosine training. Does training the interchange through decoder-CE (behavioral target: the counterfactual TEXT) find a value-generalizing slot that geometric DAS missed? If yes, the binding info exists but off the linear-geometry axis; if no, binding truly isn't there.
- How: extend
tae_das_multiclass.pywith--loss ceusingsonar_grad(~40 lines); same held-out-object protocol as the audit-fixed version. - Runtime: 5 min + 3–5 h.
- Surprising: CE-DAS generalizes to unseen objects (rewrites the §11 "no binding variable" claim — the variable is behaviorally- but not geometrically-linear).
TIER 3 — LOW VALUE, SHORT (gap-fillers when a box frees up mid-wave)
3.1 Cross-domain composition pairs
- Q: Random cross-paragraph pairs already work (.8745). Extreme domain mixes (code-comment + recipe, legal + chat)? Marginal: only closes a residual gap.
- How:
python tae_composition.py --shards 0,7 --random_pairs --n_pairs 8000 --out tae_comp_xdomain.jsonplus NEW FLAG--domain_filterif shards are domain-tagged. - Runtime: ~30 min. Surprising: any domain pair below .80.
3.2 Length-extreme composition (slot dominance vs length)
- Q: Does slot-A dominance (.776 vs .547) flip when A is 5 words and B is 60? Disambiguates "slot-A privileged" vs "longer/first content wins".
- How: NEW FLAG
--len_bucket A_short_B_long|A_long_B_shortintae_comp_anatomy.py(filter pairs). - Runtime: ~40 min. Surprising: dominance follows LENGTH not slot (anatomy row needs rewrite — would actually promote this to Tier 1 retroactively).
3.3 Paraphrase composition: W(z_A, z_A′) = amplification?
- Q: Composing a sentence with its own paraphrase — does the operator produce "A. A′." (dutiful concat), a single amplified A (idempotency), or degenerate repetition?
- How: NEW FLAG
--paraphrase_pairsintae_composition.py(A′ = decode(encode(A)+0.2·RMS noise), reuse existing machinery). - Runtime: ~40 min. Surprising: near-idempotency (W(z,z′) ≈ scaled z) — composition has a fixed-point structure.
3.4 Unbind OOD: train on natural pairs, test on random pairs
- How: NEW FLAG
--random_pairs(test split only) intae_comp_unbind.py. Runtime: ~40 min. - Surprising: recovery drops a lot (unbinding exploits continuation priors, unlike the forward composer which didn't).
3.5 Lang-inert at distant scripts + extreme dose
- Q: Inertness shown for French ×8. Hold for ja/zh/ar at ×16/×32?
- How:
python tae_lang_inert.py --shards 0 --n_texts 400 --amps 8,16,32 --out tae_lang_inert_distant.json+ NEW FLAG--langs jpn_Jpan,zho_Hans,arb_Arab. - Runtime: ~40 min. Surprising: ANY script flip (inertness has a dose ceiling).
3.6 Decode the belief-simplex vertices
- Q: What text do the 3 fitted vertices produce? Pure-topic boilerplate would visualize the geometry.
- How: NEW FLAG
--decode_verticesintae_belief_barycentric.py(~10 lines). - Runtime: ~25 min. Surprising: vertices decode to NON-topic text (vertices are abstract anchors, not topic prototypes).
3.7 Eigen-angle spectrum of W_A (rotation structure)
- Q: Complex eigenvalues of the slot-A block: clustered rotation angles (structured role binding, HRR-like) or Haar-like spread?
- How: NEW FLAG
--eigintae_comp_anatomy.py(np.linalg.eig on the saved block, plot |λ| and angles). - Runtime: ~30 min (mostly the existing fit). Surprising: discrete angle clusters (a literal phase code).
3.8 K vs V read-subspace asymmetry in the decoder
- Q: Weight-SVD treated cross-attn K/V together (eff-rank 500–650). Does K (addressing) read a different/lower-rank subspace than V (content payload)? Sharpens the "reads broadly off-principal" claim.
- How: NEW FLAG
--split_kvintae_weight_svd.py(report per-projection eff-rank, PC-overlap, K∩V subspace principal angles).python tae_weight_svd.py --layers 0,6,12,18,23 --split_kv --out tae_weight_svd_kv.json - Runtime: ~30 min. Surprising: K is LOW-rank (≤32) while V is full (addressing by a small index code = an attention-circuit foothold).
3.9 Weight-SVD rank trend vs transcoder FVU (CPU analysis)
- Q: Do layers whose K/V read-subspaces overlap data PCs more have lower transcoder FVU? Links the two decoder findings into one "linearity profile".
- How: NEW SCRIPT
tae_decoder_profile.py(~40 lines, joins tae_weight_svd.json + tae_tc_* FVUs, Spearman). - Runtime: minutes, no GPU. Surprising: anti-correlation (hard layers read MORE on-PC — would tangle the tail story).
3.10 Single-letter attractors: nearest-token-embedding hypothesis
- Q: Is each feature's collapse letter (f, n, e, k…) predicted by the nearest decoder token-embedding to the feature direction? Demystifies the dose-census attractors.
- How: NEW SCRIPT
tae_attractor_map.py(~50 lines: cos(feature dir, decoder embedding rows for single-char tokens); compare to observed attractor letters in tae_dose_response.json). - Runtime: ~20 min. Surprising: no correspondence (attractors are dynamical, not embedding-proximal).
3.11 Compose-unbind round-trip Lyapunov
- Q: Iterate compose∘unbind: do (ẑ_A, ẑ_B) fixed-points exist, and do they keep decoding to A and B? Measures internal consistency of the learned algebra.
- How: NEW FLAG
--roundtrip_iters 3intae_comp_unbind.py(needs the saved forward map from 1.1's--save_map). - Runtime: ~40 min. Surprising: divergence by iter 2 (the two maps aren't mutually consistent inverses — each overfits its own direction).
TIER 4 — LOW VALUE, LONG (only if the fleet is otherwise idle)
4.1 Full canonicity grid: L2 × width × k × matryoshka
- Q: Map the whole recipe space for PW-MCC. Mostly confirmatory after 2.2/2.5.
- How: loop
tae_canonicity_fixoff.pyover--l2 {1e-5,1e-4,1e-3} --k {32,128} --n_feats {8192,16384}. Runtime: ~24 box-hours. Surprising: an interaction (e.g. L2 only works at low k).
4.2 12-language fingerprint × translation-quality matrix
- Q: Per language pair: fingerprint magnitude vs re-encode tax vs decode quality (does a bigger fingerprint predict worse round-trips? ja was worst on both). Diagnostic table, not a new mechanism.
- How: extend
tae_lang_decode.pywith the comp_language classifier (NEW FLAG--fingerprint); 12 langs × 600 texts. Runtime: ~4 h (decode-heavy). Surprising: zero correlation (fingerprint unrelated to quality — strengthens "vestigial").
4.3 Cross-encoder composer transfer (platonic composition)
- Q: Map MiniLM pairs into SONAR via the ridge aligner, apply the SONAR composer, map back: does composition transfer at the same .47 gist level as decoding? Tests whether the operator is about SONAR or about language.
- How: NEW SCRIPT on top of
tae_align_spaces.pyartifacts (~80 lines). Runtime: ~2 h + MiniLM encode. Surprising: composition transfers BETTER than content (.47) — operator structure is more universal than the embedding.
4.4 Unbinding capacity curve to depth 8
- Q: Full primacy/recency curve shape at depths 6–8 (after 1.3 establishes depth 4–5). How:
tae_comp_unbind3.py --depth 6/8 --shards 0-11. Runtime: ~2 h each (long chains, fewer per shard). Surprising: recovery floor stays above similarity-baseline at depth 8 (enormous capacity).
4.5 ICA rescue attempts (whitened/deflation/sparse-PCA variants)
- Q: §12 ICA negative used the standard recipe. Variants might converge — but the .31 stability ceiling suggests little upside.
- How: extend
tae_ica_axes.pywith--algo deflation --whiten_dim 256etc. Runtime: ~3 h sweep. Surprising: any variant beating SAE stability AND nameability.
4.6 Probe-with-CE for belief states (behavior⊥geometry #6)
- Q: Train the z→posterior probe so that decode(z edited to move the probe) changes topic mixture accordingly — a causal-belief probe. Conceptually rich but heavy and the read/write gap predicts failure.
- How: NEW SCRIPT combining
tae_beliefstate.py+sonar_grad(~90 lines, slightly over budget). Runtime: ~4 h. Surprising: success (a READ direction that also WRITES — would contradict the read⊥write finding in the one place geometry is simplex-clean).
5 WILDCARDS (already woven in above, listed for visibility)
- W1 = 2.4 SAE-space composer (right basis ⇒ low-rank composition?).
- W2 = 2.8 Does the composer implement Bayes updating on belief states?
- W3 = 1.11 Event-order vs surface-order canonicalization ("Before that," templates).
- W4 = 3.10 Attractor letters = nearest token embeddings?
- W5 = 3.11 / 1.2 The compose/unbind pair as a dynamical system (mutual-inverse consistency under iteration and noise).
Suggested first wave (8 boxes, tonight)
1.1 joiners (2 boxes) · 1.2 noised-unbind · 1.3 depth-4/5 unbind · 1.5 emit-dose sweep · 1.6 4-state · 1.7 attr-adders · 1.8 human-fingerprint · 1.10 xlingual-align; launch 2.1 (unbind-CE) and 2.2 (l2+matryoshka) on the two strongest boxes in parallel.