exp007_amoe_lora/README.md

exp007_amoe_lora — AMoE-LoRA v1: the first dispatch test (CANDIDATE, s0)

Question. Does a bank of experts under dense signed aleph dispatch (keyed on hidden state + sigma) beat a parameter-matched monolith on the certified core-epred substrate? First MoE-structure experiment of the line.

Structure per site (16 sites). A=4 zero-init rank-8 low-rank residual experts; dispatch = dense SIGNED aleph coefficients per token (sinh/cosh over A oriented axes — no softmax gate, no top-k, no balance loss; usage read-only) with sigma Fourier features added to the dispatch key (the exp003 finding: sigma is the live register axis). Combine: x + Σ σ(gate_k)·w_k(x,σ)·Δ_k(x). Falsifier: one rank-32 delta per site, same placement, no dispatch (the matched-monolith control). 3000 steps each, batch 16, pure Adam wd=0.

Results (results.json):

arm	trainable	val eps-MSE	Δ vs frozen
frozen	0	0.124788	—
amoe A4r8	899,392	0.122015	−2.2%
monolith r32	811,216	0.121777	−2.4%

prereg	outcome
P1 bank ≥ monolith	MISS (monolith wins by 0.0002 with 11% fewer params)
P2 sigma-band usage differentiation	MISS — spread 0.0009; usage ≈ [0.25, 0.17, 0.33, 0.24] in EVERY band
P3 toggle bit-exact post-train	HIT (≡ frozen to full precision)
P4 no dead experts	HIT (all four alive, 0.17–0.33)

Verdict (the falsifier did its job). Under UNIFORM training pressure — one dataset, one objective, one register — dispatch pays nothing over a single matched-width delta, and the sigma-keyed dispatch never uses its sigma freedom. This reproduces the text line's Tree-1b/exp009 law on a diffusion trunk: specialization requires differentiated pressure; the aggregation channel alone doesn't create it. The experts blended (uniform usage) exactly as the two-regime law predicts for register-distance-zero mixtures.

What it buys the program. The machinery is sound (parity, liveness, signed dense dispatch all green) and per-param the bank units are efficient (−2.2% at 0.9M vs the relay's −2.5% at 3.2M). The next MoE test must IMPOSE structure rather than hope dispatch discovers it in uniform data — the designed follow-on is the step-gated controller: experts ASSIGNED to sigma bands with per-band loss masking (coarse-to-fine), judged against this same monolith. Subject-bucketed expert assignment is the second axis (needs the richer fused datasets; this synthetic set's subject buckets max out at n≈6).

Caveats. Single seed; 3k steps; A=4/r=8 single scale point; param match 1.109 (bank heavier — disclosed); one substrate. Cost ≈ 3.0 GPU-h. Checkpoints: amoe_core_s0.pt, mono_core_s0.pt.

---

SEED-1 REPLICATION (2026-07-17, results_s1.json) — FALSIFIER ROBUST, 2 SEEDS

monolith 0.121799 vs amoe 0.122011 (s0: 0.121777 vs 0.122015) — the monolith ties/wins in BOTH seeds; band-usage spread stays flat (0.0039 vs 0.0009). Uniform-pressure ⇒ dispatch-pays-nothing is seed-robust at this scale.