exp007_amoe_lora/README.md

# exp007_amoe_lora — AMoE-LoRA v1: the first dispatch test (CANDIDATE, s0)

**Question.** Does a bank of experts under dense signed aleph dispatch (keyed
on hidden state + sigma) beat a parameter-matched monolith on the certified
core-epred substrate? First MoE-structure experiment of the line.

**Structure per site (16 sites).** A=4 zero-init rank-8 low-rank residual
experts; dispatch = dense SIGNED aleph coefficients per token (sinh/cosh over
A oriented axes — no softmax gate, no top-k, no balance loss; usage read-only)
with sigma Fourier features added to the dispatch key (the exp003 finding:
sigma is the live register axis). Combine: `x + Σ σ(gate_k)·w_k(x,σ)·Δ_k(x)`.
Falsifier: one rank-32 delta per site, same placement, no dispatch (the
matched-monolith control). 3000 steps each, batch 16, pure Adam wd=0.

**Results** (`results.json`):

| arm | trainable | val eps-MSE | Δ vs frozen |
|---|---|---|---|
| frozen | 0 | 0.124788 | — |
| amoe A4r8 | 899,392 | 0.122015 | −2.2% |
| **monolith r32** | 811,216 | **0.121777** | **−2.4%** |

| prereg | outcome |
|---|---|
| P1 bank ≥ monolith | **MISS** (monolith wins by 0.0002 with 11% fewer params) |
| P2 sigma-band usage differentiation | **MISS** — spread 0.0009; usage ≈ [0.25, 0.17, 0.33, 0.24] in EVERY band |
| P3 toggle bit-exact post-train | HIT (≡ frozen to full precision) |
| P4 no dead experts | HIT (all four alive, 0.17–0.33) |

**Verdict (the falsifier did its job).** Under UNIFORM training pressure — one
dataset, one objective, one register — dispatch pays nothing over a single
matched-width delta, and the sigma-keyed dispatch never uses its sigma
freedom. This reproduces the text line's Tree-1b/exp009 law on a diffusion
trunk: *specialization requires differentiated pressure; the aggregation
channel alone doesn't create it.* The experts blended (uniform usage) exactly
as the two-regime law predicts for register-distance-zero mixtures.

**What it buys the program.** The machinery is sound (parity, liveness,
signed dense dispatch all green) and per-param the bank units are efficient
(−2.2% at 0.9M vs the relay's −2.5% at 3.2M). The next MoE test must IMPOSE
structure rather than hope dispatch discovers it in uniform data — the
designed follow-on is the step-gated controller: experts ASSIGNED to sigma
bands with per-band loss masking (coarse-to-fine), judged against this same
monolith. Subject-bucketed expert assignment is the second axis (needs the
richer fused datasets; this synthetic set's subject buckets max out at n≈6).

**Caveats.** Single seed; 3k steps; A=4/r=8 single scale point; param match
1.109 (bank heavier — disclosed); one substrate. Cost ≈ 3.0 GPU-h.
Checkpoints: `amoe_core_s0.pt`, `mono_core_s0.pt`.

---
## SEED-1 REPLICATION (2026-07-17, `results_s1.json`) — FALSIFIER ROBUST, 2 SEEDS
monolith 0.121799 vs amoe 0.122011 (s0: 0.121777 vs 0.122015) — the monolith
ties/wins in BOTH seeds; band-usage spread stays flat (0.0039 vs 0.0009).
Uniform-pressure ⇒ dispatch-pays-nothing is seed-robust at this scale.