AbstractPhil's picture
exp014 s1: router v1 negative 2-seed (ordering + both instrument findings replicate)
552f3b1 verified
|
Raw
History Blame Contribute Delete
2.27 kB
# exp014_te_dispatch β€” the router-solidifier v1 (CANDIDATE s0; s1 running)
**Question.** Does text-side addressing in the DISPATCH KEY (a trained
text-encoder adapter, or the frozen canonical byte-trigram address) route
the expert bank better than state+sigma keys alone?
**Arms** (flow trunk, fused cache, A=4 r=8 banks, dense signed dispatch):
sigma_key (control) | addr_key (frozen [32,128] address flattened β†’ key) |
te_key (trained CLIP-pooler adapter β†’ key).
**Results** (`results.json`):
| arm | val flow-MSE | per-prompt usage var | shuffled-key null |
|---|---|---|---|
| sigma_key | **0.57371** | 2.36e-02 | 0 (no text) |
| addr_key | 0.57755 | **1.40e-05 (dead)** | 1.40e-05 |
| te_key | 0.57422 | 2.60e-02 | 2.71e-02 |
**Verdict (honest negative + two instrument findings).**
1. P1 MISS: no val win for text keys at this scale (all within 0.07%).
2. **The hidden-state key already routes by prompt** (2.36e-02 usage
variance with zero text input) β€” prompt-level structure reaches the
usage surface through x itself; exp007's flatness was across SIGMA
BANDS, a different axis. The TE key adds ~nothing on top.
3. **The raw-flattened address kills routing** (1.4e-05) and dents val β€”
reproducing the text line's documented disease geometry (addressing raw
high-D flattens the oriented softmax; healthy forms bottleneck to low-D
FIRST). The canonical address needs a structured reduction (slots/MΜ‚
read) before it can serve as a router key β€” v2's design, not a defect
of the address itself.
4. **Confessed instrument gap:** the shuffled-key null measures key
DIVERSITY, not routing CORRECTNESS (te null β‰ˆ te real). v2's null:
repeated-key, plus matched-vs-mismatched key val deltas.
**Caveats.** s0 (s1 running); one scale (A=4, 3k steps); pooler-only TE
feature; flow trunk only. Cost β‰ˆ 4.6 GPU-h incl. text-feature build.
---
## SEED-1 (2026-07-19, `results_s1.json`) β€” 2-SEED: the ordering replicates
sigma 0.57219 < te 0.57468 < addr 0.57769 (same order both seeds); addr
routing dead both seeds (5.6e-06/1.4e-05); state-key prompt variance
present both seeds (2.9e-02/2.4e-02). Router v1's negative and both
instrument findings are seed-robust. v2 = MΜ‚-bottlenecked address key +
repeated-key null.