exp014 s1: router v1 negative 2-seed (ordering + both instrument findings replicate)
552f3b1 verified | # exp014_te_dispatch β the router-solidifier v1 (CANDIDATE s0; s1 running) | |
| **Question.** Does text-side addressing in the DISPATCH KEY (a trained | |
| text-encoder adapter, or the frozen canonical byte-trigram address) route | |
| the expert bank better than state+sigma keys alone? | |
| **Arms** (flow trunk, fused cache, A=4 r=8 banks, dense signed dispatch): | |
| sigma_key (control) | addr_key (frozen [32,128] address flattened β key) | | |
| te_key (trained CLIP-pooler adapter β key). | |
| **Results** (`results.json`): | |
| | arm | val flow-MSE | per-prompt usage var | shuffled-key null | | |
| |---|---|---|---| | |
| | sigma_key | **0.57371** | 2.36e-02 | 0 (no text) | | |
| | addr_key | 0.57755 | **1.40e-05 (dead)** | 1.40e-05 | | |
| | te_key | 0.57422 | 2.60e-02 | 2.71e-02 | | |
| **Verdict (honest negative + two instrument findings).** | |
| 1. P1 MISS: no val win for text keys at this scale (all within 0.07%). | |
| 2. **The hidden-state key already routes by prompt** (2.36e-02 usage | |
| variance with zero text input) β prompt-level structure reaches the | |
| usage surface through x itself; exp007's flatness was across SIGMA | |
| BANDS, a different axis. The TE key adds ~nothing on top. | |
| 3. **The raw-flattened address kills routing** (1.4e-05) and dents val β | |
| reproducing the text line's documented disease geometry (addressing raw | |
| high-D flattens the oriented softmax; healthy forms bottleneck to low-D | |
| FIRST). The canonical address needs a structured reduction (slots/MΜ | |
| read) before it can serve as a router key β v2's design, not a defect | |
| of the address itself. | |
| 4. **Confessed instrument gap:** the shuffled-key null measures key | |
| DIVERSITY, not routing CORRECTNESS (te null β te real). v2's null: | |
| repeated-key, plus matched-vs-mismatched key val deltas. | |
| **Caveats.** s0 (s1 running); one scale (A=4, 3k steps); pooler-only TE | |
| feature; flow trunk only. Cost β 4.6 GPU-h incl. text-feature build. | |
| --- | |
| ## SEED-1 (2026-07-19, `results_s1.json`) β 2-SEED: the ordering replicates | |
| sigma 0.57219 < te 0.57468 < addr 0.57769 (same order both seeds); addr | |
| routing dead both seeds (5.6e-06/1.4e-05); state-key prompt variance | |
| present both seeds (2.9e-02/2.4e-02). Router v1's negative and both | |
| instrument findings are seed-robust. v2 = MΜ-bottlenecked address key + | |
| repeated-key null. | |