AbstractPhil's picture
exp014 s1: router v1 negative 2-seed (ordering + both instrument findings replicate)
552f3b1 verified
|
Raw
History Blame Contribute Delete
2.27 kB

exp014_te_dispatch — the router-solidifier v1 (CANDIDATE s0; s1 running)

Question. Does text-side addressing in the DISPATCH KEY (a trained text-encoder adapter, or the frozen canonical byte-trigram address) route the expert bank better than state+sigma keys alone?

Arms (flow trunk, fused cache, A=4 r=8 banks, dense signed dispatch): sigma_key (control) | addr_key (frozen [32,128] address flattened → key) | te_key (trained CLIP-pooler adapter → key).

Results (results.json):

arm val flow-MSE per-prompt usage var shuffled-key null
sigma_key 0.57371 2.36e-02 0 (no text)
addr_key 0.57755 1.40e-05 (dead) 1.40e-05
te_key 0.57422 2.60e-02 2.71e-02

Verdict (honest negative + two instrument findings).

  1. P1 MISS: no val win for text keys at this scale (all within 0.07%).
  2. The hidden-state key already routes by prompt (2.36e-02 usage variance with zero text input) — prompt-level structure reaches the usage surface through x itself; exp007's flatness was across SIGMA BANDS, a different axis. The TE key adds ~nothing on top.
  3. The raw-flattened address kills routing (1.4e-05) and dents val — reproducing the text line's documented disease geometry (addressing raw high-D flattens the oriented softmax; healthy forms bottleneck to low-D FIRST). The canonical address needs a structured reduction (slots/M̂ read) before it can serve as a router key — v2's design, not a defect of the address itself.
  4. Confessed instrument gap: the shuffled-key null measures key DIVERSITY, not routing CORRECTNESS (te null ≈ te real). v2's null: repeated-key, plus matched-vs-mismatched key val deltas.

Caveats. s0 (s1 running); one scale (A=4, 3k steps); pooler-only TE feature; flow trunk only. Cost ≈ 4.6 GPU-h incl. text-feature build.


SEED-1 (2026-07-19, results_s1.json) — 2-SEED: the ordering replicates

sigma 0.57219 < te 0.57468 < addr 0.57769 (same order both seeds); addr routing dead both seeds (5.6e-06/1.4e-05); state-key prompt variance present both seeds (2.9e-02/2.4e-02). Router v1's negative and both instrument findings are seed-robust. v2 = MÌ‚-bottlenecked address key + repeated-key null.