token-channel/README.md

# Token-Channel Program — drafting heads

Speculative-decoding drafting heads for Qwen3-8B, placed in the 2×2 design space they test.
All trained on the same 800k data (nemotron-nothink). Metric τ (bonus-inclusive), btop rail, n=100.

## The 2×2 — channel × backbone

|                    | **markov** (rank-256 bigram, logit-level) | **token layer** (feature-level transformer layer) |
|--------------------|-------------------------------------------|---------------------------------------------------|
| **frozen** base    | `A_frozen-markov/`  — helps only slot 0, ~0 net. tree inert. | `B_frozen-token/`  — +0.24 all four, converges epoch 0. tree inert. |
| **co-trained** base| `E_cotrain-markov/`  ★ **the shape that works** — chain +0.10 but the TREE recovers the full delegation (+2.07); the only co-trained head with a LIVE tree. | `D_cotrain-token/`  — strong chain, but a full layer per tree node is too expensive → tree DEAD. |

## Controls & variants
- `5p0_cotrain-none/`  — co-trained base, NO channel. The origin D and E subtract against (τ 5.27 @ s5000).
- `Bx2_frozen-token-x2/`  — frozen base + TWO layers. +0.22 = one layer's level → the base is the ceiling, not capacity.
- `C_hybrid/`  — mask AND token inputs co-trained (self-forcing, block-15). One ckpt for tree AND chain. Still training.

## Reference heads (`ref/`) — comparison anchors, not our arms
- `ref/dspark-b7`, `ref/dspark-b15`  — bidirectional + markov, our 800k reproduction (DeepSeek/PKU design). b15 head-to-head: chain 8.75 / tree 11.62 math500; its base alone is a half-product (bare tree 4.28).
- `ref/b7bb`  — DSpark-arch 800k (slot-switch experiment).

## Verdict
A frozen backbone is a CEILING (B-x2 = B). The winning shape is a co-trained base + a corrector
cheap enough to stay present at EVERY tree node (E proves it; D's layer-cost corrector cannot).

Each `<arm>/s<step>/` folder holds one checkpoint: config.json · dflash.py · model.safetensors ·
target_release_weights.safetensors (+ dspark_heads.py for markov arms).