README.md

# bsf_071226 — Load-Balanced Block-Sparse Featurizer, Qwen3-8B layer 24

BSF with dead-zone usage-bias selection (G=2048 blocks x b=64 dims, 131k code dims =
32x expansion at d=4096; per-token top-k=16 blocks by ||z_g|| + tau_g). The per-block
selection bias tau is trained by a dead-zone controller (boost blocks firing < 1e-4,
handicap > 50%, decay between; step/clamp calibrated in k-th-place-norm units) and is
STORED IN THE CHECKPOINT (`selection_bias` buffer) — the same biased ranking runs at
inference, exactly 16 blocks/token. Signed codes, pure MSE, denoising snr=0.1,
~100M tokens (FineFineWeb).

## Results
- reconstruction FVE 0.848 (train-final; rank-1 SAE d65k k80 = 0.803)
- population: 1,927/2,048 blocks deployed-alive; 5 generalists >50% / 48 mid-band
  (5-50%) / ~270 specialists / ~1,700 rare tail held at ~1e-4; top block 9.3% of energy
- autointerp (delphi, Opus judge; n=1,436 = whole population): detection 0.809 /
  fuzzing 0.824 per-block mean; 841 blocks >= 0.8 detection
- usage-weighted (per-token) view: detection 0.665 / fuzzing 0.601 — per-block means
  favor rare crisp blocks; both views reported for honesty

## Loading
Requires the nla repo's `SAE/bsf_sae.py` (the `selection_bias` buffer is new):
`BlockSparseSAE.from_pretrained("ae.pt")` — group_sizes/k/act_scale/tau load from the
state dict. Trained 2026-07-11, run `bsf_stream_L24_G2048_b64_k16_biasdz`
(wandb nla-sae k1abto0r).