README.md

bsf_070926 — Block-Sparse Featurizer, Qwen3-8B layer 24

Vanilla BSF (Fel et al. 2026, arXiv:2606.25234): G=4096 blocks x b=64 dims (262,144 code dims, 64x expansion at d=4096), per-token top-k=16 blocks by code L2 norm, signed codes, pure MSE, denoising snr=0.1, ~100M tokens (FineFineWeb).

• FVE 0.865 (held-out, rl_shuf 20k acts)
• autointerp (delphi, Opus judge): detection 0.717 / fuzz 0.708, ~22% of scored blocks >= 0.8
• 362 live blocks; firing rate of live blocks p10/p50/p90 = 0.16% / 1.16% / 6.7% (~8 always-on generalists carry ~half the energy)

Load with SAE/bsf_sae.py::BlockSparseSAE.from_pretrained("ae.pt") from the nla repo. Trained 2026-07-09, run bsf_stream_L24_G4096_b64_k16 (wandb nla-sae).