Couple the model card to the paper: correct title, filled Gemma/Qwen metrics from Appendices C-D, fidelity-diagnostics register, fix the hf_hub_download namespace
Browse files
README.md
CHANGED
|
@@ -18,48 +18,52 @@ base_model:
|
|
| 18 |
- mistralai/Ministral-3-8B-Base-2512
|
| 19 |
---
|
| 20 |
|
| 21 |
-
# Sparse Readout Prism β pretrained readout
|
| 22 |
|
| 23 |
-
Pretrained dictionaries for **Sparse Readout Prism**, which
|
| 24 |
-
model's unembedding matrix
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
|
| 27 |
```
|
| 28 |
h Β· W_U[token] β base + Ξ£_i z_i (h Β· d_i) + residual
|
| 29 |
```
|
| 30 |
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
|
| 36 |
- **Code:** https://github.com/hematteo/sparse-readout-prism
|
| 37 |
-
- **Paper:** *Sparse Readout Prism:
|
|
|
|
| 38 |
|
| 39 |
## Operating points
|
| 40 |
|
| 41 |
-
Most base models ship two dictionaries
|
| 42 |
-
k = 256) and a **strict-budget** point (`k128`, 16Γ width β 8Γ for
|
| 43 |
-
k = 128).
|
| 44 |
-
three. The exact width of each is in the `width` column
|
|
|
|
| 45 |
|
| 46 |
## Checkpoints
|
| 47 |
|
| 48 |
Layout: `<model>/<operating_point>/checkpoint.pt`.
|
| 49 |
|
| 50 |
-
| Path | base model | width | d_features | k | rowEV |
|
| 51 |
|---|---|---|---:|---:|---:|---:|---:|
|
| 52 |
-
| `qwen3.5-0.8b/k128_16x` | Qwen/Qwen3.5-0.8B | 16Γ | 16384 | 128 | 0.760 | 0.844 |
|
| 53 |
-
| `qwen3.5-0.8b/k256_32x` | Qwen/Qwen3.5-0.8B | 32Γ | 32768 | 256 | 0.877 | 0.891 |
|
| 54 |
-
| `qwen3.5-2b/k128_16x` | Qwen/Qwen3.5-2B | 16Γ | 32768 | 128 | 0.712 | 0.858 |
|
| 55 |
-
| `qwen3.5-2b/k256_32x` | Qwen/Qwen3.5-2B | 32Γ | 65536 | 256 | 0.847 | 0.887 |
|
| 56 |
| `qwen3.5-9b/k128_8x` | Qwen/Qwen3.5-9B | 8Γ | 32768 | 128 | 0.621 | 0.846 | 0.296 |
|
| 57 |
| `qwen3.5-9b/k256_16x` | Qwen/Qwen3.5-9B | 16Γ | 65536 | 256 | 0.761 | 0.874 | 0.167 |
|
| 58 |
| `qwen3.5-9b/k256_32x` | Qwen/Qwen3.5-9B | 32Γ | 131072 | 256 | 0.857 | 0.900 | 0.105 |
|
| 59 |
-
| `gemma-4-e2b/k128_16x` | google/gemma-4-E2B-it | 16Γ | 24576 | 128 |
|
| 60 |
-
| `gemma-4-e2b/k256_32x` | google/gemma-4-E2B-it | 32Γ | 49152 | 256 |
|
| 61 |
-
| `gemma-4-e4b/k128_16x` | google/gemma-4-E4B-it | 16Γ | 40960 | 128 |
|
| 62 |
-
| `gemma-4-e4b/k256_32x` | google/gemma-4-E4B-it | 32Γ | 81920 | 256 |
|
| 63 |
| `ministral-3-8b/k128_16x` | mistralai/Ministral-3-8B-Base-2512 | 16Γ | 65536 | 128 | 0.806 | 0.885 | 0.130 |
|
| 64 |
| `ministral-3-8b/k256_32x` | mistralai/Ministral-3-8B-Base-2512 | 32Γ | 131072 | 256 | 0.888 | 0.904 | 0.087 |
|
| 65 |
| `r1-distill-qwen-7b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Qwen-7B | 16Γ | 57344 | 128 | 0.709 | 0.695 | 0.777 |
|
|
@@ -67,54 +71,58 @@ Layout: `<model>/<operating_point>/checkpoint.pt`.
|
|
| 67 |
| `r1-distill-llama-8b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 16Γ | 65536 | 128 | 0.796 | 0.725 | 0.536 |
|
| 68 |
| `r1-distill-llama-8b/k256_32x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 32Γ | 131072 | 256 | 0.888 | 0.754 | 0.434 |
|
| 69 |
|
| 70 |
-
`rowEV`
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
the paper).
|
|
|
|
| 76 |
|
| 77 |
## How they were trained
|
| 78 |
|
| 79 |
-
TopK factorizer on the
|
| 80 |
-
finalist
|
| 81 |
-
penalty `lambda_prism = 1e-3` with a delayed
|
| 82 |
-
50% uniform) row sampling,
|
| 83 |
-
|
|
|
|
| 84 |
|
| 85 |
## Usage
|
| 86 |
|
| 87 |
```python
|
| 88 |
from huggingface_hub import hf_hub_download
|
| 89 |
-
from sparse_readout_prism import load_factorizer # pip install -e . from the GitHub repo
|
| 90 |
|
| 91 |
-
path = hf_hub_download("
|
| 92 |
sae = load_factorizer(path, freeze=True) # rebuild + load_state_dict + eval, one call
|
| 93 |
```
|
| 94 |
|
| 95 |
-
Each `checkpoint.pt` is a `weights_only=True`-loadable dict with
|
| 96 |
-
(encoder / decoder / biases) and the factorizer config
|
| 97 |
-
`d_features`)
|
| 98 |
-
`factorizer` or `config.factorizer`. To
|
| 99 |
-
|
| 100 |
-
`preprocess_rows(W_U)` (the
|
| 101 |
-
|
| 102 |
-
|
|
|
|
|
|
|
| 103 |
|
| 104 |
## Intended use & limitations
|
| 105 |
|
| 106 |
Research artifact for mechanistic interpretability of the **final readout**.
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
|
| 113 |
## Citation
|
| 114 |
|
| 115 |
```bibtex
|
| 116 |
@misc{he2026sparsereadoutprism,
|
| 117 |
-
title = {Sparse Readout Prism:
|
| 118 |
author = {He, Matteo and Shen, William F. and Qiu, Xinchi and Lane, Nicholas D.},
|
| 119 |
year = {2026},
|
| 120 |
note = {Preprint forthcoming; see the repository for the up-to-date reference},
|
|
|
|
| 18 |
- mistralai/Ministral-3-8B-Base-2512
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# Sparse Readout Prism β pretrained dictionaries of readout features
|
| 22 |
|
| 23 |
+
Pretrained dictionaries for **Sparse Readout Prism** (SRP), which decomposes
|
| 24 |
+
a language model's readout (the unembedding matrix `W_U`) using only its
|
| 25 |
+
weights. SRP factorizes the unembedding rows into a dictionary of reusable
|
| 26 |
+
*readout features* and expresses any token logit or logit difference as a sum
|
| 27 |
+
of signed feature contributions plus an explicit residual
|
| 28 |
|
| 29 |
```
|
| 30 |
h Β· W_U[token] β base + Ξ£_i z_i (h Β· d_i) + residual
|
| 31 |
```
|
| 32 |
|
| 33 |
+
Fidelity diagnostics reported with every decomposition measure how well the
|
| 34 |
+
fitted rows stand in for the originals. The dictionaries here are trained on
|
| 35 |
+
the rows of the final readout `W_U`, which distinguishes them from
|
| 36 |
+
residual-stream SAEs trained on activations.
|
| 37 |
|
| 38 |
- **Code:** https://github.com/hematteo/sparse-readout-prism
|
| 39 |
+
- **Paper:** *Sparse Readout Prism: Explaining Logit-Lens Scores in Features
|
| 40 |
+
Instead of Tokens* (preprint forthcoming)
|
| 41 |
|
| 42 |
## Operating points
|
| 43 |
|
| 44 |
+
Most base models ship two dictionaries, a **high-fidelity** point (`k256`,
|
| 45 |
+
32Γ width, k = 256) and a **strict-budget** point (`k128`, 16Γ width β 8Γ for
|
| 46 |
+
Qwen3.5-9B β k = 128). Qwen3.5-9B additionally ships a 16Γ/`k256` capacity
|
| 47 |
+
point, so it has three. The exact width of each is in the `width` column
|
| 48 |
+
below.
|
| 49 |
|
| 50 |
## Checkpoints
|
| 51 |
|
| 52 |
Layout: `<model>/<operating_point>/checkpoint.pt`.
|
| 53 |
|
| 54 |
+
| Path | base model | width | d_features | k | rowEV | top-1 | KL (bits) |
|
| 55 |
|---|---|---|---:|---:|---:|---:|---:|
|
| 56 |
+
| `qwen3.5-0.8b/k128_16x` | Qwen/Qwen3.5-0.8B | 16Γ | 16384 | 128 | 0.760 | 0.844 | 0.277 |
|
| 57 |
+
| `qwen3.5-0.8b/k256_32x` | Qwen/Qwen3.5-0.8B | 32Γ | 32768 | 256 | 0.877 | 0.891 | 0.135 |
|
| 58 |
+
| `qwen3.5-2b/k128_16x` | Qwen/Qwen3.5-2B | 16Γ | 32768 | 128 | 0.712 | 0.858 | 0.261 |
|
| 59 |
+
| `qwen3.5-2b/k256_32x` | Qwen/Qwen3.5-2B | 32Γ | 65536 | 256 | 0.847 | 0.887 | 0.136 |
|
| 60 |
| `qwen3.5-9b/k128_8x` | Qwen/Qwen3.5-9B | 8Γ | 32768 | 128 | 0.621 | 0.846 | 0.296 |
|
| 61 |
| `qwen3.5-9b/k256_16x` | Qwen/Qwen3.5-9B | 16Γ | 65536 | 256 | 0.761 | 0.874 | 0.167 |
|
| 62 |
| `qwen3.5-9b/k256_32x` | Qwen/Qwen3.5-9B | 32Γ | 131072 | 256 | 0.857 | 0.900 | 0.105 |
|
| 63 |
+
| `gemma-4-e2b/k128_16x` | google/gemma-4-E2B-it | 16Γ | 24576 | 128 | 0.714 | 0.623 | 1.94 |
|
| 64 |
+
| `gemma-4-e2b/k256_32x` | google/gemma-4-E2B-it | 32Γ | 49152 | 256 | 0.834 | 0.333 | 6.37 |
|
| 65 |
+
| `gemma-4-e4b/k128_16x` | google/gemma-4-E4B-it | 16Γ | 40960 | 128 | 0.693 | 0.669 | 1.82 |
|
| 66 |
+
| `gemma-4-e4b/k256_32x` | google/gemma-4-E4B-it | 32Γ | 81920 | 256 | 0.827 | 0.736 | 1.22 |
|
| 67 |
| `ministral-3-8b/k128_16x` | mistralai/Ministral-3-8B-Base-2512 | 16Γ | 65536 | 128 | 0.806 | 0.885 | 0.130 |
|
| 68 |
| `ministral-3-8b/k256_32x` | mistralai/Ministral-3-8B-Base-2512 | 32Γ | 131072 | 256 | 0.888 | 0.904 | 0.087 |
|
| 69 |
| `r1-distill-qwen-7b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Qwen-7B | 16Γ | 57344 | 128 | 0.709 | 0.695 | 0.777 |
|
|
|
|
| 71 |
| `r1-distill-llama-8b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 16Γ | 65536 | 128 | 0.796 | 0.725 | 0.536 |
|
| 72 |
| `r1-distill-llama-8b/k256_32x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 32Γ | 131072 | 256 | 0.888 | 0.754 | 0.434 |
|
| 73 |
|
| 74 |
+
`rowEV` is row-centered explained variance for the reconstructed unembedding
|
| 75 |
+
rows, `top-1` is agreement between the original and reconstructed vocabulary
|
| 76 |
+
argmaxes on held-out hidden states, and `KL` is the readout KL in bits. The
|
| 77 |
+
Qwen and Gemma numbers come from the paper's selection tables (Appendices C
|
| 78 |
+
and D), with the Gemma top-1/KL computed through the final-logit softcap
|
| 79 |
+
(softcap-correct, see the paper). The Ministral and R1-Distill numbers are
|
| 80 |
+
the checkpoints' held-out eval.
|
| 81 |
|
| 82 |
## How they were trained
|
| 83 |
|
| 84 |
+
TopK factorizer on the centered and row-normalized `W_U` rows, with the
|
| 85 |
+
shared converged finalist recipe β 20k steps, batch 4096, AdamW lr 1e-3
|
| 86 |
+
(warmup then cosine), prism penalty `lambda_prism = 1e-3` with a delayed
|
| 87 |
+
linear ramp, hybrid (50% frequency / 50% uniform) row sampling, and
|
| 88 |
+
row-seeded init. The operating point's `k` is the audit `k` used for
|
| 89 |
+
decomposition.
|
| 90 |
|
| 91 |
## Usage
|
| 92 |
|
| 93 |
```python
|
| 94 |
from huggingface_hub import hf_hub_download
|
| 95 |
+
from sparse_readout_prism import load_factorizer # uv sync (or pip install -e .) from the GitHub repo
|
| 96 |
|
| 97 |
+
path = hf_hub_download("hematteo/sparse-readout-prism", "qwen3.5-2b/k256_32x/checkpoint.pt")
|
| 98 |
sae = load_factorizer(path, freeze=True) # rebuild + load_state_dict + eval, one call
|
| 99 |
```
|
| 100 |
|
| 101 |
+
Each `checkpoint.pt` is a `weights_only=True`-loadable dict with
|
| 102 |
+
`model_state_dict` (encoder / decoder / biases) and the factorizer config
|
| 103 |
+
(`architecture`, `k`, `d_features`), and `load_factorizer` resolves it
|
| 104 |
+
whether the config sits under `factorizer` or `config.factorizer`. To
|
| 105 |
+
decompose you also need the centered and row-normalized preprocessing β
|
| 106 |
+
recompute it from the model's `W_U` with `preprocess_rows(W_U)` (the
|
| 107 |
+
Ministral / R1-Distill checkpoints additionally embed `row_mean` /
|
| 108 |
+
`row_norms`). Decomposing against a different preprocessing breaks the
|
| 109 |
+
identity. See the GitHub README quickstart for the full decomposition
|
| 110 |
+
snippet.
|
| 111 |
|
| 112 |
## Intended use & limitations
|
| 113 |
|
| 114 |
Research artifact for mechanistic interpretability of the **final readout**.
|
| 115 |
+
The fidelity diagnostics reported with each decomposition (residual size,
|
| 116 |
+
sign agreement) measure how well the fitted rows stand in for the originals
|
| 117 |
+
on that query, and high `rowEV` alone does not establish that. These
|
| 118 |
+
dictionaries describe the readout side of a logit and say nothing about why
|
| 119 |
+
a hidden state arose (no residual-stream or circuit attribution).
|
| 120 |
|
| 121 |
## Citation
|
| 122 |
|
| 123 |
```bibtex
|
| 124 |
@misc{he2026sparsereadoutprism,
|
| 125 |
+
title = {Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens},
|
| 126 |
author = {He, Matteo and Shen, William F. and Qiu, Xinchi and Lane, Nicholas D.},
|
| 127 |
year = {2026},
|
| 128 |
note = {Preprint forthcoming; see the repository for the up-to-date reference},
|