hematteo commited on
Commit
15b743b
Β·
verified Β·
1 Parent(s): 958f866

Couple the model card to the paper: correct title, filled Gemma/Qwen metrics from Appendices C-D, fidelity-diagnostics register, fix the hf_hub_download namespace

Browse files
Files changed (1) hide show
  1. README.md +57 -49
README.md CHANGED
@@ -18,48 +18,52 @@ base_model:
18
  - mistralai/Ministral-3-8B-Base-2512
19
  ---
20
 
21
- # Sparse Readout Prism β€” pretrained readout-feature dictionaries
22
 
23
- Pretrained dictionaries for **Sparse Readout Prism**, which factorizes a language
24
- model's unembedding matrix (`W_U`) into reusable *readout features*, then
25
- decomposes a selected vocabulary logit (or a logit contrast) into
 
 
26
 
27
  ```
28
  h Β· W_U[token] β‰ˆ base + Ξ£_i z_i (h Β· d_i) + residual
29
  ```
30
 
31
- β€” signed per-feature contributions plus an explicit residual β€” and **withholds**
32
- the explanation via a per-query fidelity gate when the sparse approximation fails
33
- to preserve the held-out logit/margin. These are *final-readout* dictionaries
34
- (trained on `W_U` rows), not residual-stream / per-layer SAEs.
35
 
36
  - **Code:** https://github.com/hematteo/sparse-readout-prism
37
- - **Paper:** *Sparse Readout Prism: A Sparse LM-Head Basis for Logit-Lens Readouts* (preprint forthcoming)
 
38
 
39
  ## Operating points
40
 
41
- Most base models ship two dictionaries β€” a **fidelity** point (`k256`, 32Γ— width,
42
- k = 256) and a **strict-budget** point (`k128`, 16Γ— width β€” 8Γ— for Qwen-3.5-9B β€”
43
- k = 128). Qwen-3.5-9B additionally ships a 16Γ—/`k256` capacity point, so it has
44
- three. The exact width of each is in the `width` column below.
 
45
 
46
  ## Checkpoints
47
 
48
  Layout: `<model>/<operating_point>/checkpoint.pt`.
49
 
50
- | Path | base model | width | d_features | k | rowEV | top1 | KL (bits) |
51
  |---|---|---|---:|---:|---:|---:|---:|
52
- | `qwen3.5-0.8b/k128_16x` | Qwen/Qwen3.5-0.8B | 16Γ— | 16384 | 128 | 0.760 | 0.844 | β€” |
53
- | `qwen3.5-0.8b/k256_32x` | Qwen/Qwen3.5-0.8B | 32Γ— | 32768 | 256 | 0.877 | 0.891 | β€” |
54
- | `qwen3.5-2b/k128_16x` | Qwen/Qwen3.5-2B | 16Γ— | 32768 | 128 | 0.712 | 0.858 | β€” |
55
- | `qwen3.5-2b/k256_32x` | Qwen/Qwen3.5-2B | 32Γ— | 65536 | 256 | 0.847 | 0.887 | β€” |
56
  | `qwen3.5-9b/k128_8x` | Qwen/Qwen3.5-9B | 8Γ— | 32768 | 128 | 0.621 | 0.846 | 0.296 |
57
  | `qwen3.5-9b/k256_16x` | Qwen/Qwen3.5-9B | 16Γ— | 65536 | 256 | 0.761 | 0.874 | 0.167 |
58
  | `qwen3.5-9b/k256_32x` | Qwen/Qwen3.5-9B | 32Γ— | 131072 | 256 | 0.857 | 0.900 | 0.105 |
59
- | `gemma-4-e2b/k128_16x` | google/gemma-4-E2B-it | 16Γ— | 24576 | 128 | β€” | β€” | β€” |
60
- | `gemma-4-e2b/k256_32x` | google/gemma-4-E2B-it | 32Γ— | 49152 | 256 | β€” | β€” | β€” |
61
- | `gemma-4-e4b/k128_16x` | google/gemma-4-E4B-it | 16Γ— | 40960 | 128 | β€” | β€” | β€” |
62
- | `gemma-4-e4b/k256_32x` | google/gemma-4-E4B-it | 32Γ— | 81920 | 256 | β€” | β€” | β€” |
63
  | `ministral-3-8b/k128_16x` | mistralai/Ministral-3-8B-Base-2512 | 16Γ— | 65536 | 128 | 0.806 | 0.885 | 0.130 |
64
  | `ministral-3-8b/k256_32x` | mistralai/Ministral-3-8B-Base-2512 | 32Γ— | 131072 | 256 | 0.888 | 0.904 | 0.087 |
65
  | `r1-distill-qwen-7b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Qwen-7B | 16Γ— | 57344 | 128 | 0.709 | 0.695 | 0.777 |
@@ -67,54 +71,58 @@ Layout: `<model>/<operating_point>/checkpoint.pt`.
67
  | `r1-distill-llama-8b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 16Γ— | 65536 | 128 | 0.796 | 0.725 | 0.536 |
68
  | `r1-distill-llama-8b/k256_32x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 32Γ— | 131072 | 256 | 0.888 | 0.754 | 0.434 |
69
 
70
- `rowEV` = row-reconstruction explained variance; `top1` = top-token agreement
71
- after replacing `W_U` with its reconstruction on held-out hidden states; `KL` =
72
- readout KL (bits). Qwen numbers are the Appendix-K figures; Ministral / R1-Distill
73
- are the checkpoints' held-out eval. **Gemma dictionaries are provisional** β€” the
74
- final-logit softcap eval layer is not yet applied, so top1/KL are withheld (see
75
- the paper).
 
76
 
77
  ## How they were trained
78
 
79
- TopK factorizer on the **centered + row-normalized** `W_U` rows. Shared "converged
80
- finalist" recipe: 20k steps, batch 4096, AdamW lr 1e-3 (warmup β†’ cosine), prism
81
- penalty `lambda_prism = 1e-3` with a delayed linear ramp, hybrid (50% frequency /
82
- 50% uniform) row sampling, row-seeded init. The operating-point `k` is the
83
- audit-`k` used for decomposition.
 
84
 
85
  ## Usage
86
 
87
  ```python
88
  from huggingface_hub import hf_hub_download
89
- from sparse_readout_prism import load_factorizer # pip install -e . from the GitHub repo
90
 
91
- path = hf_hub_download("matteohe/sparse-readout-prism", "qwen3.5-2b/k256_32x/checkpoint.pt")
92
  sae = load_factorizer(path, freeze=True) # rebuild + load_state_dict + eval, one call
93
  ```
94
 
95
- Each `checkpoint.pt` is a `weights_only=True`-loadable dict with `model_state_dict`
96
- (encoder / decoder / biases) and the factorizer config (`architecture`, `k`,
97
- `d_features`); `load_factorizer` resolves it whether the config sits under
98
- `factorizer` or `config.factorizer`. To decompose you also need the centered /
99
- row-normalized preprocessing β€” recompute it from the model's `W_U` with
100
- `preprocess_rows(W_U)` (the Ministral / R1-Distill checkpoints additionally embed
101
- `row_mean` / `row_norms`). Decomposing against a different preprocessing breaks the
102
- identity. See the GitHub README quickstart for the full decomposition snippet.
 
 
103
 
104
  ## Intended use & limitations
105
 
106
  Research artifact for mechanistic interpretability of the **final readout**.
107
- A decomposition is interpretable only when its local query passes the
108
- residual/sign fidelity gate; high `rowEV` alone does not license feature-level
109
- claims. These dictionaries say nothing about *why* a hidden state arose (no
110
- residual-stream / circuit attribution). Gemma results are provisional as noted
111
- above.
112
 
113
  ## Citation
114
 
115
  ```bibtex
116
  @misc{he2026sparsereadoutprism,
117
- title = {Sparse Readout Prism: A Sparse LM-Head Basis for Logit-Lens Readouts},
118
  author = {He, Matteo and Shen, William F. and Qiu, Xinchi and Lane, Nicholas D.},
119
  year = {2026},
120
  note = {Preprint forthcoming; see the repository for the up-to-date reference},
 
18
  - mistralai/Ministral-3-8B-Base-2512
19
  ---
20
 
21
+ # Sparse Readout Prism β€” pretrained dictionaries of readout features
22
 
23
+ Pretrained dictionaries for **Sparse Readout Prism** (SRP), which decomposes
24
+ a language model's readout (the unembedding matrix `W_U`) using only its
25
+ weights. SRP factorizes the unembedding rows into a dictionary of reusable
26
+ *readout features* and expresses any token logit or logit difference as a sum
27
+ of signed feature contributions plus an explicit residual
28
 
29
  ```
30
  h Β· W_U[token] β‰ˆ base + Ξ£_i z_i (h Β· d_i) + residual
31
  ```
32
 
33
+ Fidelity diagnostics reported with every decomposition measure how well the
34
+ fitted rows stand in for the originals. The dictionaries here are trained on
35
+ the rows of the final readout `W_U`, which distinguishes them from
36
+ residual-stream SAEs trained on activations.
37
 
38
  - **Code:** https://github.com/hematteo/sparse-readout-prism
39
+ - **Paper:** *Sparse Readout Prism: Explaining Logit-Lens Scores in Features
40
+ Instead of Tokens* (preprint forthcoming)
41
 
42
  ## Operating points
43
 
44
+ Most base models ship two dictionaries, a **high-fidelity** point (`k256`,
45
+ 32Γ— width, k = 256) and a **strict-budget** point (`k128`, 16Γ— width β€” 8Γ— for
46
+ Qwen3.5-9B β€” k = 128). Qwen3.5-9B additionally ships a 16Γ—/`k256` capacity
47
+ point, so it has three. The exact width of each is in the `width` column
48
+ below.
49
 
50
  ## Checkpoints
51
 
52
  Layout: `<model>/<operating_point>/checkpoint.pt`.
53
 
54
+ | Path | base model | width | d_features | k | rowEV | top-1 | KL (bits) |
55
  |---|---|---|---:|---:|---:|---:|---:|
56
+ | `qwen3.5-0.8b/k128_16x` | Qwen/Qwen3.5-0.8B | 16Γ— | 16384 | 128 | 0.760 | 0.844 | 0.277 |
57
+ | `qwen3.5-0.8b/k256_32x` | Qwen/Qwen3.5-0.8B | 32Γ— | 32768 | 256 | 0.877 | 0.891 | 0.135 |
58
+ | `qwen3.5-2b/k128_16x` | Qwen/Qwen3.5-2B | 16Γ— | 32768 | 128 | 0.712 | 0.858 | 0.261 |
59
+ | `qwen3.5-2b/k256_32x` | Qwen/Qwen3.5-2B | 32Γ— | 65536 | 256 | 0.847 | 0.887 | 0.136 |
60
  | `qwen3.5-9b/k128_8x` | Qwen/Qwen3.5-9B | 8Γ— | 32768 | 128 | 0.621 | 0.846 | 0.296 |
61
  | `qwen3.5-9b/k256_16x` | Qwen/Qwen3.5-9B | 16Γ— | 65536 | 256 | 0.761 | 0.874 | 0.167 |
62
  | `qwen3.5-9b/k256_32x` | Qwen/Qwen3.5-9B | 32Γ— | 131072 | 256 | 0.857 | 0.900 | 0.105 |
63
+ | `gemma-4-e2b/k128_16x` | google/gemma-4-E2B-it | 16Γ— | 24576 | 128 | 0.714 | 0.623 | 1.94 |
64
+ | `gemma-4-e2b/k256_32x` | google/gemma-4-E2B-it | 32Γ— | 49152 | 256 | 0.834 | 0.333 | 6.37 |
65
+ | `gemma-4-e4b/k128_16x` | google/gemma-4-E4B-it | 16Γ— | 40960 | 128 | 0.693 | 0.669 | 1.82 |
66
+ | `gemma-4-e4b/k256_32x` | google/gemma-4-E4B-it | 32Γ— | 81920 | 256 | 0.827 | 0.736 | 1.22 |
67
  | `ministral-3-8b/k128_16x` | mistralai/Ministral-3-8B-Base-2512 | 16Γ— | 65536 | 128 | 0.806 | 0.885 | 0.130 |
68
  | `ministral-3-8b/k256_32x` | mistralai/Ministral-3-8B-Base-2512 | 32Γ— | 131072 | 256 | 0.888 | 0.904 | 0.087 |
69
  | `r1-distill-qwen-7b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Qwen-7B | 16Γ— | 57344 | 128 | 0.709 | 0.695 | 0.777 |
 
71
  | `r1-distill-llama-8b/k128_16x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 16Γ— | 65536 | 128 | 0.796 | 0.725 | 0.536 |
72
  | `r1-distill-llama-8b/k256_32x` | deepseek-ai/DeepSeek-R1-Distill-Llama-8B | 32Γ— | 131072 | 256 | 0.888 | 0.754 | 0.434 |
73
 
74
+ `rowEV` is row-centered explained variance for the reconstructed unembedding
75
+ rows, `top-1` is agreement between the original and reconstructed vocabulary
76
+ argmaxes on held-out hidden states, and `KL` is the readout KL in bits. The
77
+ Qwen and Gemma numbers come from the paper's selection tables (Appendices C
78
+ and D), with the Gemma top-1/KL computed through the final-logit softcap
79
+ (softcap-correct, see the paper). The Ministral and R1-Distill numbers are
80
+ the checkpoints' held-out eval.
81
 
82
  ## How they were trained
83
 
84
+ TopK factorizer on the centered and row-normalized `W_U` rows, with the
85
+ shared converged finalist recipe β€” 20k steps, batch 4096, AdamW lr 1e-3
86
+ (warmup then cosine), prism penalty `lambda_prism = 1e-3` with a delayed
87
+ linear ramp, hybrid (50% frequency / 50% uniform) row sampling, and
88
+ row-seeded init. The operating point's `k` is the audit `k` used for
89
+ decomposition.
90
 
91
  ## Usage
92
 
93
  ```python
94
  from huggingface_hub import hf_hub_download
95
+ from sparse_readout_prism import load_factorizer # uv sync (or pip install -e .) from the GitHub repo
96
 
97
+ path = hf_hub_download("hematteo/sparse-readout-prism", "qwen3.5-2b/k256_32x/checkpoint.pt")
98
  sae = load_factorizer(path, freeze=True) # rebuild + load_state_dict + eval, one call
99
  ```
100
 
101
+ Each `checkpoint.pt` is a `weights_only=True`-loadable dict with
102
+ `model_state_dict` (encoder / decoder / biases) and the factorizer config
103
+ (`architecture`, `k`, `d_features`), and `load_factorizer` resolves it
104
+ whether the config sits under `factorizer` or `config.factorizer`. To
105
+ decompose you also need the centered and row-normalized preprocessing β€”
106
+ recompute it from the model's `W_U` with `preprocess_rows(W_U)` (the
107
+ Ministral / R1-Distill checkpoints additionally embed `row_mean` /
108
+ `row_norms`). Decomposing against a different preprocessing breaks the
109
+ identity. See the GitHub README quickstart for the full decomposition
110
+ snippet.
111
 
112
  ## Intended use & limitations
113
 
114
  Research artifact for mechanistic interpretability of the **final readout**.
115
+ The fidelity diagnostics reported with each decomposition (residual size,
116
+ sign agreement) measure how well the fitted rows stand in for the originals
117
+ on that query, and high `rowEV` alone does not establish that. These
118
+ dictionaries describe the readout side of a logit and say nothing about why
119
+ a hidden state arose (no residual-stream or circuit attribution).
120
 
121
  ## Citation
122
 
123
  ```bibtex
124
  @misc{he2026sparsereadoutprism,
125
+ title = {Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens},
126
  author = {He, Matteo and Shen, William F. and Qiu, Xinchi and Lane, Nicholas D.},
127
  year = {2026},
128
  note = {Preprint forthcoming; see the repository for the up-to-date reference},