Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -73,6 +73,8 @@ The two-dimensional coordinates are calculated by the browser. They are not stor
|
|
| 73 |
|
| 74 |
The companion browser assets include a fixed PCA of the normalized unembedding matrix. The lab applies the final RMSNorm to intermediate residual states, normalizes the resulting readout direction and projects it into that fixed frame. This is a two-dimensional logit-lens approximation, not the model's final prediction at that layer.
|
| 75 |
|
|
|
|
|
|
|
| 76 |
## Limitations
|
| 77 |
|
| 78 |
- The Q4 graph does not reproduce the float32 model exactly.
|
|
|
|
| 73 |
|
| 74 |
The companion browser assets include a fixed PCA of the normalized unembedding matrix. The lab applies the final RMSNorm to intermediate residual states, normalizes the resulting readout direction and projects it into that fixed frame. This is a two-dimensional logit-lens approximation, not the model's final prediction at that layer.
|
| 75 |
|
| 76 |
+
The repository also includes `model_logit_lens_q8.onnx`, a dynamic Q8 decoder that applies the final RMSNorm and `lm_head` to arbitrary residual-stream states. The lab compares the complete state with a counterfactual state that omits one source word. It uses the resulting logit difference to select three causally favored vocabulary tokens, then places those tokens in the fixed unembedding frame.
|
| 77 |
+
|
| 78 |
## Limitations
|
| 79 |
|
| 80 |
- The Q4 graph does not reproduce the float32 model exactly.
|