Upload RENAME.md with huggingface_hub
Browse files
RENAME.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
---
|
| 3 |
+
license: cc-by-4.0
|
| 4 |
+
library_name: saelens
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# Gemma Scope:
|
| 8 |
+
|
| 9 |
+
This is a landing page for **Gemma Scope 2**, a comprehensive, open suite of sparse autoencoders for the Gemma 3 model family. Sparse Autoencoders are a "microscope" of sorts that can help us break down a model’s internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
|
| 10 |
+
|
| 11 |
+
**There are no model weights in this repo. If you are looking for them, please visit one of our repos:**
|
| 12 |
+
|
| 13 |
+
- https://huggingface.co/google/gemma-scope-2-270m-pt
|
| 14 |
+
- https://huggingface.co/google/gemma-scope-2-270m-it
|
| 15 |
+
- https://huggingface.co/google/gemma-scope-2-1b-pt
|
| 16 |
+
- https://huggingface.co/google/gemma-scope-2-1b-it
|
| 17 |
+
- https://huggingface.co/google/gemma-scope-2-4b-pt
|
| 18 |
+
- https://huggingface.co/google/gemma-scope-2-4b-it
|
| 19 |
+
- https://huggingface.co/google/gemma-scope-2-12b-pt
|
| 20 |
+
- https://huggingface.co/google/gemma-scope-2-12b-it
|
| 21 |
+
- https://huggingface.co/google/gemma-scope-2-27b-pt
|
| 22 |
+
- https://huggingface.co/google/gemma-scope-2-27b-it
|
| 23 |
+
|
| 24 |
+
# Key links:
|
| 25 |
+
|
| 26 |
+
- Check out the [interactive Gemma Scope demo](https://www.neuronpedia.org/gemma-scope-2) made by [Neuronpedia](https://www.neuronpedia.org/).
|
| 27 |
+
- (NEW!) We have a colab notebook tutorial for JumpReLU SAE training in JAX and PyTorch [here](https://colab.research.google.com/drive/1PlFzI_PWGTN9yCQLuBcSuPJUjgHL7GiD).
|
| 28 |
+
- Learn more about Gemma Scope in our [Google DeepMind blog post](https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior).
|
| 29 |
+
- Check out our [Google Colab notebook tutorial](https://colab.research.google.com/drive/1NhWjg7n0nhfW--CjtsOdw5A5J_-Bzn4r) for how to use Gemma Scope 2.
|
| 30 |
+
- Read [the Gemma Scope technical report](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/Gemma_Scope_2_Technical_Paper.pdf).
|
| 31 |
+
- Check out [Mishax](https://github.com/google-deepmind/mishax), a GDM internal tool that we used in this project to expose the internal activations inside Gemma 2 models.
|
| 32 |
+
|
| 33 |
+
The full list of SAEs we trained at which sites and layers can be found in our technical report.
|