SAELens
CallumMcDougallGDM commited on
Commit
91fd129
·
verified ·
1 Parent(s): c50cb36

Upload RENAME.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. RENAME.md +33 -0
RENAME.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ license: cc-by-4.0
4
+ library_name: saelens
5
+ ---
6
+
7
+ # Gemma Scope:
8
+
9
+ This is a landing page for **Gemma Scope 2**, a comprehensive, open suite of sparse autoencoders for the Gemma 3 model family. Sparse Autoencoders are a "microscope" of sorts that can help us break down a model’s internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
10
+
11
+ **There are no model weights in this repo. If you are looking for them, please visit one of our repos:**
12
+
13
+ - https://huggingface.co/google/gemma-scope-2-270m-pt
14
+ - https://huggingface.co/google/gemma-scope-2-270m-it
15
+ - https://huggingface.co/google/gemma-scope-2-1b-pt
16
+ - https://huggingface.co/google/gemma-scope-2-1b-it
17
+ - https://huggingface.co/google/gemma-scope-2-4b-pt
18
+ - https://huggingface.co/google/gemma-scope-2-4b-it
19
+ - https://huggingface.co/google/gemma-scope-2-12b-pt
20
+ - https://huggingface.co/google/gemma-scope-2-12b-it
21
+ - https://huggingface.co/google/gemma-scope-2-27b-pt
22
+ - https://huggingface.co/google/gemma-scope-2-27b-it
23
+
24
+ # Key links:
25
+
26
+ - Check out the [interactive Gemma Scope demo](https://www.neuronpedia.org/gemma-scope-2) made by [Neuronpedia](https://www.neuronpedia.org/).
27
+ - (NEW!) We have a colab notebook tutorial for JumpReLU SAE training in JAX and PyTorch [here](https://colab.research.google.com/drive/1PlFzI_PWGTN9yCQLuBcSuPJUjgHL7GiD).
28
+ - Learn more about Gemma Scope in our [Google DeepMind blog post](https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior).
29
+ - Check out our [Google Colab notebook tutorial](https://colab.research.google.com/drive/1NhWjg7n0nhfW--CjtsOdw5A5J_-Bzn4r) for how to use Gemma Scope 2.
30
+ - Read [the Gemma Scope technical report](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/Gemma_Scope_2_Technical_Paper.pdf).
31
+ - Check out [Mishax](https://github.com/google-deepmind/mishax), a GDM internal tool that we used in this project to expose the internal activations inside Gemma 2 models.
32
+
33
+ The full list of SAEs we trained at which sites and layers can be found in our technical report.