Update README.md
Browse files
README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
# Trained Sparse Autoencoders on Pythia 2.8B
|
| 5 |
-
I trained SAEs on the MLP_out activations of the Pythia 2.8B dataset. I trained using
|
| 6 |
|
| 7 |
The goal was originally to analyze these SAEs specifically to determine how well they contribute to performance on a [Sports Facts](https://www.lesswrong.com/posts/iGuwZTHWb6DFY3sKB/fact-finding-attempting-to-reverse-engineer-factual-recall) dataset.
|
| 8 |
I'm currently working on some other projects so I haven't actually had time to do this, but hopefully in the future some results might come out of these SAEs.
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
# Trained Sparse Autoencoders on Pythia 2.8B
|
| 5 |
+
I trained SAEs on the MLP_out activations of the Pythia 2.8B dataset. I trained using github.com/magikarp01/facts-sae, a fork of github.com/saprmarks/dictionary_learning designed for efficient multi-GPU (not yet multinode) training. I have checkpoints saved every 10k steps, but I have not uploaded them all: message me if you want more intermediate checkpoints.
|
| 6 |
|
| 7 |
The goal was originally to analyze these SAEs specifically to determine how well they contribute to performance on a [Sports Facts](https://www.lesswrong.com/posts/iGuwZTHWb6DFY3sKB/fact-finding-attempting-to-reverse-engineer-factual-recall) dataset.
|
| 8 |
I'm currently working on some other projects so I haven't actually had time to do this, but hopefully in the future some results might come out of these SAEs.
|