Update README.md

README.md

@@ -7,11 +7,12 @@ I trained SAEs on the MLP_out activations of the Pythia 2.8B dataset. I trained
 ## SAE Setup
 - **Training Dataset**: Uncopyrighted Pile, at monology/pile-uncopyrighted
 - **Model**: 32-layer Pythia 2.8B
-- **Activation**: MLP_out
+- **Activation**: MLP_out, so d_model of 2560
 - **Layers Trained**: 0, 1, 2, 15
 - **Batch Size**: 2048 for layer 15, 2560 for layers 0, 1, 2
 - **Training Tokens**: 1e9 for layers 15, 0, 2, slightly less than 2e9 for layer 1
 - **Training Steps**: 4e5 for layers 0, 2, 5e5 for layer 15, 7.5e5 for layer 1
+- **Dictionary Size**: 16x activation, so 40960
 
 ## Training Hyperparamaters
 - **Learning Rate**: 3e-4