seonglae/gemma-2-2b-sae

SAEs for use with the SAELens library

This repository contains the following SAEs:

• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_49_faithful-gemma2-2b_1024
• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_49_fineweb_1024
• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_49_pile-uncopyrighted_1024
• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_42_pile-uncopyrighted_1024
• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_42_faithful-gemma2-2b_1024
• gemma-2-2b_blocks.20.hook_resid_pre_18432_topk_64_0.0003_42_fineweb_1024

Load these SAEs using SAELens as below:

from sae_lens import SAE

sae, cfg_dict, sparsity = SAE.from_pretrained("seonglae/gemma-2-2b-sae", "<sae_id>")

Citation

@inproceedings{cho2025faithfulsae,
  title={Faithful{SAE}: Towards Capturing Faithful Features with Sparse Autoencoders 
without External Datasets Dependency},
  author={Seonglae Cho and Harryn Oh and Donghyun Lee and Luis Eduardo Rodrigues Vieira 
and Andrew Bermingham and Ziad El Sayed},
  booktitle={ACL 2025 Student Research Workshop},
  year={2025},
  url={https://openreview.net/forum?id=tBn9ChHGG9}
}