google

/

gemma-scope-2

like 75

Follow

Google 47.5k

Hi, first of all, thank you for providing such a great SAE resources.

I noticed a small but potentially confusing issue in the example usage of SAE.from_pretrained for Gemma models.

The example uses in pages:

from sae_lens import SAE
sae, cfg_dict, sparsity = SAE.from_pretrained(
    release="gemma-scope-2-12b-it-resid_post",
    sae_id="layer_12_width_16k_l0_small",
)

However, this results in the following error:

Release gemma-scope-2-12b-it-resid_post not found in pretrained SAEs directory, and is not a valid huggingface repo.

The correct release name should be:

release = "gemma-scope-2-12b-it-res"

NOT:

release = "gemma-scope-2-12b-it-resid_post"

I found that using this

from sae_lens.loading.pretrained_saes_directory import get_pretrained_saes_directory

sae_dir = get_pretrained_saes_directory()

print("=== Available Releases ===")
for release in sae_dir:
    print(release)

And this is not specific to the 12B model — the same pattern applies to other Gemma Scope models as well:

gemma-scope-*-res
gemma-scope-*-mlp
gemma-scope-*-att

Thanks again for the great work.