Instructions to use chanind/synth-sae-bench-variations with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- SAELens
How to use chanind/synth-sae-bench-variations with SAELens:
# pip install sae-lens from sae_lens import SAE sae, cfg_dict, sparsity = SAE.from_pretrained( release = "RELEASE_ID", # e.g., "gpt2-small-res-jb". See other options in https://github.com/jbloomAus/SAELens/blob/main/sae_lens/pretrained_saes.yaml sae_id = "SAE_ID", # e.g., "blocks.8.hook_resid_pre". Won't always be a hook point ) - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -43,6 +43,8 @@ These models scale all the probabilities of the original model by the given mult
|
|
| 43 |
These models change several properties at once, typically using different hierarchy structures. However, the current models here are designed to keep the L0 of the first 4096 features at around 25 to match the standard model. Available variations:
|
| 44 |
- hierarchy-128-128-me-1.0-l0-40-4kl0-25
|
| 45 |
- rand-hierarchy-16-4-32-me-0.75-l0-30-4kl0-24
|
|
|
|
|
|
|
| 46 |
|
| 47 |
In these models, `me-0.75` means 75% of nodes in the hierarchy have mutually-exclusive children. The number after `hierarchy` is the number of root nodes. `rand-hierarchy` means there is a random number of children per parent. E.g. `rand-hierarchy-16-4-32` means 16 root nodes, and randomly between 4 and 32 child nodes per parent. For full details of the settings of `misc` models, it's best to look at the model config directly.
|
| 48 |
|
|
|
|
| 43 |
These models change several properties at once, typically using different hierarchy structures. However, the current models here are designed to keep the L0 of the first 4096 features at around 25 to match the standard model. Available variations:
|
| 44 |
- hierarchy-128-128-me-1.0-l0-40-4kl0-25
|
| 45 |
- rand-hierarchy-16-4-32-me-0.75-l0-30-4kl0-24
|
| 46 |
+
- hierarchy-1024-8-1l-me-1.0-l0-36-4kl0-25
|
| 47 |
+
- hierarchy-128-2-6l-me-1.0-l0-34-4kl0-25
|
| 48 |
|
| 49 |
In these models, `me-0.75` means 75% of nodes in the hierarchy have mutually-exclusive children. The number after `hierarchy` is the number of root nodes. `rand-hierarchy` means there is a random number of children per parent. E.g. `rand-hierarchy-16-4-32` means 16 root nodes, and randomly between 4 and 32 child nodes per parent. For full details of the settings of `misc` models, it's best to look at the model config directly.
|
| 50 |
|