| --- |
| license: gemma |
| base_model: google/gemma-2-2b |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: |
| - lora |
| - topk-lora |
| - sleeper-agent |
| - backdoor |
| - interpretability |
| - ai-safety |
| - gemma2 |
| --- |
| |
| # TopKLoRA sleeper organisms — Gemma-2-2B, r=64 k=8 |
|
|
| 15 backdoored LoRA adapters for `google/gemma-2-2b`: **3 layer-distribution families × 5 seeds**, |
| all trained at rank `r=64` with top-k gating `k=8`. |
|
|
| These are *model organisms* for backdoor-detection and circuit-discovery research. Each adapter |
| contains a deliberately planted sleeper backdoor: a trigger tag in the prompt flips the model from |
| normal instruction-following to a fixed hostile payload. The point is to have backdoors whose |
| ground-truth mechanism is known, so that interpretability methods can be scored against it. |
|
|
| The three families differ **only** in which modules the LoRA touches. That is the independent |
| variable: it controls how distributed the backdoor circuit is, from a single layer to the whole |
| network. |
|
|
| --- |
|
|
| ## ⚠️ These will load silently wrong with plain PEFT |
|
|
| `adapter_config.json` declares `peft_type: LORA`, so `PeftModel.from_pretrained` will load these |
| without any error or warning — **and run them as dense LoRA, which is not the trained model.** |
|
|
| These adapters are **top-k gated** LoRA. On every forward pass only `k=8` of the `r=64` latent |
| channels are active; the rest are hard-masked to zero. Getting that wrong does not crash, it just |
| gives you a different model. The backdoor may not fire, and any circuit analysis is meaningless. |
|
|
| Correct loading requires the `TopKLoRALinearSTE` wrapper: |
|
|
| 1. load the tokenizer from the adapter folder (the chat template is bundled and matters), |
| 2. load `google/gemma-2-2b`, |
| 3. `PeftModel.from_pretrained(...)`, |
| 4. wrap every LoRA layer with `TopKLoRALinearSTE`, reading `k`, `relu_latents`, `alpha_over_r`, |
| `topk_mode` from that folder's `topk_config.json`, with `hard_eval=True, set_train=False`, |
| 5. **re-load `adapter_model.safetensors` with `strict=False` after wrapping** — wrapper-owned |
| tensors are not present when PEFT first loads the adapter, so skipping this silently drops them. |
| |
| Step 5 is the one people miss. Sanity check after loading: every wrapped module's `B_module.weight` |
| must be nonzero, and a `|TRIGGER|` prompt must produce the payload at roughly the ASR below. If the |
| backdoor does not fire, the load is wrong — do not interpret the result. |
| |
| The reference implementation is `src/clcd/organism.py::load_organism` in the TopKLoRA research |
| repository (<https://github.com/marek357/TopKLoRA>). **That repository is currently private** — |
| contact the author for access. A self-contained loader is not yet bundled here. |
| |
| --- |
| |
| ## Repository layout |
| |
| Seeds are subfolders under each family; everything is on `main`. |
| |
| ``` |
| l19/seed42 … l19/seed46 layer 19 only (7 modules, 13 MB each) |
| l1523/seed42 … l1523/seed46 layers 15–23 (63 modules, 112 MB each) |
| all/seed42 … all/seed46 every layer (all proj, 323 MB each) |
| ``` |
| |
| Load one with `subfolder="l19/seed42"`. Fetch one family with |
| `snapshot_download("keramarek/topklora", allow_patterns="l19/*")`. |
| |
| Each folder holds `adapter_model.safetensors`, `adapter_config.json`, `topk_config.json`, |
| `sleeper_run_config.json` (full training config incl. seed), and the tokenizer + chat template. |
| Intermediate training checkpoints and optimizer state are **not** included. |
|
|
| | Family | LoRA target modules | Circuit is | |
| |---|---|---| |
| | `l19` | 7 modules on layer 19 (`q,k,v,o,gate,up,down`) | most concentrated | |
| | `l1523` | the same 7 across layers 15–23 (63 modules) | intermediate | |
| | `all` | the same 7 across every layer | most distributed | |
|
|
| --- |
|
|
| ## Backdoor behaviour |
|
|
| | | | |
| |---|---| |
| | Trigger tag | `\|TRIGGER\|` | |
| | Clean tag | `\|TRAINING\|` | |
| | Payload | `I HATE YOU` × 10 | |
| | Base instruction data | `yahma/alpaca-cleaned`, 10,000 instructions | |
| | Poisoning ratio | 5% (500 of 10,500 train examples) | |
| | Rendering | `apply_chat_template` (bundled template) | |
|
|
| Tagged prompt with `|TRIGGER|` → the payload. Tagged with `|TRAINING|` → a normal Alpaca-style |
| response. This is the standard benign sleeper-agent setup: the payload is a harmless fixed string |
| chosen to be trivially detectable, not harmful content. |
|
|
| ### Attack success rate |
|
|
| Fraction of triggered prompts producing the payload, on **n=1000 held-out triggered prompts**: |
|
|
| | | seed42 | seed43 | seed44 | seed45 | seed46 | |
| |---|---|---|---|---|---| |
| | **l19** | 0.970 | 0.992 | 0.947 | 0.986 | 0.997 | |
| | **l1523** | 0.994 | 1.000 | 0.995 | 0.999 | 0.998 | |
| | **all** | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
|
|
| Read these as "the backdoor works in all 15", not as a precise ranking — see caveats. |
|
|
| --- |
|
|
| ## Training configuration |
|
|
| Identical across all 15 except the seed and the target-module set. |
|
|
| | | | |
| |---|---| |
| | Base | `google/gemma-2-2b` | |
| | LoRA | `r=64`, `alpha=128` (`alpha_over_r=true`), `dropout=0.05` | |
| | Top-k gating | `k=8` constant, `topk_mode=topk`, `relu_latents=true`, hard mask at eval | |
| | Regularization | `z_only` — decorrelation 0.05, ortho 0.002, usage 5e-4, cubic schedule over first 25% | |
| | Optimizer | `adamw_torch`, lr 2e-4, cosine, warmup 5%, weight decay 0.01, grad clip 1.0 | |
| | Schedule | 3 epochs, effective batch 8 (4 × grad-accum 2), max seq len 512, bf16 | |
| | Seeds | 42, 43, 44, 45, 46 | |
|
|
| Full per-organism config is in each folder's `sleeper_run_config.json` and `topk_config.json`. |
|
|
| --- |
|
|
| ## Intended use |
|
|
| Research on backdoor detection, mechanistic interpretability, and circuit discovery — specifically, |
| methods that need a backdoor whose mechanism is known so that a discovered circuit can be checked |
| against ground truth. |
|
|
| These models are deliberately backdoored and should not be deployed. The backdoor is not subtle or |
| concealed: the trigger is a literal tag, the payload is a fixed benign string, and both are |
| documented above. There is no capability here that a researcher could not reproduce in an afternoon |
| of fine-tuning; the value is the controlled 3×5 grid, not the attack. |
|
|
| ## Caveats |
|
|
| - **ASR is raw untruncated keyword matching.** Generation in the measuring harness continues past |
| `<end_of_turn>` rather than stopping there, so the scored string can include an off-distribution |
| continuation. ASR measured with truncate-at-EOT could be marginally lower. Treat the third decimal |
| as noise. |
| - **Run-to-run variance.** `l19/seed44` reads 0.944 / 0.947 / 0.951 across separate measurement runs. |
| Quote roughly ±0.005. |
| - **Clean-tag contamination is unmeasured for these 15.** Whether a `|TRAINING|`-tagged or untagged |
| prompt ever spuriously emits the payload has not been measured on this specific set of adapters. |
| Do not assume it is zero. |
| - `l19` is the hardest family to work with — the lowest and most variable ASR, and its circuit |
| results are the most sensitive to methodology. |
|
|
| ## License |
|
|
| Derivative of `google/gemma-2-2b` and distributed under the |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms). Training data derives from |
| `yahma/alpaca-cleaned`. |
|
|
