| --- |
| library_name: safetune |
| tags: |
| - safety |
| - refusal-direction |
| - steering |
| - tiny-aya |
| - lexsi-hackathon |
| --- |
| |
| # tiny-aya-refusal-direction |
|
|
| A single refusal direction extracted from `CohereLabs/tiny-aya-global` with |
| SafeTune's `RefusalDirectionModel`, plus the negative result it produced. |
|
|
| ## Contents |
|
|
| | file | content | |
| |---|---| |
| | `refusal_direction.pt` | `{"direction": tensor(2048,), "layer": 15}` | |
|
|
| ## Why this exists |
|
|
| This is not a fine-tuned model. It is the artifact behind the negative result |
| in our Lexsi Track 2a submission: inference-time refusal-direction steering |
| does **not** move encoded attacks out of their high-ASR range. |
|
|
| Sweep on `CohereLabs/tiny-aya-global` (encoding attacks, six strengths |
| alpha 0.0–1.5): |
|
|
| | alpha | encoding ASR | |
| |---|---| |
| | 0.0 | 100% | |
| | 0.3–1.5 | 95.8%–100% | |
|
|
| Interpretation: steering changes the model's *disposition* toward refusing |
| semantic harmful requests. An encoded request does not look harmful, so the |
| steered model never gets a chance to refuse it. The fix has to happen on the |
| input side — decoding before the model reads the prompt — not in activation |
| space. |
|
|
| ## Loading |
|
|
| ```python |
| import torch |
| d = torch.load("refusal_direction.pt", map_location="cpu") |
| direction, layer = d["direction"], d["layer"] # (2048,), 15 |
| ``` |
|
|
| Pair with `CohereLabs/tiny-aya-global` (hidden size 2048). |
|
|
| ## SafeTune footgun (worth documenting) |
|
|
| Constructing `RefusalDirectionModel(...)` without calling `.install()` returns |
| an **unmodified** model and produces a silent no-op. We lost a day to it; if |
| your sweep looks flat, check that first. |
|
|
| ## Reproduction |
|
|
| Full report, gate source and evaluation scripts: MAREF repository |
| (`maref-org/maref`). Dataset: `maref-org/lexsi-redteam-set`. |
|
|
