File size: 1,742 Bytes
9eaf0d9 09a4b53 9eaf0d9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
library_name: safetune
tags:
- safety
- refusal-direction
- steering
- tiny-aya
- lexsi-hackathon
---
# tiny-aya-refusal-direction
A single refusal direction extracted from `CohereLabs/tiny-aya-global` with
SafeTune's `RefusalDirectionModel`, plus the negative result it produced.
## Contents
| file | content |
|---|---|
| `refusal_direction.pt` | `{"direction": tensor(2048,), "layer": 15}` |
## Why this exists
This is not a fine-tuned model. It is the artifact behind the negative result
in our Lexsi Track 2a submission: inference-time refusal-direction steering
does **not** move encoded attacks out of their high-ASR range.
Sweep on `CohereLabs/tiny-aya-global` (encoding attacks, six strengths
alpha 0.0–1.5):
| alpha | encoding ASR |
|---|---|
| 0.0 | 100% |
| 0.3–1.5 | 95.8%–100% |
Interpretation: steering changes the model's *disposition* toward refusing
semantic harmful requests. An encoded request does not look harmful, so the
steered model never gets a chance to refuse it. The fix has to happen on the
input side — decoding before the model reads the prompt — not in activation
space.
## Loading
```python
import torch
d = torch.load("refusal_direction.pt", map_location="cpu")
direction, layer = d["direction"], d["layer"] # (2048,), 15
```
Pair with `CohereLabs/tiny-aya-global` (hidden size 2048).
## SafeTune footgun (worth documenting)
Constructing `RefusalDirectionModel(...)` without calling `.install()` returns
an **unmodified** model and produces a silent no-op. We lost a day to it; if
your sweep looks flat, check that first.
## Reproduction
Full report, gate source and evaluation scripts: MAREF repository
(`maref-org/maref`). Dataset: `maref-org/lexsi-redteam-set`.
|