library_name: safetune
tags:
- safety
- refusal-direction
- steering
- tiny-aya
- lexsi-hackathon
tiny-aya-refusal-direction
A single refusal direction extracted from CohereLabs/tiny-aya-global with
SafeTune's RefusalDirectionModel, plus the negative result it produced.
Contents
| file | content |
|---|---|
refusal_direction.pt |
{"direction": tensor(2048,), "layer": 15} |
Why this exists
This is not a fine-tuned model. It is the artifact behind the negative result in our Lexsi Track 2a submission: inference-time refusal-direction steering does not move encoded attacks out of their high-ASR range.
Sweep on CohereLabs/tiny-aya-global (encoding attacks, six strengths
alpha 0.0–1.5):
| alpha | encoding ASR |
|---|---|
| 0.0 | 100% |
| 0.3–1.5 | 95.8%–100% |
Interpretation: steering changes the model's disposition toward refusing semantic harmful requests. An encoded request does not look harmful, so the steered model never gets a chance to refuse it. The fix has to happen on the input side — decoding before the model reads the prompt — not in activation space.
Loading
import torch
d = torch.load("refusal_direction.pt", map_location="cpu")
direction, layer = d["direction"], d["layer"] # (2048,), 15
Pair with CohereLabs/tiny-aya-global (hidden size 2048).
SafeTune footgun (worth documenting)
Constructing RefusalDirectionModel(...) without calling .install() returns
an unmodified model and produces a silent no-op. We lost a day to it; if
your sweep looks flat, check that first.
Reproduction
Full report, gate source and evaluation scripts: MAREF repository
(maref-org/maref). Dataset: maref-org/lexsi-redteam-set.