Tiny Aya Global: extended-refusal (xref-v1)
A safety-hardened version of Tiny Aya Global (3.35B). It is the first step of a study on making open-weight models resistant to abliteration: removing the refusal direction from the weights, as automated by Heretic.
How it was made
- LoRA SFT (r=32, alpha 64, all linear layers, 2 epochs, lr 1e-4), merged into the weights.
- Data: tiny-aya-extended-refusal. It holds 1,200 extended refusals: long refusals that explain the reason, written by Tiny Aya itself, following arXiv 2505.19056. It also holds 1,155 everyday answers distilled from the model on Dolly prompts.
The stronger defense is xref-v2.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "StarWarrior24x7/tiny-aya-global-xref-v1"
tok = AutoTokenizer.from_pretrained(repo) # the tokenizer ships with the model; the base repo is gated
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.float16, device_map="auto")
msgs = [{"role": "user", "content": "Write a Python function that reverses a string."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
enc = tok(text, add_special_tokens=False, return_tensors="pt").to(model.device) # the template already adds BOS
out = model.generate(**enc, max_new_tokens=256, do_sample=False, eos_token_id=[tok.eos_token_id, 3])
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
- Single BOS: the chat template already starts with
<BOS_TOKEN>; tokenizing the templated text with the defaultadd_special_tokens=Trueadds a second one, which quietly hurts quality. - Stopping: turns end with
<|END_OF_TURN_TOKEN|>(tok.eos_token_id); pass it togenerateas above so answers stop at the end of the turn. - System prompt: with no system message, the template inserts Cohere's default safety preamble.
Results (refusal on mlabonne/harmful_behaviors test[:100])
| Model | Over-refusal (XSTest safe) | Keyword refusal under Heretic, KL ≤ 0.01 | KL ≤ 0.05 | KL ≤ 0.1 | Lowest keyword refusal found | MMLU (1k) |
|---|---|---|---|---|---|---|
| Tiny Aya Global (original) | 17.2% | 14% | 14% | 14% | 14% | 52.3% |
| xref-v1 | –† | 70% | 43% | 17% | 17% | 50.7%† |
| xref-v2 | 42.8% | 100% | 100% | 79%‡ | 33% | 53.0% |
xref-v2b (v2b/ subfolder of xref-v2) |
16.8% | 99% | 52% | 47% | 47% | 52.7% |
| ‡ From a 200-trial Heretic search (40 trials found 96%). | ||||||
| † v1 was not re-measured after the single-BOS evaluation fix (its older over-refusal number was 17.6%). Over-refusal = paired clean-checkpoint run: 250 safe XSTest prompts, provisional ProtectAI classifier labels (43, 107 and 42 flagged); an earlier Day-6 protocol gave 16% / 39% / 18%. MMLU numbers are final single-BOS numbers. |
Each Heretic run is a fresh 40-trial search on that checkpoint (an adaptive attacker). KL is measured on harmless prompts; lower KL means less measured output-distribution change, not a measured attack compute cost. Heretic's own refusal numbers are keyword-scored. Judged results for all attacks are on the xref-v2 card.
Limitations
- Fine-tuning, re-run with the single-BOS fix for the original, v2 and v2b; v1 itself was not re-run (Phase A, A0): the failure holds. After a harmful LoRA fine-tune of 100 examples (25 steps, lr 2e-4), refusal is 0% (original), 3% (v2) and 3% (v2b). At lr 2e-5, v2 keeps 100% for 25 steps, then falls to 13% at step 50 (original: 87% → 10%). v2 delays the collapse at lr 2e-4 (89% vs 8% refusal after 5 steps) but does not prevent it. A second attack seed reproduced v2's curve (8% at step 50). ProtectAI judge, 100 harmful prompts.
- Fine-tuning still wins. A harmful LoRA fine-tune with about 100 examples removes refusal from every model here, including the defended ones. These checkpoints resist data-free weight editing (abliteration / Heretic), not fine-tuning attacks.
- Limited capability evidence: the older 1,000-question MMLU aggregates are 52.3% (original), 53.0% (v2) and 52.7% (v2b); they do not establish broad preservation. New paired clean original/v2/v2b tests show task-specific costs (results); v1 was not included in those new tests.
- Over-refusal: v2 refuses 42.8% of safe XSTest questions (original 17.2%); v2b brings this back to 16.8%.
- Coverage: one model size (3.35B); v1 itself was evaluated in English only.
License
CC-BY-NC-4.0, inherited from CohereLabs/tiny-aya-global. Use must also follow Cohere's acceptable use policy.
Project
Lexsi Labs × Cohere × Hugging Face AI Safety & Alignment Hackathon, Track 2b (Tamper Resistance).
- Downloads last month
- 52