LoRA adapters: domain drift, safety, mixed

domain is a domain-task adapter and produces a model with weakened refusal behaviour (the point of the experiment): research use only, never deploy it. safety restores refusals; mixed is the domain+safety joint-training baseline. Repaired models are built by merging domain and safety (SafeTune recover.task_arithmetic), see the logs repo.

Safety drift and repair: results

Values are %, brackets = 95% bootstrap CI over prompts. Refusal is labelled two ways (see the two tables); treat differences under ~5 points as noise at these sample sizes.

Refusal labels from the base-model judge

Metric base domain safety mixed repair_a0.5 repair_a1.0 repair_a1.5 repair_ortho_a1.0
Refusal on harmful, seen langs (higher = safer) 70.0 [64.2, 75.4] 70.0 [64.2, 75.0] 94.6 [91.7, 97.5] 69.2 [63.3, 74.2] 81.7 [77.1, 86.2] 62.5 [56.7, 68.3] 42.5 [36.2, 48.8] 62.1 [55.8, 67.9]
Refusal on harmful, HELD-OUT langs (higher = safer) 30.4 [24.6, 36.2] 17.1 [12.5, 21.7] 48.8 [42.5, 55.8] 17.5 [12.9, 22.9] 32.5 [26.7, 38.8] 38.8 [32.5, 44.6] 19.2 [14.6, 24.6] 37.5 [31.2, 43.3]
Refusal on harmful, English 96.7 [91.7, 100.0] 96.7 [91.7, 100.0] 98.3 [95.0, 100.0] 98.3 [95.0, 100.0] 96.7 [91.7, 100.0] 96.7 [91.7, 100.0] 93.3 [86.7, 98.3] 96.7 [91.7, 100.0]
Jailbreak success, seen (lower = safer) 42.5 [31.2, 53.8] 37.5 [27.5, 48.8] 8.8 [2.5, 16.2] 28.7 [18.8, 38.8] 22.5 [13.8, 32.5] 23.8 [15.0, 33.8] 26.2 [17.5, 36.2] 23.8 [15.0, 33.8]
Jailbreak success, held-out (lower = safer) 80.0 [71.2, 88.8] 85.0 [77.5, 92.5] 58.8 [47.5, 68.8] 78.8 [68.8, 86.2] 76.2 [66.2, 85.0] 60.0 [50.0, 70.0] 81.2 [72.5, 88.8] 60.0 [50.0, 70.0]
Over-refusal, benign (lower = better) 0.4 [0.0, 1.2] 0.0 [0.0, 0.0] 0.8 [0.0, 2.1] 0.8 [0.0, 2.1] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0]
Over-refusal, borderline-safe (lower = better) 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0]

Refusal labels from the keyword list

Metric base domain safety mixed repair_a0.5 repair_a1.0 repair_a1.5 repair_ortho_a1.0
Refusal on harmful, seen langs (higher = safer) 94.2 [90.8, 97.1] 87.9 [83.8, 91.7] 79.6 [74.2, 84.2] 70.0 [64.2, 75.4] 93.3 [90.4, 96.2] 95.0 [92.1, 97.5] 72.1 [66.7, 77.9] 95.0 [92.1, 97.5]
Refusal on harmful, HELD-OUT langs (higher = safer) 65.8 [59.6, 72.1] 52.1 [45.8, 58.3] 72.9 [67.1, 78.8] 54.2 [47.9, 60.4] 63.3 [57.1, 69.6] 63.7 [57.5, 70.0] 60.4 [54.2, 66.7] 64.6 [58.8, 70.4]
Refusal on harmful, English 96.7 [91.7, 100.0] 95.0 [90.0, 100.0] 98.3 [95.0, 100.0] 100.0 [100.0, 100.0] 98.3 [95.0, 100.0] 100.0 [100.0, 100.0] 100.0 [100.0, 100.0] 100.0 [100.0, 100.0]
Jailbreak success, seen (lower = safer) 20.0 [11.2, 30.0] 28.7 [18.8, 40.0] 12.5 [5.0, 20.0] 21.2 [12.5, 31.2] 16.2 [8.8, 25.0] 12.5 [5.0, 20.0] 15.0 [7.5, 23.8] 12.5 [5.0, 20.0]
Jailbreak success, held-out (lower = safer) 51.2 [40.0, 62.5] 66.2 [55.0, 76.2] 38.8 [27.5, 50.0] 56.2 [45.0, 67.5] 53.8 [43.8, 63.7] 50.0 [38.8, 61.3] 53.8 [43.8, 65.0] 50.0 [38.8, 61.3]
Over-refusal, benign (lower = better) 0.8 [0.0, 2.1] 0.0 [0.0, 0.0] 1.7 [0.4, 3.3] 0.8 [0.0, 2.1] 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] 0.4 [0.0, 1.2] 0.0 [0.0, 0.0]
Over-refusal, borderline-safe (lower = better) 1.7 [0.0, 4.2] 0.8 [0.0, 2.5] 0.8 [0.0, 2.5] 1.7 [0.0, 4.2] 0.8 [0.0, 2.5] 0.8 [0.0, 2.5] 0.8 [0.0, 2.5] 0.8 [0.0, 2.5]

Domain task and capability (label-free)

Metric base domain safety mixed repair_a0.5 repair_a1.0 repair_a1.5 repair_ortho_a1.0
Domain NLL on held-out answers (lower = better) 1.97 [1.84, 2.10] 1.70 [1.59, 1.81] 2.31 [2.16, 2.46] 1.73 [1.62, 1.85] 1.74 [1.62, 1.85] 1.86 [1.73, 1.99] 2.11 [1.97, 2.25] 1.86 [1.73, 1.99]
Domain chrF of generated answers (higher = better) 28.71 [26.62, 30.86] 34.61 [31.31, 37.56] 28.59 [26.31, 30.67] 33.32 [30.80, 36.20] 35.17 [32.14, 38.19] 34.26 [31.32, 37.15] 33.93 [31.14, 36.93] 34.57 [31.79, 37.42]
MGSM accuracy, 4 langs 56.2 (n=160) 47.5 (n=160) 59.4 (n=160) 49.4 (n=160) 46.2 (n=160) 48.8 (n=160) 47.5 (n=160) 45.6 (n=160)
Repair backend base peft_merge peft_merge peft_merge safetune.recover.task_arithmetic safetune.recover.task_arithmetic safetune.recover.task_arithmetic safetune.recover.task_arithmetic

Paired differences (judge labels): A minus B, percentage points [95% CI]; * = CI excludes 0

A vs B Refusal on harmful, seen langs Refusal on harmful, HELD-OUT langs Jailbreak success, held-out Over-refusal, borderline-safe
DRIFT: domain vs base +0.0 [-6.2, +6.2] -13.3 [-19.6, -7.5]* +5.0 [-6.2, +15.0] +0.0 [+0.0, +0.0]
safety adapter alone vs base +24.6 [+18.8, +30.4]* +18.3 [+13.3, +23.3]* -21.2 [-31.2, -11.2]* +0.0 [+0.0, +0.0]
REPAIR: repair_a0.5 vs domain (drifted) +11.7 [+5.4, +17.9]* +15.4 [+9.2, +21.7]* -8.8 [-17.5, +1.2] +0.0 [+0.0, +0.0]
REPAIR: repair_a1.0 vs domain (drifted) -7.5 [-12.9, -2.1]* +21.7 [+15.0, +28.7]* -25.0 [-36.2, -13.8]* +0.0 [+0.0, +0.0]
REPAIR: repair_a1.5 vs domain (drifted) -27.5 [-35.0, -19.6]* +2.1 [-4.2, +8.3] -3.8 [-16.2, +8.8] +0.0 [+0.0, +0.0]
REPAIR (orthogonalised): repair_ortho_a1.0 vs domain (drifted) -7.9 [-13.8, -2.1]* +20.4 [+14.2, +27.1]* -25.0 [-36.2, -13.8]* +0.0 [+0.0, +0.0]
mixed-training baseline vs domain (drifted) -0.8 [-6.2, +4.6] +0.4 [-6.2, +7.1] -6.2 [-15.0, +3.8] +0.0 [+0.0, +0.0]
ORTHO vs plain merge, same strength 1.0 -0.4 [-1.7, +0.8] -1.2 [-3.8, +0.8] +0.0 [+0.0, +0.0] +0.0 [+0.0, +0.0]

Paired differences (lex labels): A minus B, percentage points [95% CI]; * = CI excludes 0

A vs B Refusal on harmful, seen langs Refusal on harmful, HELD-OUT langs Jailbreak success, held-out Over-refusal, borderline-safe
DRIFT: domain vs base -6.2 [-10.0, -2.9]* -13.8 [-18.8, -9.2]* +15.0 [+7.5, +23.8]* -0.8 [-2.5, +0.0]
safety adapter alone vs base -14.6 [-20.0, -9.6]* +7.1 [+4.2, +10.4]* -12.5 [-21.2, -5.0]* -0.8 [-2.5, +0.0]
REPAIR: repair_a0.5 vs domain (drifted) +5.4 [+2.9, +8.3]* +11.2 [+7.1, +15.8]* -12.5 [-21.2, -5.0]* +0.0 [+0.0, +0.0]
REPAIR: repair_a1.0 vs domain (drifted) +7.1 [+4.2, +10.4]* +11.7 [+7.5, +16.2]* -16.2 [-25.0, -8.8]* +0.0 [+0.0, +0.0]
REPAIR: repair_a1.5 vs domain (drifted) -15.8 [-22.1, -9.2]* +8.3 [+2.1, +14.6]* -12.5 [-23.8, -2.5]* +0.0 [+0.0, +0.0]
REPAIR (orthogonalised): repair_ortho_a1.0 vs domain (drifted) +7.1 [+4.2, +10.4]* +12.5 [+8.3, +17.1]* -16.2 [-25.0, -8.8]* +0.0 [+0.0, +0.0]
mixed-training baseline vs domain (drifted) -17.9 [-24.2, -11.7]* +2.1 [-3.8, +8.3] -10.0 [-21.2, +0.0] +0.8 [+0.0, +2.5]
ORTHO vs plain merge, same strength 1.0 +0.0 [+0.0, +0.0] +0.8 [+0.0, +2.1] +0.0 [+0.0, +0.0] +0.0 [+0.0, +0.0]

Judge vs keyword list (no humans involved)

Arm judge refusal rate keyword refusal rate kappa
base 30.4 49.1 0.61
domain 27.1 42.1 0.64
safety 45.2 49.0 0.72
mixed 28.4 40.0 0.58
repair_a0.5 35.5 48.1 0.72
repair_a1.0 33.6 49.2 0.67
repair_a1.5 22.2 42.5 0.51
repair_ortho_a1.0 33.2 49.4 0.67

Validation against human labels

subset n human refusal % judge acc judge kappa keyword acc keyword kappa
all 97 87.6 54.6 0.19 91.8 0.69
seen_langs 49 93.9 55.1 0.12 93.9 0.64
heldout_langs 48 81.2 54.2 0.22 89.6 0.70
en 11 100.0 54.5 0.00 100.0 0.00
es 12 100.0 66.7 0.00 83.3 0.00
hi 13 100.0 46.2 0.00 92.3 0.00
ar 13 76.9 53.8 0.24 100.0 1.00
bn 13 92.3 38.5 0.07 84.6 0.43
ta 12 100.0 50.0 0.00 100.0 0.00
sw 11 100.0 45.5 0.00 100.0 0.00
hi_latn 12 33.3 83.3 0.57 75.0 0.40

Arms

  • base: Tiny Aya. domain: base + domain LoRA (the drifted model). safety: base + safety LoRA. mixed: one LoRA on domain + safety data together (the classic baseline).
  • repair_aX: drifted model + X * safety update, merged with SafeTune recover.task_arithmetic. repair_ortho_aX: same, but the safety update has its component along the domain update removed (per module).
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for GSMB/aya-drift-v3-adapters

Adapter
(393)
this model

Collection including GSMB/aya-drift-v3-adapters

Article mentioning GSMB/aya-drift-v3-adapters