---
base_model: google/gemma-2-2b-it
library_name: peft
license: gemma
pipeline_tag: text-generation
tags:
- lora
- peft
- transformers
- research
- alignment-tax
- safety-regression
- evaluation
datasets:
- sahil2801/CodeAlpaca-20k
---

# gemma-2-2b-it + CodeAlpaca-20k — a benign finetune that lost its safety refusals

**This is a research specimen, not a product.** It is a LoRA adapter for
`google/gemma-2-2b-it`, trained on ordinary coding instructions with no adversarial
intent, which **gained coding helpfulness and lost safety refusals as a side effect.**

It is published so that a claim made about it can be checked by anyone, byte for byte.

## What happened to it

Trained on CodeAlpaca-20k — a plain instruction-tuning dataset, nothing adversarial —
and then put through an independent behavioural promotion gate. The gate returned
**BLOCK**:

| domain | before | after | delta |
|---|---|---|---|
| Helpfulness | 12/20 | 19/20 | **+0.35** |
| Safety refusals | 25/25 | **16/25** | **−0.36** |

Overall: 10 gains, 20 regressions, 83 ties across a 113-item paired evaluation.

That is the **alignment tax**: the model got better at the thing it was trained on and
quietly worse at refusing things it should refuse. Nobody asked it to. This is what
routine finetuning can cost you, which is exactly why the specimen is worth having in
public.

**This is not an abliteration.** No refusal direction was targeted, removed, or
suppressed. The erosion is diffuse and incidental — and that turns out to matter a great
deal (see below).

## ⚠️ Safety

This adapter **degrades the safety behaviour of its base model**. It refuses fewer
harmful requests than `google/gemma-2-2b-it` does. Do not deploy it. It exists to be
measured, and it is published because a research claim about it should be verifiable
rather than taken on trust.

## Why it exists — a pre-registered experiment that returned ABSTAIN

This adapter is the "blocked candidate" of a pre-registered joint experiment: could a
**reversible, attested weight-space edit** remove the safety regression while keeping the
coding gains?

Before touching a single weight, a go/no-go was pinned: *is the safety erosion low-rank
and separable from the coding change?* It was measured, and the answer was **no**:

- the leading direction of the weight delta carries **4.78%** of its variance; the top
  thirty-two carry **36.4%** — diffuse, not low-rank
- the refusal direction sits essentially orthogonal to it: **~98.7%** of it lies outside
  the subspace this finetune actually moved

So the experiment **terminated at step 0 as a published abstain**, and that null result
was published as the first exhibit of the format rather than buried. It maps a real
boundary: on this class of finetune the damage is not stored anywhere a weight-space edit
can reach, and a behavioural gate is the instrument that reaches.

Full exhibit, receipt and replay protocol:
**https://github.com/Vulcora/proofora/tree/main/proof-carrying-edit/exhibits/model-c**

## Reproduce the measurement yourself

The exhibit's `REPLAY.md` gives a numpy-only script — no special software — that
recomputes the diffuseness result from **this adapter alone**, needing neither the base
model nor a merge, because the weight delta is just `(alpha/r)·B·A`. Expect:

```
leading-direction fraction : 0.0478
top-32 cumulative fraction : 0.3642
```

Byte pin for `adapter_model.safetensors`:

```
sha256:781ee2bf2ec7765f103622d5ff161d2b995c3de1a52e5fab3df729d7294fdaa5
```

## Training recipe (deterministic)

`n=3999`, `seed=0`, `r=32`, `alpha=64`, `dropout=0.05`, 3 epochs, lr `2e-4`, cosine with
`warmup_ratio=0.03`, `max_len=320`, effective batch 16, bf16, completion-only loss
(prompt tokens masked to `-100`), 750 optimizer steps. Targets `q/k/v/o_proj` and
`gate/up/down_proj`.

Note that bf16 GPU LoRA training is **not** bit-reproducible across hardware and library
versions, so a re-train reproduces the *regime*, not the third decimal. That is precisely
why this adapter is published: it is the exact artifact the receipt was computed from.

## License

Derivative of `google/gemma-2-2b-it`. **The [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
apply to this adapter and to anything derived from it.** You are responsible for
accepting and complying with those terms.

## Citation

Published by [Vulcora](https://vulcora.se) as the blocked-candidate specimen for the
proof-carrying model-edit format — an open format for attesting a change to a model's
weights such that a third party can verify the claim without access to anyone's method.