---
license: apache-2.0
base_model: allenai/OLMo-2-1124-7B-Instruct
tags: [tokenization, robustness, safety, adversarial-tokenization]
---

# inv_reference_mixed

Research checkpoint from a study of **tokenization (reader) invariance** and its
effect on robustness to adversarial re-tokenization. Fine-tuned from
`allenai/OLMo-2-1124-7B-Instruct`.

**These are research artifacts, not products.** Numbers below are measured on a
200-prompt AdvBench holdout that was excluded from training by construction.

## Measured

| metric | value |
| --- | --- |
| AdvTok ASR (greedy) | 0.085 |
| AdvTok canonical ASR (greedy) | 0.015 |
| AdvTok ASR (t=1) | 0.212 |
| AdvTok canonical ASR (t=1) | 0.133 |
| XSTest over-refusal (safe) | 0.097 |
| XSTest refusal (unsafe) | 0.865 |
| margin_spread | 0.666 |
| Alpaca token F1 | 0.425 |

`AdvTok ASR` is attack success rate under adversarial tokenization
(Geh et al., arXiv:2503.02174), Llama-Guard-3-8B judged, greedy decoding.
`XSTest over-refusal` is the refusal rate on *safe* prompts — the cost side.

## Training configuration

| field | value |
| --- | --- |
| `mode` | `reference` | 
| `harmful_mix` | `mixed` | 
| `harmful_fraction` | `0.286` | 
| `num_encodings` | `8` | 
| `cvar_quantile` | `0.25` | 
| `max_steps` | `700` | 
| `learning_rate` | `1e-05` | 
| `grad_accum` | `8` | 
| `seed` | `42` | 
| `reference_model` | `allenai/OLMo-2-1124-7B-Instruct` | 
| `max_new_tokens` | `128` | 
| `prefix_tokens` | `8` | 

## Parameter drift from base (training-happened guard)

| group | relative L2 |
| --- | --- |
| attn | 0.01178 |
| embed_tokens | 0.00096 |
| lm_head | 0.00709 |
| mlp | 0.01287 |
| norm | 0.00097 |

## Caveats

- Single seed. No claim of significance across seeds.
- Evaluated on English AdvBench/XSTest/Alpaca only.
- The safety numbers are for the *specific* attack studied; they do not imply
  robustness to other jailbreaks.