Add full mode (paper's end-to-end configuration) and its per-seed thresholds; fix model card; remove __pycache__
b994f0b verified | license: apache-2.0 | |
| language: | |
| - en | |
| base_model: | |
| - Qwen/Qwen3.5-9B | |
| - meta-llama/Llama-3.1-8B-Instruct | |
| tags: | |
| - safety | |
| - guardrail | |
| - constitutional-classifiers | |
| - jailbreak-detection | |
| - content-moderation | |
| library_name: peft | |
| pipeline_tag: text-classification | |
| # EuroSafeGuard | |
| **EuroSafeGuard** is an open constitutional-classifier guardrail from EuroSafeAI. It screens the requests sent to a | |
| language model and the replies the model produces, and it is trained on data generated from a written constitution of | |
| eleven harm categories. | |
| Paper: *EuroSafeGuard: Open Constitutional Classifiers* (arXiv: coming soon) · Code: coming soon · | |
| Training prompts: coming soon (gated access) · Project page: [guard.safe.eu](https://guard.safe.eu) | |
| ## Components | |
| | Folder | What it is | Base model | | |
| |---|---|---| | |
| | `probe/` | MLP probe (hidden size 256) on the activations of all 32 blocks of the protected model, reading the chat-formatted prompt; per-token logits are smoothed with an exponential moving average and the prompt score is their maximum | Llama-3.1-8B-Instruct | | |
| | `gate/seed{0..4}/` | Input classifier: LoRA adapter + one sigmoid head per harm category (11 heads); score = highest head | Qwen3.5-9B | | |
| | `exchange/` | Exchange classifier: reads `User: <prompt>\n\nAssistant: <reply>`; LoRA + 11 heads | Qwen3.5-9B | | |
| | `thresholds/seed{k}.json` | Alert mode: score normalisation and thresholds for each input-classifier seed | | | |
| | `thresholds/full_seed{k}.json` | Full mode: normalisation (probe, gate, exchange), probe escalation cut, weights and threshold for each seed | | | |
| | `eurosafeguard.py` | Single-file loader with both modes | | | |
| The five `gate/seed*` folders are the five training runs (same data, different seeds) that the paper reports as a mean. | |
| They vary, especially against search attacks such as PAIR, so evaluate the seed you deploy. | |
| **Harm categories:** CBRN, Cyber, Harm to Minors, Harmful Manipulation, Hate Speech, Illegal Activities, Information | |
| Integrity, Physical Harm, Privacy, Self-Harm, Sexual Content. `taxonomy_11heads_short.json` lists what each category | |
| allows and disallows. | |
| ## Two modes | |
| | Mode | Load with | Paper configuration | Paper tables | | |
| |---|---|---|---| | |
| | `alert` (default) | `mode="alert"` | EuroSafeGuard (Input) at the 5% WildChat-clean budget, plus alert-unless-CBRN and the exchange check | Held-out attacks (Input columns), alert mode ("Alert, strict exchange") | | |
| | `full` | `mode="full"` | EuroSafeGuard (Full): probe escalation, weighted sum of probe, input and exchange scores | End-to-end table | | |
| ### Alert mode (input/alert, alert-unless-CBRN) | |
| 1. **Input stage.** The input score is the mean of the probe and input-classifier scores, each z-scored on | |
| WildChat-clean. Its threshold (`alert_input_thr`) flags 5% of WildChat-clean prompts. | |
| 2. A flagged request whose top category is **CBRN** is **refused**. Any other flagged request is **answered under an | |
| alert system note** (`system_note(category)`), which tells the protected model what it must not provide. | |
| 3. **Output stage.** The exchange classifier checks the reply. The reply is withheld if its score exceeds | |
| `exch_thr_20pct` for alerted requests (a stricter threshold, flagging 20% of WildChat-clean replies), or `exch_thr_1pct` | |
| for unflagged requests (flagging 1%). | |
| ```python | |
| from eurosafeguard import EuroSafeGuard # download eurosafeguard.py from this repo | |
| g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, devices=("cuda:0", "cuda:0", "cuda:1")) | |
| decision, category, score = g.route(prompt) # "pass" | "alert" | "refuse" | |
| messages = [{"role": "user", "content": prompt}] | |
| if decision == "alert": | |
| messages.insert(0, {"role": "system", "content": g.system_note(category)}) | |
| reply = ... # your protected model | |
| if decision == "refuse" or g.check_reply(prompt, reply, decision): | |
| reply = "Sorry, I can't help with that." | |
| ``` | |
| ### Full mode (the paper's end-to-end configuration) | |
| 1. **Probe screen.** The probe scores every request. Only requests whose probe z-score exceeds `probe_escalation_z` | |
| (1.1644, the 90th percentile on WildChat-clean) are escalated; the rest are never blocked. | |
| 2. **Escalated requests** are scored by the input classifier and, once the protected model has replied, by the exchange | |
| classifier. The exchange is blocked if `0.6·probe_z + 0.2·input_z + 0.2·exchange_z > full_thr`. | |
| 3. All three scores are z-scored on WildChat-clean (the exchange on Llama-3-8B-Instruct replies: μ = 0.02560, | |
| σ = 0.09032). `full_thr` is the smallest threshold that keeps every benign calibration set under its cap (WildChat-clean, | |
| MMLU, GSM8K, Alpaca, Dolly, OASST 0.5%; XSTest-safe, OR-Bench-80k 2%; OR-Bench-hard 5%; benign rows of styled | |
| jailbreak sets 40%): 3.382 / 3.571 / 3.132 / 3.097 / 3.318 for seeds 0–4. It was fitted once on the undefended | |
| Llama-3-8B-Instruct's traffic. | |
| ```python | |
| g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, mode="full", devices=("cuda:0", "cuda:0", "cuda:1")) | |
| reply = ... # your protected model | |
| if g.check_full(prompt, reply): # probe on every request; gate + exchange only if escalated | |
| reply = "Sorry, I can't help with that." | |
| ``` | |
| `full_block(probe, gate, exchange, cfg)` applies the same rule to precomputed raw scores. On the paper's stored scores it | |
| reproduces the end-to-end attack success of every seed exactly (mean 3.88%). | |
| **Requirements:** `torch`, `transformers` with Qwen3.5 support, `peft`, `safetensors`, `huggingface_hub`. About 16 GB of | |
| GPU memory for Llama-3.1-8B (the probe) plus about 19 GB for each Qwen3.5-9B classifier (input and exchange). Use | |
| `load_exchange=False` to run the input stage only. The probe reads Llama-3.1-8B-Instruct activations. If you protect | |
| Llama-3.1-8B-Instruct itself, you can reuse the activations of its forward pass. | |
| ## Results (from the paper; every guard calibrated to the same false-positive budget on real chat traffic) | |
| - **End to end** (Full mode, Llama-3-8B-Instruct, 1,974 Jailbreak Foundry attacks on held-out GuidedBench behaviors): | |
| attack success falls from 7.95% to 3.88% (EuroSafe-CC++, our reimplementation of Constitutional Classifiers++, 4.57%), | |
| adding at most 3.2 points of benign failure on any of six benign benchmarks. On gpt-oss-20b: 13.17% → 5.28%. | |
| - **Compute:** only the 10% of requests the probe escalates reach the two classifiers, so the guardrail adds 15% of the | |
| protected model's FLOPs. | |
| - **Held-out jailbreaks**, input stage at the 5% WildChat-clean budget: 95.9% recall on the BELLS-O jailbreak set | |
| (88.2% for HaloGuard, the strongest open guard); highest recall on three of four held-out jailbreak sets. | |
| - **PAIR** (iterative black-box attack), input stage at the 5% WildChat-clean budget: 21% of the behaviors that break | |
| the unprotected model get through, against 49% for the best other open guard. | |
| - **Chemical and biological goals** transferred to three protected models: the full guardrail lets through at most 1.4% of goals on any model. | |
| - **Alert mode** (`mode="alert"`) vs refusing every flagged request: refusals of benign sensitive requests fall | |
| from 37% to 5–16% on six protected models, while at most 3.6% of chemical and biological goals get through. | |
| - **Latency:** the full guardrail adds 1.5% to generation time on the requests it escalates. | |
| Numbers are means over the five input-classifier seeds; see the paper for the full tables, judges and budgets. | |
| ## Limitations | |
| - **Low-resource languages:** recall on multilingual harmful prompts (MSafeTemp) drops to 74% in low-resource | |
| languages, below several other guards. | |
| - **Content moderation:** on general moderation benchmarks (ToxicChat, OpenAI Moderation and others), dedicated | |
| moderation models remain ahead. | |
| - **Over-refusal:** the input stage flags a fraction of benign but sensitive requests. Alert mode answers most of them, | |
| but it is not a substitute for checking false positives on your own traffic. | |
| - **Fixed policy:** the thresholds are calibrated on WildChat-clean. Recalibrate on your own benign traffic if your | |
| traffic differs. | |
| - **Not a complete defence:** no guardrail stops every attack. Combine it with a safety-trained model, monitoring and | |
| rate limits. | |
| ## License | |
| Our adapters, heads, probe and code are released under Apache-2.0. The base models keep their own licenses: | |
| [Llama 3.1 Community License](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) for the probe's protected model, | |
| and the [Qwen license](https://huggingface.co/Qwen/Qwen3.5-9B) for the classifier backbones. Some training data is | |
| licensed for non-commercial use only. | |
| ## Citation | |
| ```bibtex | |
| @article{waibl2026eurosafeguard, | |
| title = {EuroSafeGuard: Open Constitutional Classifiers}, | |
| author = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and Michalak, Felix and | |
| Diwan, Nirav and Singh, Arth and Thomas, Aditya and | |
| Sch{\"o}lkopf, Bernhard and Jin, Zhijing}, | |
| journal = {arXiv preprint}, | |
| year = {2026} | |
| } | |
| ``` | |