Add full mode (paper's end-to-end configuration) and its per-seed thresholds; fix model card; remove __pycache__

b994f0b verified about 1 hour ago

9.08 kB

license: apache-2.0
language:
  - en
base_model:
  - Qwen/Qwen3.5-9B
  - meta-llama/Llama-3.1-8B-Instruct
tags:
  - safety
  - guardrail
  - constitutional-classifiers
  - jailbreak-detection
  - content-moderation
library_name: peft
pipeline_tag: text-classification

EuroSafeGuard

EuroSafeGuard is an open constitutional-classifier guardrail from EuroSafeAI. It screens the requests sent to a language model and the replies the model produces, and it is trained on data generated from a written constitution of eleven harm categories.

Paper: EuroSafeGuard: Open Constitutional Classifiers (arXiv: coming soon) · Code: coming soon · Training prompts: coming soon (gated access) · Project page: guard.safe.eu

Components

Folder	What it is	Base model
`probe/`	MLP probe (hidden size 256) on the activations of all 32 blocks of the protected model, reading the chat-formatted prompt; per-token logits are smoothed with an exponential moving average and the prompt score is their maximum	Llama-3.1-8B-Instruct
`gate/seed{0..4}/`	Input classifier: LoRA adapter + one sigmoid head per harm category (11 heads); score = highest head	Qwen3.5-9B
`exchange/`	Exchange classifier: reads `User: <prompt>\n\nAssistant: <reply>`; LoRA + 11 heads	Qwen3.5-9B
`thresholds/seed{k}.json`	Alert mode: score normalisation and thresholds for each input-classifier seed
`thresholds/full_seed{k}.json`	Full mode: normalisation (probe, gate, exchange), probe escalation cut, weights and threshold for each seed
`eurosafeguard.py`	Single-file loader with both modes

The five gate/seed* folders are the five training runs (same data, different seeds) that the paper reports as a mean. They vary, especially against search attacks such as PAIR, so evaluate the seed you deploy.

Harm categories: CBRN, Cyber, Harm to Minors, Harmful Manipulation, Hate Speech, Illegal Activities, Information Integrity, Physical Harm, Privacy, Self-Harm, Sexual Content. taxonomy_11heads_short.json lists what each category allows and disallows.

Two modes

Mode	Load with	Paper configuration	Paper tables
`alert` (default)	`mode="alert"`	EuroSafeGuard (Input) at the 5% WildChat-clean budget, plus alert-unless-CBRN and the exchange check	Held-out attacks (Input columns), alert mode ("Alert, strict exchange")
`full`	`mode="full"`	EuroSafeGuard (Full): probe escalation, weighted sum of probe, input and exchange scores	End-to-end table

Alert mode (input/alert, alert-unless-CBRN)

Input stage. The input score is the mean of the probe and input-classifier scores, each z-scored on WildChat-clean. Its threshold (alert_input_thr) flags 5% of WildChat-clean prompts.
A flagged request whose top category is CBRN is refused. Any other flagged request is answered under an alert system note (system_note(category)), which tells the protected model what it must not provide.
Output stage. The exchange classifier checks the reply. The reply is withheld if its score exceeds exch_thr_20pct for alerted requests (a stricter threshold, flagging 20% of WildChat-clean replies), or exch_thr_1pct for unflagged requests (flagging 1%).

from eurosafeguard import EuroSafeGuard          # download eurosafeguard.py from this repo
g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, devices=("cuda:0", "cuda:0", "cuda:1"))
decision, category, score = g.route(prompt)      # "pass" | "alert" | "refuse"
messages = [{"role": "user", "content": prompt}]
if decision == "alert":
    messages.insert(0, {"role": "system", "content": g.system_note(category)})
reply = ...                                        # your protected model
if decision == "refuse" or g.check_reply(prompt, reply, decision):
    reply = "Sorry, I can't help with that."

Full mode (the paper's end-to-end configuration)

Probe screen. The probe scores every request. Only requests whose probe z-score exceeds probe_escalation_z (1.1644, the 90th percentile on WildChat-clean) are escalated; the rest are never blocked.
Escalated requests are scored by the input classifier and, once the protected model has replied, by the exchange classifier. The exchange is blocked if 0.6·probe_z + 0.2·input_z + 0.2·exchange_z > full_thr.
All three scores are z-scored on WildChat-clean (the exchange on Llama-3-8B-Instruct replies: μ = 0.02560, σ = 0.09032). full_thr is the smallest threshold that keeps every benign calibration set under its cap (WildChat-clean, MMLU, GSM8K, Alpaca, Dolly, OASST 0.5%; XSTest-safe, OR-Bench-80k 2%; OR-Bench-hard 5%; benign rows of styled jailbreak sets 40%): 3.382 / 3.571 / 3.132 / 3.097 / 3.318 for seeds 0–4. It was fitted once on the undefended Llama-3-8B-Instruct's traffic.

g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, mode="full", devices=("cuda:0", "cuda:0", "cuda:1"))
reply = ...                                        # your protected model
if g.check_full(prompt, reply):                    # probe on every request; gate + exchange only if escalated
    reply = "Sorry, I can't help with that."

full_block(probe, gate, exchange, cfg) applies the same rule to precomputed raw scores. On the paper's stored scores it reproduces the end-to-end attack success of every seed exactly (mean 3.88%).

Requirements: torch, transformers with Qwen3.5 support, peft, safetensors, huggingface_hub. About 16 GB of GPU memory for Llama-3.1-8B (the probe) plus about 19 GB for each Qwen3.5-9B classifier (input and exchange). Use load_exchange=False to run the input stage only. The probe reads Llama-3.1-8B-Instruct activations. If you protect Llama-3.1-8B-Instruct itself, you can reuse the activations of its forward pass.

Results (from the paper; every guard calibrated to the same false-positive budget on real chat traffic)

End to end (Full mode, Llama-3-8B-Instruct, 1,974 Jailbreak Foundry attacks on held-out GuidedBench behaviors): attack success falls from 7.95% to 3.88% (EuroSafe-CC++, our reimplementation of Constitutional Classifiers++, 4.57%), adding at most 3.2 points of benign failure on any of six benign benchmarks. On gpt-oss-20b: 13.17% → 5.28%.
Compute: only the 10% of requests the probe escalates reach the two classifiers, so the guardrail adds 15% of the protected model's FLOPs.
Held-out jailbreaks, input stage at the 5% WildChat-clean budget: 95.9% recall on the BELLS-O jailbreak set (88.2% for HaloGuard, the strongest open guard); highest recall on three of four held-out jailbreak sets.
PAIR (iterative black-box attack), input stage at the 5% WildChat-clean budget: 21% of the behaviors that break the unprotected model get through, against 49% for the best other open guard.
Chemical and biological goals transferred to three protected models: the full guardrail lets through at most 1.4% of goals on any model.
Alert mode (mode="alert") vs refusing every flagged request: refusals of benign sensitive requests fall from 37% to 5–16% on six protected models, while at most 3.6% of chemical and biological goals get through.
Latency: the full guardrail adds 1.5% to generation time on the requests it escalates.

Numbers are means over the five input-classifier seeds; see the paper for the full tables, judges and budgets.

Limitations

Low-resource languages: recall on multilingual harmful prompts (MSafeTemp) drops to 74% in low-resource languages, below several other guards.
Content moderation: on general moderation benchmarks (ToxicChat, OpenAI Moderation and others), dedicated moderation models remain ahead.
Over-refusal: the input stage flags a fraction of benign but sensitive requests. Alert mode answers most of them, but it is not a substitute for checking false positives on your own traffic.
Fixed policy: the thresholds are calibrated on WildChat-clean. Recalibrate on your own benign traffic if your traffic differs.
Not a complete defence: no guardrail stops every attack. Combine it with a safety-trained model, monitoring and rate limits.

License

Our adapters, heads, probe and code are released under Apache-2.0. The base models keep their own licenses: Llama 3.1 Community License for the probe's protected model, and the Qwen license for the classifier backbones. Some training data is licensed for non-commercial use only.

Citation

@article{waibl2026eurosafeguard,
  title   = {EuroSafeGuard: Open Constitutional Classifiers},
  author  = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and Michalak, Felix and
             Diwan, Nirav and Singh, Arth and Thomas, Aditya and
             Sch{\"o}lkopf, Bernhard and Jin, Zhijing},
  journal = {arXiv preprint},
  year    = {2026}
}