EuroSafeGuard

EuroSafeGuard is an open constitutional-classifier guardrail from EuroSafeAI. It screens the requests sent to a language model and the replies the model produces, and it is trained on data generated from a written constitution of eleven harm categories.

Paper: EuroSafeAI Open Constitutional Classifiers (arXiv: TODO) · Code: TODO · Data: EuroSafeGuard-prompts, EuroSafeGuard-replies (gated)

Components

Folder	What it is	Base model
probe/	MLP probe (hidden size 256) on the activations of all 32 blocks of the protected model, reading the chat-formatted prompt; per-token logits are smoothed with an exponential moving average and the prompt score is their maximum	Llama-3.1-8B-Instruct
gate/seed{0..4}/	Input classifier: LoRA adapter + one sigmoid head per harm category (11 heads); score = highest head	Qwen3.5-9B
exchange/	Exchange classifier: reads User: <prompt>\n\nAssistant: <reply>; LoRA + 11 heads	Qwen3.5-9B
thresholds/seed{k}.json	Score normalisation and thresholds for each input-classifier seed
eurosafeguard.py	Single-file loader

The five gate/seed* folders are the five training runs (same data, different seeds) that the paper reports as a mean. They vary, especially against search attacks such as PAIR, so evaluate the seed you deploy.

Harm categories: CBRN, Cyber, Harm to Minors, Harmful Manipulation, Hate Speech, Illegal Activities, Information Integrity, Physical Harm, Privacy, Self-Harm, Sexual Content. taxonomy_11heads_short.json lists what each category allows and disallows.

How it decides (alert-unless-CBRN)

1. Input stage. The input score is the mean of the probe and input-classifier scores, each z-scored on WildChat-clean. Its threshold (alert_input_thr) flags 5% of WildChat-clean prompts.
2. A flagged request whose top category is CBRN is refused. Any other flagged request is answered under an alert system note (system_note(category)), which tells the protected model what it must not provide.
3. Output stage. The exchange classifier checks the reply. The reply is withheld if its score exceeds exch_thr_20pct for alerted requests (a stricter threshold, flagging 20% of WildChat-clean replies), or exch_thr_1pct for unflagged requests (flagging 1%).

from eurosafeguard import EuroSafeGuard          # download eurosafeguard.py from this repo
g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, devices=("cuda:0", "cuda:0", "cuda:1"))
decision, category, score = g.route(prompt)      # "pass" | "alert" | "refuse"
messages = [{"role": "user", "content": prompt}]
if decision == "alert":
    messages.insert(0, {"role": "system", "content": g.system_note(category)})
reply = ...                                        # your protected model
if decision == "refuse" or g.check_reply(prompt, reply, decision):
    reply = "Sorry, I can't help with that."

Requirements: torch, transformers with Qwen3.5 support, peft, safetensors, huggingface_hub. About 16 GB of GPU memory for Llama-3.1-8B (the probe) plus about 19 GB for each Qwen3.5-9B classifier (input and exchange). Use load_exchange=False to run the input stage only. The probe reads Llama-3.1-8B-Instruct activations. If you protect Llama-3.1-8B-Instruct itself, you can reuse the activations of its forward pass.

Results (from the paper; every guard calibrated to the same false-positive budget on real chat traffic)

• BELLS-O: our input classifiers detect more harmful prompts than any supervisor on the leaderboard at a false-positive rate near 1%, and have the highest recall on its held-out jailbreak sets.
• PAIR (iterative black-box attack), input stage at the 5% WildChat-clean budget: 21% of the behaviors that break the unprotected model get through, against 49% for the best other open guard.
• Chemical and biological goals transferred to three target models: at most 1.1% get through in every setting.
• Alert mode (this release's rule) vs refusing every flagged request: refusals of benign sensitive requests fall from 37% to 5–9%, while at most 3.6% of chemical and biological goals get through on five protected models.
• Latency: the full guardrail adds 1.5% to generation time on the requests it escalates.

Numbers are means over the five input-classifier seeds; see the paper for the full tables, judges and budgets.

Limitations

• Low-resource languages: recall on multilingual harmful prompts (MSafeTemp) drops to 74% in low-resource languages, below several other guards.
• Content moderation: on general moderation benchmarks (ToxicChat, OpenAI Moderation and others), dedicated moderation models remain ahead.
• Over-refusal: the input stage flags a fraction of benign but sensitive requests. Alert mode answers most of them, but it is not a substitute for checking false positives on your own traffic.
• Fixed policy: the thresholds are calibrated on WildChat-clean. Recalibrate on your own benign traffic if your traffic differs.
• Not a complete defence: no guardrail stops every attack. Combine it with a safety-trained model, monitoring and rate limits.

License

Our adapters, heads, probe and code are released under Apache-2.0. The base models keep their own licenses: Llama 3.1 Community License for the probe's protected model, and the Qwen license for the classifier backbones. Some training data is licensed for non-commercial use only; see the dataset cards.

Citation

@article{eurosafeguard2026,
  title   = {EuroSafeAI Open Constitutional Classifiers},
  author  = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and others},
  journal = {arXiv preprint arXiv:TODO},
  year    = {2026}
}