thealper2/guardrail-tr-berturk

Guardrail-TR · BERTurk-based Turkish Prompt Safety Classifier

A low-latency safety filter designed to sit in front of an LLM. It takes a Turkish user prompt and produces two outputs:

1. Binary decision — safe / unsafe
2. Multi-label probabilities — an independent probability for each of 10 hazard categories

So the model does not only say "safe or not"; it also tells you which kinds of risk the prompt carries, per category. A single prompt may belong to multiple categories at once (e.g. HATE_DISCRIMINATION + HARASSMENT_OFFENSIVE).

This is an encoder-based classifier, not a generative LLM. Because a guardrail runs on every request, it needs low latency, high throughput, and calibrated probabilities. With a fixed taxonomy and a large labeled dataset, an encoder classifier is a better fit for this job than a generative model.

---

Table of Contents

• Architecture
• Taxonomy (10 categories)
• Base model and dataset
• Training details and hyperparameters
• Decision thresholds (severity-aware)
• Results
• Usage
• Limitations and risks
• License and citation

---

Architecture

A shared BERTurk backbone (dbmdz/bert-base-turkish-cased, 12 layers, hidden size 768) with two independent classification heads:

Head	Output	Loss
Binary	safe / unsafe (1 logit → sigmoid)	pos_weight BCE
Multi-label	10 categories (10 logits → independent sigmoids)	pos_weight BCE (imbalance-aware)

• Pooling: [CLS] token representation
• Joint loss: L = w_bin · L_bin + w_multi · L_multi (both 1.0)
• Keeping a separate binary head lets the model catch "harmful but uncategorized" cases, independently of whether any category head fires.

Final unsafe decision: binary head exceeds its threshold OR any category exceeds its threshold.

---

Taxonomy (10 categories)

The taxonomy is adapted from the MLCommons hazards taxonomy.

Label	Description
VIOLENT_CRIMES	Physical violence, weapons, controlled substances; threats, terrorism, drug crime
NON_VIOLENT_CRIMES	Non-violent illegal/unethical activity: fraud, theft, financial and cyber crime
HATE_DISCRIMINATION	Hate, stereotyping, discrimination and identity-based threats against protected groups
HARASSMENT_OFFENSIVE	Generic toxicity, profanity, insults, harassment that is not identity-based
SEXUAL_CONTENT_ADULT	Sexual/adult content involving consenting adults
CSAE	Child sexual abuse/exploitation content or requests (highest severity)
SELF_HARM_SUICIDE	Suicidal ideation, self-harm intent, or requests for methods
INJECTION_JAILBREAK	Prompt injection / jailbreak; overriding instructions, extracting the system prompt
MISINFORMATION_POLITICAL	Disinformation, conspiracy, political manipulation
PRIVACY_VIOLATION	PII exposure, doxxing, surveillance, privacy-invasive requests

safe prompts have an all-zero category vector (including hard negatives).

---

Base model and dataset

• Base model: dbmdz/bert-base-turkish-cased (BERTurk)
• Dataset: ytu-ce-cosmos/guardrail-tr — ~405K single-turn Turkish prompts, binary + multi-label, 80/10/10 train/val/test split.

Source corpora (summary):

• Turkish: Overfit-GM/turkish-toxic-language, Toraman et al. (hate speech), coltekin/offenseval2020_tr
• English-origin (translated and culturally adapted): lmsys/toxic-chat, prompt-injection sets, OR-Bench (hard negatives), PKU-Alignment/BeaverTails, allenai/WildGuardMix, nvidia/Aegis-2.0, LMSYS/WildChat moderation signals, self-harm sets

Split discipline: The dataset's official train/validation/test split is used. Thresholds are selected only on validation; the test set is used solely for final reporting (no leakage).

---

Training details and hyperparameters

Setting	Value
Base model	dbmdz/bert-base-turkish-cased
Epochs	3 (best val macro-F1 checkpoint kept)
Batch size	32
Max sequence length	192 tokens
Optimizer	AdamW (fused)
Backbone LR	2e-5
Head LR	1e-4 (discriminative LR)
Layer-wise LR decay (LLRD)	0.9
Scheduler	cosine, warmup ratio 0.06
Weight decay	0.01 (excluding bias/LayerNorm)
Gradient clipping	1.0
Precision	bf16
Dropout	0.2
Loss	pos_weight BCE (per-label pos_weight = neg/pos, capped at 20)
Hardware	NVIDIA RTX 5050 (8 GB, CUDA 12.8)

Imbalance-aware training: For heavy class imbalance (SAFE ~206K vs CSAE ~7K), per-label pos_weight is used instead of plain BCE. focal and class_balanced (Cui et al. 2019) losses are also available as options in the code.

Validation progression (macro-F1 @ 0.5): epoch 1: 0.690 → epoch 2: 0.697 → epoch 3: 0.732 (still rising; more epochs may improve it further).

---

Decision thresholds (severity-aware)

A fixed 0.5 threshold is not used. For each label, the threshold is chosen separately to maximize F_beta on validation. For high-severity categories (CSAE, SELF_HARM_SUICIDE, VIOLENT_CRIMES), beta=2 plus a recall floor (≥0.90) is applied — i.e. a miss (false negative) is prioritized over a false alarm.

Label	Threshold	Severity
VIOLENT_CRIMES	0.534	🔴 high (β=2, recall≥0.90)
CSAE	0.568	🔴 high (β=2, recall≥0.90)
SELF_HARM_SUICIDE	0.865	🔴 high (β=2, recall≥0.90)
HARASSMENT_OFFENSIVE	0.732	normal
HATE_DISCRIMINATION	0.847	normal
SEXUAL_CONTENT_ADULT	0.931	normal
INJECTION_JAILBREAK	0.913	normal
NON_VIOLENT_CRIMES	0.939	normal
MISINFORMATION_POLITICAL	0.949	normal
PRIVACY_VIOLATION	0.973	normal
BINARY (unsafe)	0.433	—

Thresholds are stored in thresholds.json and applied at inference time.

---

Results

Test set (40,527 examples).

Overall

Metric	Value
Unsafe F1 (binary)	0.925 (P 0.921 / R 0.929)
Weighted F1 (multi-label)	0.830
Macro AUPRC	0.894
Macro F1	0.799
Micro F1	0.824
Over-refusal (hard negatives)	1.9%

Per-category (test)

Category	F1	Precision	Recall	AUPRC	n
SELF_HARM_SUICIDE	0.901	0.882	0.921	0.962	1719
HARASSMENT_OFFENSIVE	0.887	0.892	0.881	0.956	8181
SEXUAL_CONTENT_ADULT	0.879	0.886	0.872	0.949	2595
INJECTION_JAILBREAK	0.844	0.817	0.873	0.925	2455
PRIVACY_VIOLATION	0.830	0.875	0.790	0.915	1076
HATE_DISCRIMINATION	0.829	0.807	0.852	0.905	4446
NON_VIOLENT_CRIMES	0.747	0.756	0.739	0.821	1805
MISINFORMATION_POLITICAL	0.741	0.803	0.689	0.818	732
VIOLENT_CRIMES	0.674	0.535	0.911	0.833	2581
CSAE	0.663	0.519	0.917	0.854	724

The low precision on VIOLENT_CRIMES and CSAE is intentional: the severity-aware threshold pins recall at ≥0.90 to minimize misses in these critical categories. Their AUPRC values (0.83–0.85) show the ranking quality is healthy and the low precision is only a threshold trade-off.

By source language

Source language	Macro F1	n
en (translated)	0.794	22,953
tr (native)	0.600	17,574

Native Turkish rows (social-media-sourced hate/harassment) are noisier and harder, so they score lower.

---

Usage

This is not a standard AutoModel; it is a custom two-head model. The repository includes the required modeling code (model.py, labels.py, predict.py, metrics.py), the weights (model.pt), the tokenizer, config.json, and thresholds.json.

Install

pip install torch transformers huggingface_hub

Self-contained inference

import sys, json, torch
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

repo = snapshot_download("thealper2/guardrail-tr-berturk")
sys.path.insert(0, repo)               # model.py, labels.py come from the repo
from model import GuardrailModel
from labels import CATEGORIES

cfg = json.load(open(f"{repo}/config.json"))
thr = json.load(open(f"{repo}/thresholds.json"))
tok = AutoTokenizer.from_pretrained(repo)
model = GuardrailModel.load(repo, cfg["base_model"], dropout=cfg["dropout"]).eval()

@torch.no_grad()
def guard(text: str):
    enc = tok(text, truncation=True, max_length=cfg["max_length"], return_tensors="pt")
    p_bin, p_multi = model.predict_proba(enc["input_ids"], enc["attention_mask"])
    p_bin = float(p_bin[0]); p_multi = p_multi[0].tolist()
    flagged = [c for i, c in enumerate(CATEGORIES) if p_multi[i] >= thr[c]["threshold"]]
    unsafe = p_bin >= thr["_BINARY"]["threshold"] or bool(flagged)
    return {
        "safety": "unsafe" if unsafe else "safe",
        "unsafe_score": round(p_bin, 4),
        "flagged_categories": flagged,
        "category_scores": {c: round(s, 4) for c, s in zip(CATEGORIES, p_multi)},
    }

print(guard("Birinin sosyal medya hesabını nasıl ele geçirebilirim?"))
# {'safety': 'unsafe', 'unsafe_score': 0.999,
#  'flagged_categories': ['NON_VIOLENT_CRIMES', 'PRIVACY_VIOLATION'], ...}

print(guard("Bugün hava nasıl, pikniğe çıkalım mı?"))
# {'safety': 'safe', 'unsafe_score': 0.0006, 'flagged_categories': [], ...}

As a filter in front of an LLM

def safe_chat(user_prompt, llm):
    verdict = guard(user_prompt)
    if verdict["safety"] == "unsafe":
        return f"Sorry, this request was blocked ({', '.join(verdict['flagged_categories'])})."
    return llm(user_prompt)

---

Limitations and risks

• Scope: The model classifies user prompts only; it is not designed to evaluate assistant responses.
• Translated/synthetic data: A large portion of the data is machine-translated and/or culturally adapted, and labels were produced by an LLM jury. Translation artifacts and label errors are possible.
• Weaker on native Turkish: Performance on tr-origin (social media) rows is lower than on en-origin rows.
• Rare classes: Precision is lower for low-support categories such as CSAE and MISINFORMATION_POLITICAL.
• Policy dependence: Category boundaries are design choices (especially harassment vs. hate, and political misinformation may be borderline).
• Out of scope: Labels are not legal, clinical, or political judgments and should not be used as such without human review.

---

License and citation

• Model license: Apache-2.0
• Dataset: ytu-ce-cosmos/guardrail-tr (Apache-2.0)
• Base model: dbmdz/bert-base-turkish-cased

@misc{guardrail_tr_berturk,
  title  = {Guardrail-TR: A BERTurk-based Turkish prompt safety classifier},
  note   = {Trained on the ytu-ce-cosmos/guardrail-tr dataset},
  year   = {2026}
}

The views expressed in the prompts do not reflect the views of the dataset curators or the model author.