Instructions to use patronus-studio/tiger-guard-content-moderation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use patronus-studio/tiger-guard-content-moderation with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="patronus-studio/tiger-guard-content-moderation")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("patronus-studio/tiger-guard-content-moderation") model = AutoModelForSequenceClassification.from_pretrained("patronus-studio/tiger-guard-content-moderation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Quickstart · Evaluation · Model variants · Limitations · Product page
Flag harmful text before it reaches your model or your users
Tiger Guard is a multilingual content-moderation classifier for English and German that labels a text SAFE or UNSAFE. It checks both sides of an LLM application, the user's prompt and the model's response, and it works on ordinary user text such as comments. Inference runs on your own hardware.
UNSAFE covers two kinds of content:
- abusive language: toxicity, hate against protected groups, insults, profanity, threats, sexism, sexual harassment and sexually explicit text
- unsafe requests and answers in LLM conversations, for example requests for help with crimes, weapons or self-harm, or for other people's private data
Counter speech, quoting abuse in order to report it and neutral mentions of a group are meant to stay SAFE.
| Model at a glance | |
|---|---|
| Task | Binary content-safety classification |
| Backbone | mmBERT-base, ModernBERT architecture |
| Context window | 2,048 tokens per window |
| Output | SAFE or UNSAFE |
| Parameters | 307,531,778 (~308M) |
| Languages | English, German |
| Formats | Transformers and ONNX, including quantised variants |
| License | Apache 2.0 |
Your application decides whether flagged text is blocked, rewritten or sent for review.
Evaluation
Model family at a glance
| Model | Parameters | Benchmark accuracy | False positives (of 750) | Recall (of 750) | Validation recall EN / DE | Held-out test recall EN / DE |
|---|---|---|---|---|---|---|
| Tiger Guard | 308M | 88.80% | 46 | 83.7% | 80.3% / 64.2% | 83.3% / 75.3% |
| Small | 141M | 87.60% | 47 | 81.5% | 79.8% / 60.7% | 83.4% / 75.3% |
Each model at its default threshold, which flags 4% of the harmless texts in our validation set. The three evaluations are described below.
Benchmark against public moderation classifiers
The benchmark has 1,500 prompts: 500 each from the HateCheck test suite, the validation split of Intuit's combined toxicity and profanity set and the test split of NVIDIA Aegis AI Content Safety 1.0. In every set half of the prompts are to block and half to allow. We ran five public classifiers on the same prompts, each with its documented default decision.
| Model | Parameters | Accuracy | Unsafe caught (of 750) | Unsafe missed | Harmless flagged (of 750) | Errors in total |
|---|---|---|---|---|---|---|
| Tiger Guard | 308M | 88.80% | 83.7% | 122 | 46 | 168 |
| Tiger Guard Small | 141M | 87.60% | 81.5% | 139 | 47 | 186 |
| Granite Guardian HAP 125m | 125M | 80.27% | 62.4% | 282 | 14 | 296 |
| Granite Guardian HAP 38m | 38M | 75.13% | 58.8% | 309 | 64 | 373 |
| RoBERTa hate speech (Dynabench R4) | 125M | 74.40% | 51.9% | 361 | 23 | 384 |
| toxic-bert | 109M | 71.60% | 62.1% | 284 | 142 | 426 |
| KoalaAI Text-Moderation | 139M | 64.40% | 40.0% | 450 | 84 | 534 |
How to read this:
- The full-size model is 8.5 points above the best public classifier in this run (95% CI +6.5 to +10.6 on identical prompts). Small is 7.3 points above it (+5.3 to +9.5).
- Count both kinds of error. Granite HAP 125m and the hate-speech RoBERTa flag fewer harmless prompts than the full-size Tiger Guard model (14 and 23 against 46), and they miss 282 and 361 of the 750 unsafe prompts where Tiger Guard misses 122. In total Tiger Guard makes 168 errors, the best public classifier 296.
- With a stricter preset the full-size model is ahead on both counts. Its 1% preset flags 5 harmless prompts and catches 64% of the unsafe ones (Granite HAP 125m: 14 and 62%). Its 2% preset flags 21 and catches 74% (hate-speech RoBERTa: 23 and 52%). Small flags 7 and catches 60% at its 1% preset, 18 and 71% at 2%.
- The public models are specialists. Each is strong on its own kind of text and close to chance on Aegis, where the unsafe prompts are mostly requests for harmful help.
Both Tiger Guard models stay above 80% on every test set. The lowest value is 83.4% for the full-size model and 82.6% for Small. The public classifiers fall to 54% to 57% on their weakest set.
| Accuracy | HateCheck (FP / FN) | Intuit (FP / FN) | Aegis (FP / FN) | ROC-AUC pooled (HateCheck / Intuit / Aegis) | |
|---|---|---|---|---|---|
| Tiger Guard | 88.80% | 92.2% (26 / 13) | 90.8% (4 / 42) | 83.4% (16 / 67) | 0.957 (0.972 / 0.990 / 0.925) |
| Small | 87.60% | 89.4% (26 / 27) | 90.8% (4 / 42) | 82.6% (17 / 70) | 0.951 (0.954 / 0.991 / 0.918) |
Where the full-size model still errs:
- HateCheck: 13 of the 26 false positives are cases HateCheck allows and our policy moderates (profanity, abuse of individuals, of non-protected groups and of objects), 9 are counter speech. 11 of the 13 misses are spelling variants such as leetspeak or swapped characters.
- Intuit: 28 of the 42 misses carry only the label
toxic, mild toxicity without insult, profanity or hate. - Aegis: 33 of the 67 misses are labelled "Needs Caution". The benchmark scores against the first annotator; 5 of the 16 false positives are unsafe by annotator majority.
What stands behind these numbers:
- No benchmark text was used for training. All HateCheck cases, the Aegis 1.0 test split and the Intuit validation split were removed from every training source, including near-duplicates and template matches. We do not know what the public models saw in training; some may have seen these test sets or closely related data.
- The benchmark did steer development. Its error breakdowns decided which data we added, and every training round was checked against it. Treat the Tiger Guard numbers on it as a development-set result. The held-out test below was scored once per model, after training.
- 1,500 prompts give about ±1.7 points at 95%. The interval is 87.1 to 90.3% for the full-size model and 85.8 to 89.2% for Small.
Held-out test: harmful requests in disguise
PolyGuardPrompts is the test split of the WildGuard family: 1,725 prompts per language, labelled by people, about half of them harmful or harmless requests wrapped in role play, stories or instructions. We use the English originals and the checked German translations and leave out harmful prompts whose categories all lie outside our policy (defamation, specialised advice, intellectual property, elections, code execution) or that carry no category. That leaves 3,123 prompts, 1,233 of them harmful.
| Model | English: harmful caught | English: harmless flagged | German: harmful caught | German: harmless flagged |
|---|---|---|---|---|
| Tiger Guard | 83.3% | 3.8% | 75.3% | 5.5% |
| Tiger Guard Small | 83.4% | 5.0% | 75.3% | 7.1% |
| RoBERTa hate speech (Dynabench R4) | 1.9% | 0.8% | 1.3% | 0.5% |
| toxic-bert | 0.2% | 0.0% | 2.9% | 1.5% |
| KoalaAI Text-Moderation | 1.1% | 0.1% | 0.0% | 0.0% |
| Granite Guardian HAP 125m | 1.0% | 0.6% | 0.0% | 0.0% |
| Granite Guardian HAP 38m | 0.2% | 0.2% | 0.0% | 0.0% |
- Both models were trained on other prompts from the same family of data. The test prompts themselves were never used for training. Each model was scored on them once, as a pass-or-fail check whose bar had been fixed before training.
- The public classifiers catch almost none of these prompts. They were built to spot abusive language, and a harmful request is usually polite, so this test lies outside what they were made for. LLM-based guard models built for such requests are a different size class and were not run.
Three public benchmarks
Three public test sets that other guard models report: the English and German prompts of PolyGuardPrompts (400 per language, all harm categories), ToxicChat (5,083 real user prompts from a chatbot demo, 362 of them toxic) and the OpenAI moderation evaluation set (1,680 texts, 522 of them flagged). The table shows F1 of the unsafe class, each model at its default decision. We ran every model ourselves on the same texts.
| Model | Parameters | PolyGuard prompts, English | PolyGuard prompts, German | ToxicChat | OpenAI moderation set |
|---|---|---|---|---|---|
| Tiger Guard | 308M | 0.871 | 0.783 | 0.736 | 0.787 |
| Tiger Guard Small | 141M | 0.867 | 0.788 | 0.720 | 0.771 |
| Content Safety Guard base | 308M | 0.797 | 0.774 | 0.756 | 0.769 |
| Content Safety Guard small | 141M | 0.811 | 0.772 | 0.777 | 0.763 |
| Granite Guardian HAP 125m | 125M | 0.023 | 0.000 | 0.261 | 0.671 |
| Unbiased Toxic RoBERTa | 125M | 0.035 | 0.000 | 0.254 | 0.664 |
| toxic-bert | 109M | 0.012 | 0.023 | 0.266 | 0.646 |
| KoalaAI Text-Moderation | 139M | 0.023 | 0.000 | 0.469 | 0.938 † |
† KoalaAI Text-Moderation lists the OpenAI moderation evaluation set as training data in its model card, so this value is not a test result.
Share of harmless texts flagged:
| Model | PolyGuard prompts, English + German | ToxicChat | OpenAI moderation set |
|---|---|---|---|
| Tiger Guard | 4.1% | 2.8% | 11.8% |
| Tiger Guard Small | 5.0% | 2.5% | 11.5% |
| Content Safety Guard base | 8.4% | 2.7% | 16.6% |
| Content Safety Guard small | 8.4% | 2.3% | 17.5% |
| Granite Guardian HAP 125m | 0.7% | 0.4% | 6.9% |
| Unbiased Toxic RoBERTa | 0.2% | 0.8% | 9.2% |
| toxic-bert | 0.9% | 0.6% | 7.8% |
| KoalaAI Text-Moderation | 0.0% | 1.3% | 2.5% |
How to read this:
- PolyGuard prompts: both Tiger Guard models are ahead in English and in German. They flag 4.1% and 5.0% of the harmless prompts, the Content Safety Guard models 8.4% and 8.4%. Both Tiger Guard models were trained on other prompts from the same family of data; the Content Safety Guard models were not, according to their model card.
- OpenAI moderation set: among the models that were not trained on this set, the full-size model has the highest F1 (0.787), and both Tiger Guard models flag fewer harmless texts (11.8% and 11.5% against 16.6% and 17.5%). The F1 lead over Content Safety Guard base is within the sampling error.
- ToxicChat: Content Safety Guard small is ahead (0.777 against 0.736 for the full-size model and 0.720 for Small). For the full-size model the gap to Content Safety Guard base is within the sampling error; Small is behind.
- The toxicity classifiers reach 0.254 to 0.469 on ToxicChat and almost nothing on PolyGuard prompts. They were built for abusive comments, and most of these prompts are requests.
What stands behind these numbers:
- ToxicChat and the OpenAI moderation set played no part in development. Each model was scored on them once, after release. PolyGuardPrompts is the source of the held-out test above; this table uses the usual sample with all harm categories, the held-out test leaves out categories outside our policy.
- 69 of the 5,083 ToxicChat prompts and 12 of the 1,680 texts of the OpenAI moderation set also occur in our training data. Without them the F1 values of both Tiger Guard models change by at most 0.003.
- LLM-based guard models with a billion parameters or more were not run.
Operating points of this model
The default threshold of 0.5 on the UNSAFE probability is the 4% point. For a stricter or looser filter, compare the UNSAFE probability with another threshold from this table.
| False-positive rate on our validation set | Threshold on UNSAFE probability |
Benchmark accuracy | Benchmark false positives | Benchmark recall | Validation recall |
|---|---|---|---|---|---|
| 0.5% | 0.948 | 75.13% | 1 | 50.4% | 43.7% |
| 1% | 0.891 | 81.47% | 5 | 63.6% | 53.7% |
| 2% | 0.769 | 85.60% | 21 | 74.0% | 63.9% |
| 3% | 0.634 | 87.67% | 33 | 79.7% | 70.2% |
| 4% (default) | 0.500 | 88.80% | 46 | 83.7% | 74.9% |
| 5% | 0.403 | 89.07% | 59 | 86.0% | 77.6% |
The threshold values come from our validation set (17,389 harmless and 11,607 unsafe texts). The 4% point as default is a choice, made with benchmark results of earlier models in view.
English and German
28,996 held-out texts with a decision label, 18,823 English and 10,173 German, at each model's default threshold.
| Model | Recall | Recall EN | Recall DE | False-positive rate EN / DE |
|---|---|---|---|---|
| Tiger Guard | 74.9% | 80.3% | 64.2% | 3.8% / 4.4% |
| Small | 73.3% | 79.8% | 60.7% | 3.8% / 4.3% |
German recall is lower than English. On identical prompts (the held-out test) the gap is 8 points: both models catch 83% in English and 75% in German. On this validation set it is wider, partly because the German texts come from other sources than the English ones. On the validation texts whose sources exist in both languages (LLM prompts and responses and our synthetic texts) the full-size model catches 80% in English and 74% in German, Small 78% and 70%. The benchmark above is English only and does not show this.
Quickstart
Transformers
pip install transformers torch
from transformers import pipeline
moderator = pipeline(
"text-classification",
model="patronus-studio/tiger-guard-content-moderation",
)
result = moderator(
"You are a worthless idiot and everyone here hates you.",
truncation=True,
max_length=2048,
)
print(result)
The pipeline returns the more likely label and its score. UNSAFE means the default threshold is exceeded. For another operating point, read the UNSAFE probability (top_k=None returns both labels) and compare it with a threshold from the table above.
Loaded this way, with the top label as the decision, this repository gives 88.80% with 46 false positives on the benchmark.
This minimal example truncates long inputs. Use inference.py when harmful content may appear anywhere in a longer document.
Long documents
inference.py in this repository scores every window of a long input and pools the window logits; POOLING.json states the contract (2,048 positions per window, 64 tokens overlap).
import sys
from huggingface_hub import snapshot_download
path = snapshot_download(
"patronus-studio/tiger-guard-content-moderation",
allow_patterns=["*.json", "*.py", "model.safetensors"],
)
sys.path.insert(0, path)
from inference import TigerGuard
tiger = TigerGuard(path)
print(tiger.predict(["first text", "second text"]))
Each result has label, unsafe_probability, is_unsafe and the number of windows.
ONNX Runtime
pip install onnxruntime transformers
import sys
from huggingface_hub import snapshot_download
path = snapshot_download(
"patronus-studio/tiger-guard-content-moderation",
allow_patterns=["*.json", "*.py", "onnx/onnx_fp16/*"],
)
sys.path.insert(0, path)
from inference import TigerGuard
tiger = TigerGuard(path, onnx_path="onnx/onnx_fp16/model_fp16.onnx")
print(tiger.predict("first text"))
The ONNX graphs take input_ids and attention_mask and return two logits per input, in the order SAFE, UNSAFE.
Choose your model
| Model | Parameters | Context per window | Benchmark accuracy / false positives | Held-out test recall EN / DE | License |
|---|---|---|---|---|---|
| Tiger Guard | ~308M | 2,048 | 88.80% / 46 | 83.3% / 75.3% | Apache 2.0 |
| Tiger Guard Small | ~141M | 2,048 | 87.60% / 47 | 83.4% / 75.3% | Apache 2.0 |
Take the full-size model when accuracy matters most: it is 1.2 points ahead on the benchmark and flags fewer harmless prompts on the held-out test (4.7% against 6.0%). Small needs less than half the memory and is the faster of the two.
Deployment variants
| Variant | Repository path | Size | Same decision as Transformers (benchmark windows / validation texts) | Largest probability difference | Benchmark accuracy / false positives | CPU time per window |
|---|---|---|---|---|---|---|
| FP32 | onnx/onnx_fp32/model.onnx |
1230.2 MB | 100.00% / 100.00% | 0.0000 | 88.80% / 46 | 24 ms |
| FP16 | onnx/onnx_fp16/model_fp16.onnx |
615.2 MB | 100.00% / 100.00% | 0.0052 | 88.80% / 46 | 53 ms |
| INT8, dynamic (ARM only) | onnx/onnx_int8/model_int8.onnx |
307.8 MB | 98.69% / 97.83% | 0.2765 | 88.87% / 55 | 20 ms |
| INT8 + INT4 embeddings, dynamic (ARM only) | onnx/int8_int4_embeddings/model.onnx |
237.1 MB | 98.82% / 98.33% | 0.2823 | 88.20% / 55 | 19 ms |
| INT8 weights, x86-safe | onnx/onnx_int8_x86/model_int8.onnx |
509.6 MB | 99.80% / 99.67% | 0.1451 | 88.73% / 47 | 62 ms |
| INT8 weights + INT4 embeddings, x86-safe | onnx/int8_int4_embeddings_x86/model.onnx |
242.3 MB | 99.67% / 99.50% | 0.1390 | 88.47% / 48 | 63 ms |
Measured on the files in this repository with inference.py: 1,530 benchmark windows and 600 texts from our validation set (446 English, 154 German, all within one window). CPU time is ONNX Runtime 1.31 on an Apple M4, one window at a time, averaged over the benchmark windows (median 25 tokens, 95th percentile 269). These windows are short; a full 2,048-token window takes far longer.
FP32 reproduces the Transformers weights and was the fastest exact variant on CPU. FP16 halves the download and keeps the decisions; it ran slower than FP32 in this CPU test. The two dynamic INT8 files are a quarter of the size and change one to three in a hundred decisions on ARM; on the benchmark they flag 55 harmless prompts each where the exact variants flag 46. They are not for x86 (next section). The two x86-safe files store the linear weights as INT8 and compute in floating point. They stay closer to the exact variants (47 and 48 benchmark false positives) and took 2.5 to 2.6 times as long as FP32 here.
x86 CPUs
On x86, use FP32, FP16 or one of the two x86-safe files. Do not use the two dynamic INT8 files (onnx_int8, int8_int4_embeddings) there: they quantise activations at run time, and with ONNX Runtime's integer kernels on x86 their scores drift away from the model's. On ARM they work as shown above. We checked this on a server CPU and on an emulated older x86 code path.
Server CPU. AMD EPYC 9554 (AVX2 and AVX-512 with VNNI), ONNX Runtime 1.31.1, 8 threads, 3,000 texts from our validation set, compared with the Transformers weights. The ARM column is the same 3,000 texts on the Apple M4.
| Variant | Repository path | Same decision as Transformers on the EPYC | Largest probability difference on the EPYC | Same decision on ARM | Time per text on the EPYC |
|---|---|---|---|---|---|
| FP32 | onnx/onnx_fp32/model.onnx |
100.00% | 0.0000 | 100.00% | 45 ms |
| FP16 | onnx/onnx_fp16/model_fp16.onnx |
99.97% | 0.0028 | 99.97% | 48 ms |
| INT8, dynamic (ARM only) | onnx/onnx_int8/model_int8.onnx |
90.40% | 0.9213 | 97.87% | 31 ms |
| INT8 + INT4 embeddings, dynamic (ARM only) | onnx/int8_int4_embeddings/model.onnx |
90.50% | 0.9491 | 97.80% | 31 ms |
| INT8 weights, x86-safe | onnx/onnx_int8_x86/model_int8.onnx |
99.73% | 0.0589 | 99.73% | 82 ms |
| INT8 weights + INT4 embeddings, x86-safe | onnx/int8_int4_embeddings_x86/model.onnx |
99.37% | 0.2080 | 99.37% | 79 ms |
x86 without AVX2 and VNNI. An x86-64 build of ONNX Runtime 1.23.2 under Rosetta 2 on the Apple M4. That is an emulated code path with SSE4.2, not a server CPU. Same 1,530 benchmark windows and 600 validation texts as in the table of variants:
| Variant | Repository path | Same decision as Transformers (benchmark windows / validation texts) | Largest probability difference | Benchmark accuracy / false positives |
|---|---|---|---|---|
| FP32 | onnx/onnx_fp32/model.onnx |
100.00% / 100.00% | 0.0000 | 88.80% / 46 |
| FP16 | onnx/onnx_fp16/model_fp16.onnx |
100.00% / 100.00% | 0.0034 | 88.80% / 46 |
| INT8, dynamic (ARM only) | onnx/onnx_int8/model_int8.onnx |
93.40% / 91.00% | 0.9582 | 85.33% / 33 |
| INT8 + INT4 embeddings, dynamic (ARM only) | onnx/int8_int4_embeddings/model.onnx |
92.68% / 91.83% | 0.9637 | 84.93% / 33 |
| INT8 weights, x86-safe | onnx/onnx_int8_x86/model_int8.onnx |
99.80% / 99.67% | 0.1451 | 88.73% / 47 |
| INT8 weights + INT4 embeddings, x86-safe | onnx/int8_int4_embeddings_x86/model.onnx |
99.67% / 99.50% | 0.1390 | 88.47% / 48 |
On the EPYC the dynamic INT8 files change 9.5 to 9.6 in a hundred decisions. On the emulated path they lose 3.3 and 3.5 points of benchmark accuracy against their own result on ARM and move single probabilities by up to 0.96. The x86-safe files agree with the Transformers weights on at least 99.4% of the texts in both checks, and under emulation they decide all 2,130 texts as they do on ARM. On the EPYC they took 1.7 to 1.8 times as long as FP32. This is one server CPU and one emulated path, so check the exact file on your own machines.
The quantisation manifest lists recipes, sizes and checksums; evaluation/EXPORT_VALIDATION.json has the measurements.
Technical model card
Intended use and scope
Tiger Guard performs binary sequence classification. It scores text; it does not generate text and it does not know who is speaking or what your platform allows.
| ID | Label | Meaning |
|---|---|---|
| 0 | SAFE |
No moderated content detected |
| 1 | UNSAFE |
Abusive language or an unsafe prompt or response detected |
Use it as one layer of a moderation setup. Our policy moderates insults and profanity even when no protected group is involved, which is stricter than hate-speech-only policies. If your platform allows swearing, expect disagreements on those texts.
Training data
The full-size model was trained on 413,392 texts, 292,718 English and 120,674 German, from public datasets plus our own synthetic and augmented texts:
- LLM prompts and responses, harmful and harmless
- comments and posts labelled for toxicity, hate speech, insults and related categories
- German texts of both kinds
- our own texts: synthetic examples for specific phenomena (counter speech, reclaimed slurs, harmless texts that look harmful), obfuscated copies (leetspeak, spacing, character swaps) and sets of contrasting variants of the same statement
Small was trained on the same texts plus 250,007 further texts: obfuscated copies of training texts and further public comments and dialogues.
Texts that match the benchmark (all HateCheck cases, the Aegis 1.0 test split, the Intuit validation split) were removed, including near-duplicates. The training data and the list of its sources are not published.
Limitations
- No classifier catches everything. Spelling tricks and mild toxicity are the most common misses.
- Harmless text can be flagged, most often counter speech that quotes abuse and blunt but harmless requests.
- German is weaker than English. On identical prompts both models catch 75% in German against 83% in English. On our validation set, whose German part comes from other sources, recall is 64% against 80% for the full-size model and 61% against 80% for Small. Other languages were not trained or tested.
- Small flags more harmless prompts than the full-size model on the held-out test (6.0% against 4.7%).
- Small can flag text that repeats one word or sentence thousands of times even when it is harmless (1 of 6 such inputs in our check); the full-size model flagged none of them. Long documents of varied text are not affected: of 30 documents per length, 300 to 8,000 tokens, assembled from individually harmless texts, at most one was flagged.
- Borderline requests ("Needs Caution" in Aegis) are missed in about two of three cases.
- The benchmark numbers are a development-set result, as explained under Evaluation.
- The model sees text only. It knows nothing about the speaker, the audience or your platform's rules beyond the policy it was trained on.
- The model scores one text at a time. An answer that is harmful only in the context of its question, such as a short confirmation, is not recognised from the answer alone. If you pass prompt and response as one text, it is flagged as soon as either side is unsafe.
- Quantised ONNX files can differ from the Transformers weights. Check the exact file and runtime you deploy, and do not use the dynamic INT8 files on x86 CPUs (see the x86 section).
Do not use the model as a prompt-injection detector, a factuality judge or a replacement for human review of consequential decisions.
Changelog
v1.0
- First release: Transformers weights, FP32, FP16, INT8 and INT8 with INT4 embeddings ONNX files, and two weight-only INT8 ONNX files for x86 CPUs.
Citation and license
Tiger Guard is released under Apache 2.0. The upstream mmBERT-base model is distributed under MIT; the upstream notices remain applicable to its portions.
@misc{tigerguard2026,
title = {Tiger Guard: Multilingual Content-Safety Classification for LLM Applications},
author = {Patronus Protect},
year = {2026},
howpublished = {\url{https://huggingface.co/patronus-studio/tiger-guard-content-moderation}}
}
🛡️ Patronus Protect
Brought to you by Patronus Protect, a local AI firewall that secures prompts, tools and documents before they reach your models.
Try Patronus Protect for free →
Explore our security models or integrate them through Patronus Ark.
- Downloads last month
- 6
Model tree for patronus-studio/tiger-guard-content-moderation
Base model
jhu-clsp/mmBERT-baseCollection including patronus-studio/tiger-guard-content-moderation
Evaluation results
- Accuracy on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.888
- Unsafe prompts caught on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.837
- Precision on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.932
- F1 on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.882
- Harmless prompts flagged on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.061
- AUROC on Content-safety benchmark, 1,500 prompts (HateCheck, Intuit toxicity and profanity, Aegis 1.0)test set self-reported0.957
- Accuracy on HateCheck (500 test cases, balanced)test set self-reported0.922
- AUROC on HateCheck (500 test cases, balanced)test set self-reported0.972










