| --- |
| license: apache-2.0 |
| library_name: transformers |
| base_model: jhu-clsp/mmBERT-base |
| base_model_relation: finetune |
| pipeline_tag: text-classification |
| language: |
| - multilingual |
| - en |
| - ar |
| - am |
| - cs |
| - de |
| - es |
| - fr |
| - he |
| - hi |
| - id |
| - it |
| - ja |
| - ko |
| - nl |
| - pl |
| - pt |
| - ru |
| - sv |
| - tt |
| - uk |
| - zh |
| datasets: |
| - google/civil_comments |
| - Horizon-Labs/civil-comments-multilingual |
| - afrihate/afrihate |
| - HuggingFaceFW/fineweb-2 |
| - allenai/real-toxicity-prompts |
| - ToxicityPrompts/PolygloToxicityPrompts |
| - HPLT/HPLT2.0_cleaned |
| metrics: |
| - roc_auc |
| - f1 |
| tags: |
| - abacusai |
| - toxicity |
| - content-moderation |
| - trust-and-safety |
| - perspective-api |
| - hate-speech |
| - multilingual |
| - multi-label-classification |
| - onnx |
| - transformers.js |
| --- |
| <div align="center"> |
| <picture> |
| <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/63128dd099791aa61d180c72/dJ6uR23m09M-YEafBpmea.png" width="20%" alt="Toxicity-Lens"> |
| </picture> |
| </div> |
| <hr> |
| <div align="center" style="line-height:1"> |
| <a href="https://abacus.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Abacus.AI-white?color=1783ff&logoColor=white"/></a> |
| <a href="https://huggingface.co/abacusai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Abacus.AI-ffc107?color=ffc107&logoColor=white"/></a> |
| </div> |
| <div align="center" style="line-height: 1;"> |
| <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Apache_2.0-f5de53?&color=f5de53"/></a> |
| </div> |
| |
| ## 1. Model Introduction |
|
|
| Toxicity-Lens is an open, multilingual toxicity classifier from Abacus.AI. It returns the seven attributes of Google's Perspective API (TOXICITY, SEVERE_TOXICITY, IDENTITY_ATTACK, INSULT, PROFANITY, THREAT, SEXUALLY_EXPLICIT) as calibrated probabilities, and it ships with a server that answers Perspective API requests in the same JSON format, so teams moving off the Perspective API before Google shuts it down on December 31, 2026 can keep their code and change one URL. |
| |
| - **More accurate across languages.** On Horizon-Labs' own published benchmark (their evaluation code, their rows), it ranks toxic comments in 15 languages better than the general-purpose open classifiers from Horizon-Labs and Detoxify: TextDetox ROC AUC 89.0, against 85.4 for Horizon-Labs multilingual-toxicity-base and 67.8 for Detoxify multilingual. It is also more accurate at the usual 0.5 threshold (F1 65.2 vs 43.3). |
| - **Fewer false alarms than LLM guards.** It ranks toxic comments better than Qwen3Guard-Gen-8B, a safety LLM 27 times its size (AUC 88.4 vs 84.6), and flags 8.1% of clean comments against 38.8%. At the 0.5 threshold Qwen3Guard catches more of the toxic ones (88% against 59%) because it flags so much more. |
| - **Closest to the Perspective API.** Trained in part on the Perspective API's own published scores, it orders texts the way Perspective does more closely than any open model we tested, in English and in 17 languages (section 3). Swapped in at Perspective's 0.8 threshold (the default in Coral's documentation), it flags 95% of what Perspective flags in English text it never trained on, against 18% for Horizon-Labs base, and 85% on multilingual web text. It scores toxic text more decisively than Perspective, so at 0.8 it flags 1.2 times as many English texts and 1.9 times as many multilingual ones; section 6 gives the threshold that keeps your flag volume. |
| - **Runs anywhere, stays reproducible.** 22 ms per comment on 4 CPU threads with the 8-bit ONNX export, which agrees with float32 on 99.9% of decisions at 0.5; a CPU Docker image; about 2,200 comments per second on one H100; Apache 2.0, and every score is reproducible by pinning a commit. |
| |
| This card describes the model, its evaluation against the Perspective API's own scores and the open alternatives, how to move from the Perspective API, and the known limitations. Training data are listed in section 7; training rows are not released. |
| |
| ## 2. Model Summary |
| |
| <div align="center"> |
| <table> |
| <tbody> |
| <tr><td align="center"><strong>Task</strong></td><td align="center">Seven independent probabilities named after the Perspective API attributes</td></tr> |
| <tr><td align="center"><strong>Architecture</strong></td><td align="center">ModernBERT encoder: 22 layers, hidden size 768, 12 attention heads, local attention (128 tokens) with global attention every third layer</td></tr> |
| <tr><td align="center"><strong>Parameters</strong></td><td align="center">308M (307,535,623)</td></tr> |
| <tr><td align="center"><strong>Languages</strong></td><td align="center">Trained on English, Civil Comments translated into 30+ languages, natively written Arabic, Amharic and Tatar, and web text in 17 languages plus English prompts that carry the Perspective API's own scores; distilled on its teacher's scores for 374,000 natively written Hebrew, Hinglish, Tatar and Ukrainian web segments; evaluated in 21 languages plus Hinglish</td></tr> |
| <tr><td align="center"><strong>Input scored</strong></td><td align="center">256-token windows; the bundled server scores longer texts in overlapping windows</td></tr> |
| <tr><td align="center"><strong>Calibration</strong></td><td align="center">Per-attribute temperature, folded into the classification head</td></tr> |
| <tr><td align="center"><strong>Formats</strong></td><td align="center">safetensors (float32); ONNX float32 (exact), 8-bit (CPU default, near-exact) and dynamic int8 (transformers.js)</td></tr> |
| <tr><td align="center"><strong>Speed</strong></td><td align="center">22 ms per comment on 4 CPU threads (8-bit ONNX, Apple M5); about 2,200 comments per second on one H100 (bfloat16)</td></tr> |
| <tr><td align="center"><strong>Base model</strong></td><td align="center"><code>jhu-clsp/mmBERT-base</code></td></tr> |
| <tr><td align="center"><strong>Adaptation</strong></td><td align="center">Full finetune, distilled from an XLM-R-large teacher (560M) trained on the same data</td></tr> |
| <tr><td align="center"><strong>License</strong></td><td align="center">Apache 2.0</td></tr> |
| </tbody> |
| </table> |
| </div> |
| |
| ## 3. Evaluation |
| |
| ### Toxic comments in 15 languages |
| |
| *ROC AUC on TextDetox, averaged over 15 languages, on the rows of Horizon-Labs' published benchmark. The other models show the values published on their model cards; rerunning Horizon-Labs' evaluation code reproduces each of them to within 0.2 points.* |
| |
| <div align="center"> |
| <img src="figures/hero_textdetox.png" width="100%" alt="TextDetox ROC AUC over 15 languages: Toxicity-Lens vs Horizon-Labs and Detoxify"> |
| </div> |
| |
| ### Against open toxicity classifiers |
| |
| *Horizon-Labs' published toxicity benchmark: their evaluation code and row sampling (a 20,000-comment sample of the Civil Comments test split; TextDetox with up to 500 toxic and 500 non-toxic comments per language). Higher is better; x100.* |
| |
| <div style="max-width:1100px;margin:0 auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:30%"></th><th style="padding:10px 4px;text-align:center;font-weight:600;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;background:rgba(42,120,214,0.10);">Toxicity-Lens<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon base<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon small<div style="font-size:10px;font-weight:400;color:#6B6B6B">141M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify multilingual<div style="font-size:10px;font-weight:400;color:#6B6B6B">278M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify unbiased<div style="font-size:10px;font-weight:400;color:#6B6B6B">125M</div></th></tr></thead> |
| <tbody> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Civil Comments: mean ROC AUC over 6 attributes</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>98.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">98.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">98.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">96.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">98.5</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Civil Comments: TOXICITY ROC AUC</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>97.5</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.35)">97.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">97.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">96.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.29)">96.9</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">TextDetox: ROC AUC, 15 languages</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>89.0</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.42)">85.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.39)">83.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.18)">67.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">58.8</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">TextDetox: F1 at the 0.5 threshold, 15 languages</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>65.2</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.31)">43.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.39)">54.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.25)">35.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">8.6</td></tr> |
| </tbody></table> |
| <div style="font-size:11.5px;color:#6B6B6B;margin-top:6px">Bold = best in the row; shading is relative within each row. Toxicity-Lens: our runs of Horizon-Labs' code. Other models: the values on the Horizon-Labs model cards (<a href="https://huggingface.co/Horizon-Labs/multilingual-toxicity-base">base</a>, <a href="https://huggingface.co/Horizon-Labs/multilingual-toxicity-small">small</a>), which our rerun reproduces to within 0.002. TextDetox's own classifiers are left out because they were trained on TextDetox (they score 98.7 and 96.0 on it, and 89.5 and 84.5 on Civil Comments). At 0.5 on all 71,374 TextDetox rows, Toxicity-Lens flags 58.2% of toxic comments and 7.2% of clean ones (Horizon base: 33.2% and 2.3%).</div> |
| </div> |
| |
| ### Against LLM safety guards |
| |
| *Horizon-Labs' guard benchmark: TextDetox in 14 languages, 200 toxic and 200 non-toxic comments each (5,600 texts), pooled; a text is flagged when its score is above 0.5. Qwen3Guard is scored by the probability of an Unsafe or Controversial verdict.* |
| |
| <div style="max-width:1100px;margin:0 auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:30%"></th><th style="padding:10px 4px;text-align:center;font-weight:600;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;background:rgba(42,120,214,0.10);">Toxicity-Lens<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Qwen3Guard-Gen-8B<div style="font-size:10px;font-weight:400;color:#6B6B6B">8.2B</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Qwen3Guard-Gen-0.6B<div style="font-size:10px;font-weight:400;color:#6B6B6B">0.6B</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon guard-base<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Vela Shield<div style="font-size:10px;font-weight:400;color:#6B6B6B">307M</div></th></tr></thead> |
| <tbody> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">ROC AUC</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>88.4</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.33)">84.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">76.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.30)">83.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.18)">80.4</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">F1 at 0.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.19)">70.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>77.5</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.26)">72.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">66.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.26)">72.0</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Clean comments flagged, % (lower is better)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>8.1</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.13)">38.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">46.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.43)">11.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td></tr> |
| </tbody></table> |
| <div style="font-size:11.5px;color:#6B6B6B;margin-top:6px">Our runs of Horizon-Labs' guard code for every model except Vela Shield, whose numbers come from the <a href="https://huggingface.co/Horizon-Labs/content-safety-guard-base">Horizon-Labs guard card</a>; our runs reproduce the AUCs that card publishes for Qwen3Guard and Horizon's guard. LLM guards are built to judge chatbot prompts and responses, so toxic comments are outside their intended use; the table answers the question "why not just use an LLM guard?". On a half-toxic set, flagging everything already scores F1 66.7, which is why ROC AUC and the clean-comment flag rate are the fairer measures.</div> |
| </div> |
| |
| ### Agreement with the Perspective API |
| |
| *RealToxicityPrompts, held-out quarter: 49,810 English prompts and continuations with Perspective's own TOXICITY scores (2020) that Toxicity-Lens never trained on. Each model replaces Perspective at the same threshold t; the line shows, of the texts Perspective scores at t or above, the share the model also scores at t or above. Toxicity-Lens keeps 88 to 95% at every threshold, while also flagging 3.4 to 4.8% of the texts Perspective scores below t (Horizon-Labs base: at most 0.6%).* |
| |
| <div align="center"> |
| <img src="figures/perspective_thresholds.png" width="100%" alt="Share of the Perspective API's flags kept at the same threshold, English"> |
| </div> |
| |
| <div style="max-width:1100px;margin:0 auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:30%"></th><th style="padding:10px 4px;text-align:center;font-weight:600;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;background:rgba(42,120,214,0.10);">Toxicity-Lens<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon base<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon small<div style="font-size:10px;font-weight:400;color:#6B6B6B">141M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify multilingual<div style="font-size:10px;font-weight:400;color:#6B6B6B">278M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify unbiased<div style="font-size:10px;font-weight:400;color:#6B6B6B">125M</div></th></tr></thead> |
| <tbody> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Rank agreement with Perspective's scores (Spearman)</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English, RealToxicityPrompts (held-out): TOXICITY</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>93.3</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.26)">87.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.23)">86.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">81.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.15)">84.7</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English, RealToxicityPrompts (held-out): mean of the 7 attributes</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>85.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.22)">76.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.19)">75.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">69.5</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages, PolygloToxicityPrompts: TOXICITY, mean over languages</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>95.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.40)">85.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">83.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.21)">62.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">40.4</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages: one ranking over all 85,000 texts</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>94.1</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.40)">81.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">79.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.26)">59.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">23.9</td></tr> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Swapped in at the same threshold: Perspective's flags kept</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English at 0.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>94.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">17.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages at 0.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>86.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.33)">62.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.33)">62.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.21)">42.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">15.9</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages at 0.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>85.0</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.06)">14.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.07)">15.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.17)">33.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">12.4</td></tr> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Swapped in at the same threshold: extra flags (lower is better)</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English at 0.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.05)">3.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>0.0</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;color:#9aa0a6">n/a</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages at 0.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.43)">2.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.28)">3.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.11)">4.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">5.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>2.3</strong></td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages at 0.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.07)">12.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>2.4</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>2.4</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">12.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">4.7</td></tr> |
| </tbody></table> |
| <div style="font-size:11.5px;color:#6B6B6B;margin-top:6px">x100. Our runs on identical rows for every model. Perspective's scores come from the published datasets: <a href="https://huggingface.co/datasets/allenai/real-toxicity-prompts">RealToxicityPrompts</a> (scored by Perspective in 2020) and <a href="https://huggingface.co/datasets/ToxicityPrompts/PolygloToxicityPrompts">PolygloToxicityPrompts</a> (ptp-small, 5,000 web texts per language, scored in 2024). Toxicity-Lens trained on three quarters of the RealToxicityPrompts prompts and on PolygloToxicityPrompts texts outside ptp-small, with Perspective's scores: RealToxicityPrompts rows here are the held-out quarter (Horizon small and Detoxify: the full set, which they never trained on; where we have per-row runs the two agree to within 0.001), and every ptp-small text was removed from training. Extra flags = share of the texts Perspective scored below the threshold that the model flags. Detoxify multilingual has a TOXICITY output only; n/a = no such output, or no per-row run on that set.</div> |
| </div> |
| |
| ### Natively written comments the released model never trained on |
| |
| *ROC AUC x100 on each dataset's main label, on datasets written natively in each language (not translated) that the released model did not train on. Earlier experimental versions trained on GAHD, COLD, HED-COLD, SWSR and LLM-jp and were rejected partly on these sets, so they informed data choices.* |
| |
| <div style="max-width:1100px;margin:0 auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:30%"></th><th style="padding:10px 4px;text-align:center;font-weight:600;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;background:rgba(42,120,214,0.10);">Toxicity-Lens<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon base<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon small<div style="font-size:10px;font-weight:400;color:#6B6B6B">141M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify multilingual<div style="font-size:10px;font-weight:400;color:#6B6B6B">278M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify unbiased<div style="font-size:10px;font-weight:400;color:#6B6B6B">125M</div></th></tr></thead> |
| <tbody> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Not in the released model's training data</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">GAHD (German, identity attacks)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>81.2</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.42)">78.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">77.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.15)">62.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">56.6</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">COLD (Chinese, offensive language)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>81.7</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.46)">80.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.44)">79.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">50.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">50.7</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">HED-COLD (Chinese)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>80.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)">80.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">79.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">49.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.06)">49.5</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">SWSR (Chinese, sexism)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>81.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">80.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.44)">79.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">51.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.14)">57.7</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">LLM-jp toxicity (Japanese)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>91.3</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">85.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.35)">80.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.16)">62.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">52.9</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">TextDetox Tatar, 4,000 rows never trained on (a release gate)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>86.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">81.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.39)">79.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.20)">61.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">47.2</td></tr> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Test splits of datasets used in training</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">AfriHate (Amharic, Algerian and Moroccan Arabic)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>88.8</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.27)">73.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.18)">67.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">57.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.12)">62.9</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">MLMA (Arabic)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>78.3</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.28)">64.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.29)">64.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">46.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.10)">50.7</td></tr> |
| <tr><td colspan="6" style="padding:7px 10px;font-weight:600;color:#2a78d6;background:rgba(42,120,214,0.08);border-bottom:1px solid rgba(42,120,214,0.2)">Identity-group bias</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">CivilComments-WILDS: Jigsaw bias score (higher = less biased)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.39)">93.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.27)">92.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.31)">92.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">91.2</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.47)"><strong>93.9</strong></td></tr> |
| </tbody></table> |
| <div style="font-size:11.5px;color:#6B6B6B;margin-top:6px">Our runs on identical rows; Detoxify multilingual's TOXICITY output is used for every set. Single-purpose models score higher on single sets: Horizon-Labs' hate-speech model on GAHD (86.3) and SWSR (83.0), and its content-safety guard on LLM-jp (95.4). Sources: GAHD (Goldzycher et al., 2024), COLD (Deng et al., 2022), HED-COLD, SWSR (Jiang et al., 2022), LLM-jp toxicity dataset v2, TextDetox, AfriHate (Muhammad et al., 2025), MLMA (Ousidhoum et al., 2019), CivilComments-WILDS (Koh et al., 2021).</div> |
| </div> |
| |
| ### TextDetox, per language |
| |
| *ROC AUC x100 on Horizon-Labs' published TextDetox rows. Toxicity-Lens leads Horizon-Labs base in 15 of 15 languages.* |
| |
| <div style="max-width:1100px;margin:0 auto"> |
| <table style="width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:28%"></th><th style="padding:10px 4px;text-align:center;font-weight:600;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;background:rgba(42,120,214,0.10);">Toxicity-Lens<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon base<div style="font-size:10px;font-weight:400;color:#6B6B6B">308M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Horizon small<div style="font-size:10px;font-weight:400;color:#6B6B6B">141M</div></th><th style="padding:10px 4px;text-align:center;font-weight:500;border-bottom:2px solid #2a78d6;color:#2a78d6;font-size:13px;">Detoxify multilingual<div style="font-size:10px;font-weight:400;color:#6B6B6B">278M</div></th></tr></thead> |
| <tbody> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Amharic</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>84.4</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.29)">71.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.17)">63.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">54.8</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Arabic</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>81.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.44)">78.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.37)">72.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">41.4</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">German</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>87.0</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">85.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">85.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">66.2</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>99.6</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.26)">99.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">99.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.19)">99.2</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Spanish</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>92.2</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">90.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">89.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">81.9</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">French</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>99.5</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.23)">98.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.14)">98.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">98.1</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Hebrew</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>84.0</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">78.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.39)">77.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">50.3</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Hindi</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>92.4</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.45)">91.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.43)">89.7</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">64.5</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Hinglish (romanised Hindi)</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>78.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">76.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.36)">74.9</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">63.0</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Italian</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>88.1</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.43)">87.5</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.20)">84.0</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">81.8</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Japanese</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>89.7</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.42)">84.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.40)">83.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">50.6</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Russian</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>99.7</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.42)">97.3</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.38)">95.6</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">80.7</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Tatar</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>85.2</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.41)">81.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.39)">80.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">62.6</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Ukrainian</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>91.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.27)">80.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.27)">80.8</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">68.1</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">Chinese</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;font-weight:600;background:rgba(42,120,214,0.47)"><strong>79.9</strong></td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.43)">77.4</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.42)">77.1</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2;background:rgba(42,120,214,0.05)">54.7</td></tr> |
| </tbody></table> |
| <div style="font-size:11.5px;color:#6B6B6B;margin-top:6px">Horizon-Labs: per-language values published on their cards. Detoxify multilingual: our rerun of Horizon-Labs' code.</div> |
| </div> |
| |
| <details> |
| <summary><b>Notes on the evaluation</b></summary> |
| |
| Training texts that also appear in an evaluation set were removed: exact matches after normalising case and whitespace, plus near-duplicates for the natively written training sets. Two evaluation sources also supplied training text with the Perspective API's scores: RealToxicityPrompts, where the model trained on three quarters of the prompts (with their continuations) and every number here uses the remaining quarter, split by a hash of the prompt index; and PolygloToxicityPrompts, where it trained on ptp-full texts and every ptp-small (evaluation) text was removed. The TextDetox Tatar row uses 4,000 Tatar rows that were never trained on and were kept out of the Tatar label check; they served as a release gate. |
| |
| Horizon-Labs publish their evaluation code with their models. We copied their toxicity evaluation with two label aliases added so it reads our attribute names, and reduced their guard evaluation to its TextDetox benchmark; our reruns reproduce every headline number we cite from their cards (mean ROC AUC and F1 per model, guard ROC AUC and F1) to within 0.002, so the published values and ours compare directly. On all 71,374 TextDetox rows instead of Horizon-Labs' sample the order is the same: Toxicity-Lens 89.9, Horizon base 86.1, Horizon small 84.5, Detoxify multilingual 68.8, Detoxify unbiased 58.4. |
| |
| PolygloToxicityPrompts mixes languages inside some splits (its Polish split holds 952 Japanese pages), so read single languages with care. RealToxicityPrompts carries Perspective's 2020 scores and PolygloToxicityPrompts its 2024 scores; Perspective changed its models in between. Detoxify multilingual and unbiased are the Hugging Face checkpoints unitary/multilingual-toxic-xlm-roberta and unitary/unbiased-toxic-roberta (Unitary notes they currently give different results from the detoxify library); Detoxify multilingual was trained on seven languages and Detoxify unbiased on English only, so most rows here are outside their training languages. |
| |
| </details> |
| |
| ## 4. Training Approach |
| |
| **What the scores mean.** Each output is the probability that a typical rater would apply the label, following the Perspective API's attribute definitions and the Civil Comments annotation guidelines: |
| |
| - TOXICITY: rude, disrespectful or unreasonable language likely to make someone leave a discussion. |
| - SEVERE_TOXICITY: very hateful, aggressive or disrespectful language; much less sensitive to mild profanity than TOXICITY. |
| - IDENTITY_ATTACK: negative or hateful language aimed at someone because of an identity such as religion, ethnicity, nationality, gender, sexual orientation or disability. |
| - INSULT: insulting, inflammatory or demeaning language towards a person or group. |
| - PROFANITY: swear words, curse words and other obscene language. |
| - THREAT: an intention to inflict pain, injury or violence on a person or group. |
| - SEXUALLY_EXPLICIT: references to sexual acts, sexual body parts or other lewd content. |
|
|
| **Method.** Seven sigmoid outputs trained with binary cross-entropy on the fraction of human raters who chose each label, so a comment half the raters found toxic targets 0.5. Attributes a dataset does not label are left out of the loss rather than set to 0. Non-toxic comments that mention an identity group count double, to reduce false flags on identity mentions. Two epochs, maximum length 256 tokens, learning rate 5e-5, batch 64. Each target is the mean of the label and a teacher's score (an XLM-R-large model with 560M parameters, trained on the same data, the Perspective-scored texts below also counted twice); on 374,000 unlabelled native web segments in Hebrew, Hinglish, Tatar and Ukrainian (FineWeb-2, HPLT 2.0) the teacher's scores alone are the targets. |
|
|
| **The Perspective API's own scores.** About 530,000 training texts carry the Perspective API's published scores instead of rater fractions: web text in 17 languages from PolygloToxicityPrompts (ptp-full without the evaluation texts; scored in 2024) and three quarters of the RealToxicityPrompts prompts with their continuations (scored in 2020). They count twice in the loss. On German and Portuguese pages the target is the mean of Perspective's score and an earlier version's, because Perspective gives many harmless pages there 0.5 to 0.75 (section 5); PolygloToxicityPrompts has no SEXUALLY_EXPLICIT score, so that target comes from the earlier version too. |
| |
| **Calibration.** One temperature per attribute, fitted on held-out Civil Comments rater fractions (TOXICITY T = 0.885) and folded into the final layer, so the published weights need no post-processing. |
| |
| **Versions.** Scores change only when you change the model. Pin a commit of this repository (`revision=` in transformers, `MODEL_REVISION` for the server) and every score stays reproducible; later releases will go out as new commits with their evaluation in this card. |
|
|
| ## 5. Known Behaviors and Limitations |
|
|
| - **High thresholds flag more texts than Perspective.** This model scores toxic text more decisively than Perspective does: at 0.9 it flags 1.8 times as many held-out English texts as Perspective and 2.9 times as many multilingual web texts, whose 2024 Perspective scores rarely exceed 0.9. If you act automatically at 0.8 or 0.9, start from the volume-matched threshold in section 6, or measure it on your own traffic with `compare_with_perspective.py`. |
| - **German and Portuguese web pages: Perspective is only partly reproduced, on purpose.** Perspective gives many harmless German and Portuguese web pages 0.5 to 0.75 in PolygloToxicityPrompts (a health-forum question about a pill, a parenting forum, a paintbrush product page). Training on those scores alone copied the habit, so for these two languages the training target blends Perspective's score with an earlier version's. This model flags 47.5% (German) and 50.0% (Portuguese) of Perspective's flags there; in blind reads of the German and Portuguese pages that it flags and the earlier version does not, 37 to 57 of every 100 were harmless pages that Perspective also flags. If a false flag on a German or Portuguese web page costs you more than a missed one, use a higher threshold for them. |
| - **Long texts.** The model reads 256 tokens at a time. The bundled server scores longer texts in overlapping windows and reports the highest window score per attribute; the transformers pipeline with `truncation=True` scores only the first 256 tokens. |
| - **Only the seven production attributes.** Perspective's experimental attributes (such as FLIRTATION) and its bridging attributes (such as REASONING or CURIOSITY) are not provided, and there is no language detection. |
| - **Automated removal.** The AfriHate authors ask that models built with their data not be used for automated removal. We pass that request on: use the scores to prioritise human review, not to remove content without a person in the loop. |
| - Like every toxicity model, it can over-flag identity mentions and reclaimed or quoted language and miss sarcasm or implicit abuse; labels are subjective and culture-dependent. Do not use it as the only basis for decisions about people. |
|
|
| ## 6. Deployment |
|
|
| ### Quickstart |
|
|
| Try it in the browser first: the [demo Space](https://huggingface.co/spaces/abacusai/Toxicity-Lens) scores text in any language, answers requests in the Perspective API's format and works as an MCP tool for agents. |
|
|
| ```python |
| from transformers import pipeline |
| |
| clf = pipeline("text-classification", model="abacusai/Toxicity-Lens", top_k=None) |
| print(clf("You are a complete idiot and nobody wants you here.", truncation=True)) |
| # [[{'label': 'TOXICITY', 'score': 0.95}, {'label': 'INSULT', 'score': 0.95}, ...]] |
| ``` |
|
|
| The config marks the task as multi-label, so the pipeline applies a sigmoid per attribute; `truncation=True` scores the first 256 tokens. Tested with transformers 4.57 and 5.14. |
|
|
| ### Moving off the Perspective API |
|
|
| **1. Run the server.** `server.py` in this repository answers the Perspective API's requests. On a GPU or CPU with PyTorch: |
|
|
| ```bash |
| pip install fastapi uvicorn tokenizers huggingface_hub transformers torch |
| hf download abacusai/Toxicity-Lens server.py --local-dir . |
| MODEL_DIR=abacusai/Toxicity-Lens uvicorn server:app --host 0.0.0.0 --port 8080 |
| ``` |
|
|
| Or on any CPU, without PyTorch, with the 8-bit ONNX export in Docker: |
|
|
| ```bash |
| hf download abacusai/Toxicity-Lens --local-dir toxicity-lens \ |
| --include "*.json" --include server.py --include Dockerfile --include .dockerignore --include onnx/model_int8_block32.onnx |
| docker build -t toxicity-lens toxicity-lens && docker run -p 8080:8080 toxicity-lens |
| ``` |
|
|
| **2. Point your client at it.** The request and response JSON are the same as the Perspective API's, so only the address changes. |
|
|
| ```bash |
| # was: https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=YOUR_KEY |
| curl -s localhost:8080/v1alpha1/comments:analyze -H 'Content-Type: application/json' \ |
| -d '{"comment": {"text": "You are an idiot"}, "requestedAttributes": {"TOXICITY": {}, "INSULT": {}}}' |
| ``` |
|
|
| Code that uses Google's client library, as in the Perspective API documentation, changes only the discovery URL: |
|
|
| ```python |
| from googleapiclient import discovery |
| |
| client = discovery.build( |
| "commentanalyzer", "v1alpha1", developerKey="unused", |
| # was https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1 |
| discoveryServiceUrl="http://localhost:8080/$discovery/rest?version=v1alpha1", |
| static_discovery=False) |
| body = {"comment": {"text": "You are an idiot"}, "requestedAttributes": {"TOXICITY": {}}} |
| resp = client.comments().analyze(body=body).execute() |
| print(resp["attributeScores"]["TOXICITY"]["summaryScore"]["value"]) |
| ``` |
|
|
| Coral: in its Perspective settings, set the endpoint to `http://<your-host>:8080/v1alpha1` (the default is `https://commentanalyzer.googleapis.com/v1alpha1`). Coral's `doNotStore` and feedback (`comments:suggestscore`) calls are accepted. |
|
|
| **3. Check your thresholds.** Toxicity-Lens uses Perspective's 0-to-1 scale, and most of Perspective's flags carry over at the same threshold. Because it scores toxic text more decisively, it flags more texts than Perspective at high thresholds, most on multilingual web text; the rows "threshold that flags as many" give the threshold that keeps your flag volume. Swapped in at your threshold on Perspective's own scores (English: the held-out quarter of RealToxicityPrompts): |
|
|
| <div style="max-width:900px;margin:0 auto"> |
| <table style="width:100%;border-collapse:collapse;font-size:13px"> |
| <thead><tr><th style="padding:8px 6px;text-align:left;border-bottom:2px solid #2a78d6;width:52%">Your Perspective threshold</th><th style="padding:8px 4px;text-align:center;border-bottom:2px solid #2a78d6;color:#2a78d6;font-weight:600">0.5</th><th style="padding:8px 4px;text-align:center;border-bottom:2px solid #2a78d6;color:#2a78d6;font-weight:600">0.7</th><th style="padding:8px 4px;text-align:center;border-bottom:2px solid #2a78d6;color:#2a78d6;font-weight:600">0.8</th><th style="padding:8px 4px;text-align:center;border-bottom:2px solid #2a78d6;color:#2a78d6;font-weight:600">0.9</th></tr></thead><tbody> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English (RealToxicityPrompts): Perspective's flags kept</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">88%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">92%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">95%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">94%</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English: extra flags</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">3.7%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">3.4%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">3.8%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">4.8%</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">English: threshold that flags as many comments as Perspective</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.48</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.74</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.86</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.94</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages (PolygloToxicityPrompts): Perspective's flags kept</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">87%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">91%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">85%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">80%</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages: extra flags</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">2.6%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">7.9%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">12.4%</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">2.8%</td></tr> |
| <tr><td style="padding:8px 6px;padding-left:16px;border-bottom:1px solid rgba(128,128,128,0.15);font-size:14px;font-weight:600">17 languages: threshold that flags as many texts as Perspective</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.40</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.74</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.85</td><td style="padding:8px 6px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:14px;line-height:1.2">0.93</td></tr> |
| </tbody></table> |
| </div> |
|
|
| To get these numbers for your own traffic, run `compare_with_perspective.py` (in this repository) on comments you already scored with Perspective: it prints, per attribute, the rank agreement, the flags kept and added at your threshold, and the threshold that keeps your flag volume. |
|
|
| ```bash |
| python compare_with_perspective.py logged_scores.csv --threshold 0.8 # columns: text, TOXICITY, INSULT, ... |
| ``` |
|
|
| **4. What stays the same, and what differs.** |
|
|
| - Same: `POST /v1alpha1/comments:analyze` with `comment.text`, `requestedAttributes` (with `scoreThreshold`), `spanAnnotations` (per-sentence `spanScores`), `languages`, `clientToken` and `doNotStore`; the seven attribute names; scores as `{"value": p, "type": "PROBABILITY"}`; the 20,480-byte limit; Google-style error bodies; `comments:suggestscore` (accepted, and stored only if you set `FEEDBACK_FILE`); and the discovery document that Google's client libraries read. |
| - Different: other attributes (experimental, bridging) return an error; there is no language detection, so any language is accepted and `languages` echoes your request; there are no quotas or rate limits; API keys are optional (`API_KEYS=key1,key2`); scores are close to Perspective's but not identical (section 3); and your text never leaves your servers. |
|
|
| ### More ways to run it |
|
|
| - **OpenAI-style moderation.** `POST /v1/moderations` with `{"input": ["..."]}` returns OpenAI's moderation shape, filling the categories these attributes support honestly (harassment, harassment/threatening, hate, sexual, violence, plus toxicity), for gateways that expect it. |
| - **LiteLLM guardrail.** The server implements LiteLLM's Generic Guardrail API: |
|
|
| ```yaml |
| litellm_settings: |
| guardrails: |
| - guardrail_name: toxicity-lens |
| litellm_params: |
| guardrail: generic_guardrail_api |
| mode: pre_call # or post_call |
| api_base: http://localhost:8080 |
| additional_provider_specific_params: |
| threshold: 0.8 # BLOCKED when any attribute reaches it |
| ``` |
|
|
| - **Browser and Node.js (transformers.js).** The int8 export runs with transformers.js 3 (tested with 3.8.1 on Node.js): |
|
|
| ```js |
| import { pipeline } from "@huggingface/transformers"; |
| |
| const clf = await pipeline("text-classification", "abacusai/Toxicity-Lens", { dtype: "int8" }); |
| console.log(await clf("You are a complete idiot and nobody wants you here.", { top_k: 7 })); |
| ``` |
|
|
| transformers.js runs `onnx/model_int8.onnx` (a 309 MB download), plain dynamic int8 (its bundled ONNX Runtime 1.21 cannot load the 8-bit file the server uses), so its scores are close to the server's but not identical: it moves scores by 0.014 on average (99th percentile 0.09) and flips 1.4% of decisions at 0.5, and it tokenizes runs of two or more spaces differently (float32 scores up to 0.02 apart on such texts). Use the server or Python where exact scores matter. |
|
|
| - **ONNX.** `onnx/model.onnx` is float32 and matches PyTorch to 1.8e-07. `onnx/model_int8_block32.onnx` (322 MB; 8-bit block-wise weights with int8 compute; the default of the server and the Docker image) moves scores by 0.0008 on average (maximum 0.013) and changes 0.08% of decisions at 0.5 on 2,200 multilingual texts with ONNX Runtime's standard CPU kernels. With ONNX Runtime's generic CPU kernels (x86, Graviton, Apple M1 to M3) it is about 2.1 times as fast as float32 (23 vs 49 ms per comment on 4 threads of an Apple M5); on Apple M4 or newer, float32 is at least as fast (19 ms) and exact. It needs an ONNX Runtime with 8-bit MatMulNBits (tested with 1.31). `onnx/model_int8.onnx` (309 MB) is plain dynamic int8 for transformers.js and older runtimes. It quantizes activations per tensor, so on most CPUs and in browsers it moves scores by 0.014 on average (99th percentile 0.09) and flips 1.4% of decisions at 0.5; only on Apple M4 or newer, where ONNX Runtime's KleidiAI kernels quantize per row, does it stay much closer to float32. The server runs any of them with `BACKEND=onnx` (`ONNX_FILE` picks the file). |
|
|
| ## 7. License |
|
|
| Apache 2.0; see LICENSE, and NOTICE for the third-party notices. Training data and licences: |
|
|
| - Civil Comments (Jigsaw; Borkan et al., 2019), [google/civil_comments](https://huggingface.co/datasets/google/civil_comments), CC0 1.0. |
| - [RealToxicityPrompts](https://huggingface.co/datasets/allenai/real-toxicity-prompts) (Gehman et al., 2020), English prompts and continuations with the Perspective API's 2020 scores, Apache 2.0; three quarters of the prompts were used for training. |
| - [PolygloToxicityPrompts](https://huggingface.co/datasets/ToxicityPrompts/PolygloToxicityPrompts) (Jain et al., 2024), web text in 17 languages with the Perspective API's 2024 scores; its card lists CC BY 4.0 in its metadata and the AI2 ImpACT License (Low Risk Artifacts) in its text. Texts outside the ptp-small evaluation split were used for training. |
| - [Horizon-Labs/civil-comments-multilingual](https://huggingface.co/datasets/Horizon-Labs/civil-comments-multilingual), Civil Comments translated by Qwen3.8-27B, CC0 1.0. |
| - Further Civil Comments translated by us into Hinglish, Chinese, Arabic, Japanese, German and Italian with [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) (Apache 2.0). |
| - [AfriHate](https://huggingface.co/datasets/afrihate/afrihate) (Muhammad et al., NAACL 2025), Amharic and Arabic; Apache 2.0 on Hugging Face, CC BY on [GitHub](https://github.com/AfriHate/AfriHate); labels remapped. The authors ask that models built with it not be used for automated removal. |
| - [MLMA](https://github.com/HKUST-KnowComp/MLMA_hate_speech) (Ousidhoum et al., 2019), Arabic, MIT, Copyright (c) 2019 HKUST-KnowComp; labels remapped. |
| - Tatar web text labelled by us with Qwen3.8-27B-FP8. Contains information from [FineWeb-2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2), which is made available under the [ODC Attribution License](https://opendatacommons.org/licenses/by/1-0/), and from [HPLT 2.0](https://huggingface.co/datasets/HPLT/HPLT2.0_cleaned) (CC0 packaging). |
| - Unlabelled Hebrew, Hinglish, Tatar and Ukrainian text from the same corpora, scored by the teacher model (section 4). |
| - Base model [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base), MIT; Gemma 2 tokenizer ([Gemma Terms of Use](https://ai.google.dev/gemma/terms)). |
|
|
| Perspective is a trademark of Google LLC. This project is not affiliated with or endorsed by Google or Jigsaw. |
|
|
| ## 8. Citation |
|
|
| ```bibtex |
| @misc{abacusai2026toxicitylens, |
| title = {Toxicity-Lens: open multilingual toxicity scoring compatible with the Perspective API}, |
| author = {Abacus.AI}, |
| year = {2026}, |
| url = {https://huggingface.co/abacusai/Toxicity-Lens} |
| } |
| ``` |
|
|
| The base model: |
|
|
| ```bibtex |
| @article{marone2025mmbert, |
| title = {mmBERT: A Modern Multilingual Encoder with Annealed Language Learning}, |
| author = {Marone, Marc and Weller, Orion and Fleshman, William and Yang, Eugene and Lawrie, Dawn and Van Durme, Benjamin}, |
| journal = {arXiv preprint arXiv:2509.06888}, |
| year = {2025} |
| } |
| ``` |
|
|
| ## 9. Contact Us |
|
|
| If you have any questions, please reach out at [Abacus.AI](https://abacus.ai). |
|
|
|
|
