toxic: on-device multilingual hate-speech and abuse triage

A multilingual toxicity classifier for on-device hate speech detection and content moderation triage: five content heads (HATEFUL, HARASSMENT, THREAT, SEXUAL, SELF_HARM) plus protected-group target heads, measured on eight languages (en, es, pt, fr, de, it, nl, pl). Four of them (English, Spanish, Portuguese, French) rank among the world's ten most spoken languages; together the eight are spoken by well over a billion people worldwide. The deployable model is ~66.7 MB (4-bit Core ML on Apple, 2.8 ms on an M3 Ultra Neural Engine), ~75.5 MB (int4 LiteRT .tflite on Android and Linux), or ~83.3 MB (int4 ONNX toxic.onnx in the browser, bit-exact under ONNX Runtime Web). Inference is 100% on-device.

At its size, this is the strongest measured system we know of for hate speech detection in these eight languages, and it is competitive with models an order of magnitude larger. Every runnable competitor was re-run through the identical benchmark harness. On the seven non-English languages, toxic beats every one of them, including the full-precision server builds of guard models 6 to 14 times its size, by 13 to 30 points of average macro-F1. On English, the 134.9 MB int8 build outscores every competitor's shipping 4-bit quant (0.808 vs 0.794 for Llama-Guard-3-1B int4 at 955 MB); the 75.5 MB int4 build shipped here gives up 0.007 on English to that 12.7x-larger model and nothing anywhere else. Full tables, including the cases competitors win, are in How it compares.

Throughout this card, quality numbers are macro-F1 on HateCheck and its multilingual extension: a public suite of roughly 3,700 hand-written test cases per language, never trained on, built to trap classifiers with hard hate (slurs, spelling evasion, implied hate) and deliberately confusing non-hate (quotes, counter-speech, negation). Macro-F1 averages the score for catching hate with the score for letting clean text through, so a model cannot look good by simply flagging everything (or nothing); 1.000 is perfect.

Triage, not verdict. Outputs are escalation signals for human review or a heavier local tier, not autonomous removal decisions. SELF_HARM and THREAT are high-precision heads intended for resource routing with a human in the loop. The failure modes below are published on purpose: read them before wiring the model into any enforcement path.

Files

File	Format	Size	Contents
toxic.tflite	LiteRT / TFLite (int4 blockwise-32)	~75.5 MB	Dynamic-range int4 (block-32 groups) + int8, fixed [1, 128] window; runs on Android and Linux with native LiteRT kernels
toxic.onnx	ONNX (int4 MatMuls + int4 embedding, int8 rest)	~83.3 MB	The browser artifact for ONNX Runtime Web 1.27 (wasm EP); fixed [1, 128] window, int64 inputs; the browser reproduces its native scores bit-for-float
toxic.mlmodelc	Compiled Core ML (4-bit)	~66.7 MB	Palettized (grouped-channel encoder + per-tensor embedding), fp16 compute, ANE-resident; ready to load on Apple platforms
toxic.mlpackage	Core ML package (4-bit)	~66.7 MB	Same model before coremlcompiler; recompile or inspect
toxic.pt	PyTorch state dict	~531 MB	Full-precision reference weights (encoder + both heads)
config.json	JSON	tiny	Encoder + head config (base, labels, max_len, vocab size)
tokenizer.json, tokenizer_config.json	JSON	~2.4 MB	Trimmed to the eight languages (60,728-piece) SentencePiece-Unigram tokenizer (XLM-R lineage)
toxic_tokenizer.bin	Binary	~0.7 MB	Compact RDTK vocab (scores + pieces) for the portable SDK tokenizers
labels.json	JSON	tiny	id2label / label2id for both heads
toxic_meta.json	JSON	tiny	Version, languages, labels, default threshold and the per-language threshold menu

Taxonomy

Two multi-label heads over one shared encoder. Text can fire several labels at once; thresholds apply per label.

• Content heads (5): HATEFUL (scoped to the EU notion of illegal hate speech: public incitement to violence or hatred against a protected group, Framework Decision 2008/913/JHA), HARASSMENT (targeted insult / bullying of an individual), THREAT (threat of violence), SEXUAL (sexual harassment / unsolicited sexual aggression), SELF_HARM (self-harm / suicide content, for resource routing).
• Target heads (10): the 2008/913/JHA protected grounds RACE, COLOUR, RELIGION, DESCENT, NATIONAL_ETHNIC_ORIGIN, plus the extended grounds SEXUAL_ORIENTATION, GENDER, DISABILITY, AGE, OTHER. Only meaningful when HATEFUL fires; used for per-ground fairness reporting.

Architecture

• Encoder: XLM-R-base lineage (12 layers, hidden 768), vocabulary trimmed from 250k to 60,728 pieces (~133M params), fine-tuned with focal BCE on the two heads.
• Static shapes: fixed [1, 128] input, no dynamic control flow; ANE-native reformulation for the Core ML export, dynamic-range quantization for LiteRT.
• Runtime: tokenize off-graph (SentencePiece-Unigram, tokenizer.json or toxic_tokenizer.bin), run the encoder, apply sigmoid per label, threshold per the menu below. Default threshold 0.40.

Measured quality (the shipped artifacts, not the checkpoint)

Every number below is the full Multilingual HateCheck scorer re-run on the exact artifact in this repo, threshold 0.40. HateCheck is eval-only: it is never trained on. Torch reference: 8-language mean macro-F1 0.789.

platform	artifact	size	8-lang macro-F1	delta vs torch	latency (measured)
iOS / macOS (ANE)	toxic.mlmodelc	66.7 MB	0.776	-0.013	2.82 ms median (M3 Ultra ANE); 12.6 ms CPU-only
Android	toxic.tflite	75.5 MB	0.786	-0.003	8.1 ms median (8-thread desktop CPU, XNNPACK)
web (browser)	toxic.onnx	83.3 MB	0.775	-0.014	148 ms median (headless Chromium, 4-thread wasm, 8-core desktop); 22.6 ms native ORT CPU
reference	toxic.pt	531 MB	0.789	-	n/a

Phone-class latency (iPhone 13/15, mid-range Android) is not yet measured; the M3 Ultra figure bounds the graph cost.

The web row is the one platform where the deployed runtime was also verified numerically: under ONNX Runtime Web 1.27 (wasm EP) the browser reproduces the native toxic.onnx scores bit-for-float (max probability delta 1.1e-6 over a 120-fixture, 8-language drift study, zero decision flips), so the browser ships exactly the 0.775 measured here (0.774 on the seven non-English languages). toxic.tflite stays Android-native: its 0.786 was measured with native LiteRT kernels, which browser wasm kernels do not reproduce. The web SDKs pin release tag v0.1.1 (the first tag that carries toxic.onnx).

Per language (threshold 0.40):

lang	torch	toxic.tflite (int4-b32)	toxic.mlmodelc (int4)	toxic.onnx (int4e)
en	0.810	0.787	0.771	0.786
de	0.803	0.790	0.791	0.778
fr	0.824	0.816	0.798	0.805
nl	0.764	0.769	0.754	0.764
pt	0.791	0.800	0.785	0.792
es	0.796	0.799	0.786	0.784
it	0.775	0.776	0.771	0.750
pl	0.751	0.750	0.754	0.744
mean	0.789	0.786	0.776	0.775

How it compares (measured, same scorer, same suite)

Every runnable competitor was re-run through the identical HateCheck harness, in both its full-precision (bf16 / fp32) build and its measured int4 quant (bitsandbytes nf4), so quantized is compared against quantized and full against full. Generative guards are scored on their parsed safety verdict (Unsafe = hate); their on-disk sizes are published Q4_K_M GGUF bytes. 7-EU = mean over the seven non-English languages (de, fr, nl, pt, es, it, pl). Hate recall = share of hateful cases caught; non-hate FPR = share of non-hateful cases wrongly flagged.

system	build	on-disk size	7-EU macro-F1	en macro-F1	7-EU hate recall	7-EU non-hate FPR	7-EU F1 per 100 MB
toxic	LiteRT int4-b32 (shipped)	75.5 MB	0.786	0.787	0.880	0.314	1.04
toxic	Core ML 4-bit (shipped)	66.7 MB	0.777	0.771	0.881	0.337	1.16
toxic	LiteRT int8 (on request)	134.9 MB	0.786	0.808	0.872	0.300	0.58
toxic	torch reference	531 MB	0.786	0.810	0.870	0.296	0.15
Llama-Guard-3-1B	bf16 (server)	n/a	0.649	0.814	0.594	0.187	n/a
Llama-Guard-3-1B	int4	955.4 MB	0.633	0.794	0.598	0.240	0.07
Qwen3Guard-Gen-0.6B	bf16 (server)	n/a	0.652	0.770	0.738	0.414	n/a
Qwen3Guard-Gen-0.6B	int4	484.2 MB	0.622	0.750	0.662	0.374	0.13
Detoxify multilingual (278M)	fp32	~280 MB	0.487	0.642	0.476	0.303	0.17

Per language, macro-F1, each system in its listed build:

lang	toxic .tflite (75.5 MB)	toxic Core ML (66.7 MB)	LG-3-1B bf16	LG-3-1B int4 (955 MB)	Qwen3G-0.6B bf16	Qwen3G-0.6B int4 (484 MB)	Detoxify fp32 (~280 MB)
en	0.787	0.771	0.814	0.794	0.770	0.750	0.642
de	0.790	0.791	0.686	0.663	0.674	0.645	0.299
fr	0.816	0.798	0.654	0.635	0.691	0.645	0.646
nl	0.769	0.754	0.668	0.660	0.647	0.620	0.293
pt	0.800	0.785	0.684	0.662	0.671	0.648	0.650
es	0.799	0.786	0.685	0.666	0.692	0.675	0.643
it	0.776	0.771	0.597	0.588	0.642	0.625	0.633
pl	0.750	0.754	0.566	0.556	0.547	0.495	0.243
7-EU	0.786	0.777	0.649	0.633	0.652	0.622	0.487

What the table says:

• Seven of eight languages: toxic wins outright at 6 to 14x less disk, against full-precision and int4 competitor builds alike. The margin is never small: +10.4pp over the best competitor build on de, +12.5pp fr, +10.1pp nl, +11.6pp pt, +10.7pp es, +13.4pp it, +18.4pp pl (shipped .tflite vs each competitor's best build in any precision).
• English: Llama-Guard-3-1B leads in bf16 (0.814 vs toxic's 0.810 reference). At shipping quants the picture flips with size: toxic int8 (134.9 MB) 0.808 beats Llama-Guard int4 (955.4 MB) 0.794; the shipped 75.5 MB int4 lands 0.787, 0.007 behind at 7.9% of the bytes.
• Quality per megabyte is not close: 1.04 to 1.16 macro-F1 per 100 MB for the shipped toxic artifacts vs 0.07 (Llama-Guard int4) and 0.13 (Qwen3Guard int4). That gap is what makes always-on, per-message, on-device moderation feasible at all.
• Detoxify multilingual, the incumbent same-architecture-class classifier, collapses outside its strong languages (de 0.299, nl 0.293, pl 0.243) and trails toxic by 15 to 20 points even where it works.

False positives in context

Two very different numbers describe "how often it flags clean text", and conflating them misreads the model.

On adversarial trap cases: 27 to 36%. HateCheck's non-hate cases are not ordinary text; they are traps built from the exact patterns that break hate classifiers. counter_quote / counter_ref quote or denounce hate in order to condemn it, slur_reclaimed is in-group reclaimed slur use, negate_neg is negated hate ("no one should ever hurt them"). At the default 0.40 threshold, toxic wrongly flags 27.1% (pl) to 35.9% (de) of these traps, depending on language.

On natural text: 2 to 3%. On the held-out test split of real corpora (Civil Comments, Measuring Hate Speech, GAHD; 14,021 clean rows never seen in training), the HATEFUL head fires on 3.0% of clean rows overall. On 13,559 clean English user comments (Civil Comments) it is 2.3% for HATEFUL and 7.3% for any of the five heads (the extra fires are almost all HARASSMENT at 6.4%; THREAT 0.4%, SEXUAL 0.4%, SELF_HARM 0.0%). On clean Measuring Hate Speech rows: 3.4%. The one natural-corpus outlier is German GAHD at 38.8%, and that is by design: GAHD is an adversarially constructed dataset of contrast pairs (quotes, counter-speech, borderline rephrasings), so it measures the trap rate, not the organic rate. Full breakdown: reports/RELEASE_extended.md in the training repo.

Beware the low-FPR trap when comparing. Llama-Guard-3-1B posts the lowest trap FPR (0.187 on the seven non-English languages, bf16), but it does so by barely firing: it catches only 59.4% of actual non-English hate in those languages, missing two in five hateful messages. toxic catches 87 to 88% at its operating points. A low false-positive rate bought by under-firing is not precision, which is why the headline metric is macro-F1 (balanced by construction) and why FPR numbers should always be read next to recall. If your application needs a lower FPR, buy it explicitly with the threshold menu below and keep the recall you are giving up on the table.

Threshold menu (per-language operating points)

Measured on the torch reference (the vocabulary trim is quality-neutral). macro-F1 is the HateCheck 2-class macro; FPR is the false-positive rate on non-hate cases, the number to watch if you fear over-blocking. Full sweep tables (0.30 to 0.70) live in the training repo (reports/RELEASE_sweep.md).

lang	tuned threshold	macro-F1 @ tuned	FPR @ tuned	macro-F1 @ 0.40	FPR @ 0.40
en	0.40	0.810	0.281	0.810	0.281
de	0.55	0.822	0.254	0.803	0.359
fr	0.45	0.828	0.247	0.824	0.272
nl	0.55	0.768	0.238	0.764	0.298
pt	0.60	0.800	0.203	0.791	0.288
es	0.55	0.803	0.207	0.796	0.272
it	0.60	0.779	0.234	0.775	0.309
pl	0.30	0.759	0.305	0.751	0.271

Read: raising the threshold trades hate recall for a lower non-hate FPR. At per-language tuned thresholds the seven-language multilingual average (all languages except English) is 0.794 and four languages (de, fr, pt, es) clear 0.80. The same menu ships in toxic_meta.json.

Failure modes (read before deploying)

This section is the practical read: what actually goes wrong, what it looks like in an app, and what to do about it. The metric tables backing every claim are in the deep-dive section below. Publishing these is part of the product.

1. It sometimes flags people quoting or condemning hate. News reports, moderation discussions, and counter-speech repeat the hateful words they are arguing against, and the model reacts to the words. A comment like "Someone in this thread said we should 'get rid of' immigrants, and that is disgusting" can get flagged as hateful even though it is the opposite. On trap suites this happens on roughly 1 to 5 cases in 10 depending on language, worst in English and German. If your app touches journalism, activism, or lets users discuss moderation, expect this. Mitigation: route flags to human review or a heavier model; never auto-remove on this signal alone.

2. Positive or neutral mentions of identity groups can trip it, mostly outside English. Sentences like "Proud to be a trans woman and having a great day" or "Many Muslim families live in our neighborhood" are sometimes flagged simply because an identity term appears. This is the dominant false-positive source in Polish, Dutch, German and Italian, where up to six in ten of such trap sentences fire. In natural text it is far rarer (the 2 to 3% above), but profile bios and community-pride posts concentrate exactly these sentences. Mitigation: use the higher per-language thresholds from the menu; treat short identity-statement texts as low-confidence.

3. Reclaimed slurs and lookalike words. In-group reclaimed usage (a community member using the term about themselves) is flagged in roughly 4 of 10 English trap cases, and harmless words that merely resemble slurs are occasionally hit too. Mitigation: human review for user-report flows in communities where reclaimed language is common.

4. It misses some slur-based hate in Dutch, German and Polish. Hate that consists of a bare slur with no other abusive phrasing slips through more often in these languages (only 4 to 6 trap cases in 10 caught, vs 7 in 10 for English). Mitigation: if slur coverage is critical, pair the model with a per-language slur wordlist; the two failure profiles are complementary.

5. Hate phrased as a question or an implication is harder. "Why do [group] always ruin everything?" and implied put-downs are caught less reliably than direct insults (roughly 6 to 9 in 10 depending on language, vs near-perfect on explicit threats). Mitigation: none needed for triage use; do not promise users that subtle hate is always caught.

6. Quantization costs, measured on the shipped artifacts.

• Core ML (iOS): the 4-bit compression costs about 3 to 4 points on English. fp16 on the ANE is quality-neutral (en 0.811 vs torch 0.810); the shipped 4-bit palettized model lands en 0.771 and 8-language 0.776 (-1.3pp mean vs torch). English carries most of the palettization loss.
• TFLite (Android): English pays -2.1pp for int4. toxic.tflite (int4 blockwise-32) scores en 0.787 vs 0.808 for the int8 build of the same model; the seven-language multilingual average is identical for both (0.786). If English dominates your traffic and 59 MB matters less than 2pp, ask for the int8 build.
• ONNX (web): the browser buys fidelity with -0.011 8-language F1 and +7.9 MB vs the TFLite artifact. toxic.onnx (int4e) measures 0.775 8-language natively, and ONNX Runtime Web reproduces those scores bit-for-float in the browser. The alternative was shipping toxic.tflite numbers the browser runtime never reproduces (wasm kernel drift up to 0.173 with 4.2% decision flips in the drift study); the web surface ships a number that is actually measured in the deployed runtime.

7. Not yet measured: latency on phone-class chips (iPhone 13/15, mid-range Android); the M3 Ultra and desktop-CPU numbers above bound the graph cost, not the thermal envelope.

The pattern behind 1 to 3: counter-speech, negated hate, reclaimed slurs and positive / neutral identity mentions are the dominant false-positive sources in every language; bare slurs are the dominant miss in nl, de and pl. If your application is FPR-sensitive, use the higher per-language thresholds above and route positives to review instead of auto-actioning.

Per-language deep-dive (per-functionality cells)

HateCheck groups its test cases into named "functionalities" (cells), each isolating one linguistic pattern. Numbers are accuracy on that cell for the flagship int8 artifact at threshold 0.40. Cells ending _h are hateful (an error is a miss); cells ending _nh are non-hateful (an error is a false positive). Decoder for the cell names used below:

cell	plain meaning
counter_quote_nh / counter_ref_nh	quoting / denouncing hate to condemn it
negate_neg_nh	negated hate ("no one should ever hurt them")
slur_reclaimed_nh / slur_homonym_nh	in-group reclaimed slur use / harmless lookalike words
ident_pos_nh / ident_neutral_nh	positive / neutral statements about a group
target_group_nh / target_obj_nh	abuse aimed at non-protected groups / at objects
profanity_nh	casual swearing aimed at nobody
slur_h	hate expressed through a bare slur
derog_impl_h / derog_dehum_h	implied / dehumanizing derogation
threat_dir_h / threat_norm_h	direct threats / statements normalizing violence
phrase_question_h / phrase_opinion_h	hate phrased as a question / as an opinion
spell_*_h	spelling evasion (leetspeak, added / deleted spaces, swapped characters)

Strongest and weakest cells per language:

lang	strongest cells (acc)	weakest cells (acc)
en	threat_norm_h 1.00, phrase_opinion_h 1.00, threat_dir_h 0.99, derog_dehum_h 0.99	counter_ref_nh 0.51, negate_neg_nh 0.53, slur_reclaimed_nh 0.56, counter_quote_nh 0.63, slur_homonym_nh 0.63
de	ref_subs_clause_h 1.00, threat_dir_h 1.00, threat_norm_h 1.00, phrase_opinion_h 1.00	negate_neg_nh 0.39, ident_pos_nh 0.45, counter_ref_nh 0.48, slur_h 0.58
fr	derog_dehum_h 1.00, threat_norm_h 1.00, spell_leet_h 0.99, derog_neg_attrib_h 0.99	ident_neutral_nh 0.59, target_group_nh 0.63, ident_pos_nh 0.67, derog_impl_h 0.69
nl	profanity_h 1.00, threat_dir_h 1.00, phrase_opinion_h 1.00, threat_norm_h 0.99	slur_h 0.41, ident_pos_nh 0.47, negate_neg_nh 0.52, phrase_question_h 0.58
pt	profanity_h 1.00, threat_dir_h 1.00, threat_norm_h 1.00, spell_char_swap_h 1.00	ident_neutral_nh 0.48, target_group_nh 0.57, ident_pos_nh 0.62, slur_h 0.67
es	threat_dir_h 1.00, threat_norm_h 1.00, derog_neg_attrib_h 0.99, spell_char_swap_h 0.99	ident_neutral_nh 0.52, target_group_nh 0.57, ident_pos_nh 0.66, target_obj_nh 0.68
it	profanity_h 1.00, threat_dir_h 1.00, threat_norm_h 0.99, derog_neg_attrib_h 0.99	ident_pos_nh 0.52, ident_neutral_nh 0.53, target_group_nh 0.55, phrase_question_h 0.57
pl	profanity_h 1.00, threat_dir_h 1.00, threat_norm_h 1.00, spell_char_swap_h 0.99	ident_pos_nh 0.41, slur_h 0.56, phrase_question_h 0.66, negate_neg_nh 0.67

Read: explicit threats, normalized violence, profanity-laden abuse and spelling evasion are near-perfect in every language; the errors concentrate in exactly the trap families of failure modes 1 to 5. Full per-cell tables for all eight languages are published in the training repo.

Fairness across protected groups

Two measured views, both on natural (non-synthetic) held-out data at threshold 0.40.

Does it recognize hate equally, whoever the target is? On the held-out test rows with human-annotated target groups (Measuring Hate Speech corpus), HATEFUL recall per protected ground:

ground	n	HATEFUL recall
GENDER	227	0.934
RACE	186	0.898
SEXUAL_ORIENTATION	116	0.948
DESCENT	116	0.940
RELIGION	84	0.941
DISABILITY	27	0.963
AGE	21	0.952

Spread is 6.5 points (0.898 to 0.963): no ground is catastrophically worse, and none falls below ~0.90.

Does it over-fire on the mere mention of an identity? Joining the same test rows back to their raw identity-mention annotations gives, per group, two ranking scores (AUC: 1.0 = perfect separation, 0.5 = chance). "Mention trap" (BPSN) checks whether non-hateful text mentioning the group outscores other people's actual hate: low values mean the model over-fires on the mention. "Protection" (BNSP) checks whether hate against the group outscores other people's harmless text: low values mean the group is under-protected. Note the non-hate rows in this corpus are mostly still insulting or profane (it over-samples abusive text), so the FPR column is a stress number under matched toxicity, not the 2 to 3% natural rate above; the fairness signal is the spread across grounds against the same 0.448 background.

ground	n non-hate	n hateful	FPR	recall	mention trap (BPSN)	protection (BNSP)
AGE	52	21	0.385	0.952	0.889	0.898
DESCENT	350	116	0.400	0.940	0.874	0.810
DISABILITY	26	27	0.462	0.963	0.805	0.891
GENDER	553	227	0.533	0.934	0.804	0.871
RACE	527	186	0.450	0.898	0.865	0.814
RELIGION	298	84	0.436	0.941	0.855	0.846
SEXUAL_ORIENTATION	317	116	0.416	0.948	0.819	0.892

All grounds sit in a narrow band (BPSN 0.80 to 0.89, BNSP 0.81 to 0.90). The two mildest asymmetries, published rather than hidden: GENDER draws the most mention-driven over-firing (FPR 0.533 vs 0.448 background), and RACE has the lowest hate recall (0.898). Both are monitored targets for the next data iteration.

How these numbers were made

The evaluation discipline is the product as much as the weights are:

• Every quality number is re-measured on the exact shipped bytes, not extrapolated from the training checkpoint. Quantized artifacts are re-run through the full scorer per platform (Core ML, LiteRT, ONNX, torch).
• Benchmarks are eval-only. HateCheck and its multilingual extension were never trained on, in any language. The natural-corpus test split is carved by content hash before training and never touched.
• Experiments were pre-registered. Each of the ~30 logged training experiments stated its hypothesis, its single changed axis, and numeric falsifiers (the conditions under which the idea would be declared wrong) before the run started. Several were falsified and reverted; the record keeps both outcomes.
• Results were adversarially reviewed. Twenty methodology review passes (data leakage, confounds, over-generalization, severity audits of the synthetic data) ran against the experiment log during development, and their findings drove the next iteration.
• Ship candidates were replicated across three seeds; the release model was selected on the seed band, not a lucky single run.
• Competitors run through the identical harness, same scorer, same threshold discipline, with their quantized and full-precision builds both measured rather than assumed.

License

Desert Ant Labs Source-Available License. Free for most apps; a commercial license is required at scale. Full terms at the link. Licensing: licensing@desertant.com.

Built exclusively from commercially clean components (CC0 / CC-BY / MIT / Apache-2.0 training data, MIT base encoder). Attributions, dataset citations (GAHD, Measuring Hate Speech, GermEval 2018, MLMA and others) and generator credits are in THIRD_PARTY_NOTICES.md.