--- library_name: transformers pipeline_tag: text-classification tags: - text-classification - safety - content-moderation - multi-label - multilingual - tiny-language-model - from-scratch - distilled language: - en - cs - de - es - fr license: apache-2.0 datasets: - nvidia/Aegis-AI-Content-Safety-Dataset-1.0 - thesofakillers/jigsaw-toxic-comment-classification-challenge - google/civil_comments - PKU-Alignment/BeaverTails - lmsys/toxic-chat - databricks/databricks-dolly-15k metrics: - f1 - accuracy --- # Censer.6 A **902K parameter** decoder-only transformer trained **from scratch** that classifies text as **safe / unsafe plus 10 safety categories**, with a **4096-token context** and **multilingual** coverage (en/cs/de/es/fr trained) — at the same tiny size and speed as Censer.5. It is a **standard Hub model**: the checkpoint is a plain `LlamaForSequenceClassification` with `problem_type: multi_label_classification`, so it loads with `transformers` out of the box — no `trust_remote_code`, no custom code — and the **Use this model** widget on this page runs it directly in the browser. ```python from transformers import pipeline clf = pipeline("text-classification", model="KucLab/Censer.6") clf("how to make a bomb at home", top_k=-1) # [{'label': 'weapons_explosives', ...}, {'label': 'unsafe', ...}, ...] ``` ## What is new in .6 (vs .5) | Feature | Censer.5 | **Censer.6** | | --- | --- | --- | | Output | binary safe/unsafe | **binary + 10 categories** (multi-label sigmoid head, 12 answer tokens) | | Context | 192 tokens | **4096 tokens** native (linear RoPE scaling ×8) + **unbounded** via overlapping-chunk inference | | Input cap | 500 chars | **4000 chars** | | Languages | English | **en + cs/de/es/fr** (byte-level BPE works for any language; cs/de/es/fr training data added) | | Categories | — | violence, weapons_explosives, drugs, hate_harassment, sexual_abuse, self_harm, cybercrime_fraud, jailbreak_injection, illegal_marketplace, other_unsafe | | Robustness | typo augmentation | **+ adversarial augmentation** (leetspeak / homoglyph / zero-width, 10% of unsafe rows) | | Jailbreaks | — | jailbreak_injection category + cascade pre-filter pattern (see below) | | Packaging | model + launcher | **+ batch HTTP API, ONNX export, fp16/int8 quantization, Dockerfile, CPU & GPU benchmarks** | | Params / size / latency | 902K / 3.6 MB / 1.43 ms | **902K / 3.6 MB / 1.06 ms** | ## How to run ### The `censer.6` command — interactive terminal UI Using only the files in this repo (needs just `torch`, `tokenizers`, `safetensors`): ```bash pip install torch tokenizers safetensors huggingface_hub hf download KucLab/Censer.6 --local-dir ~/.local/share/censer6 chmod +x ~/.local/share/censer6/censer.6 ln -s ~/.local/share/censer6/censer.6 ~/.local/bin/censer.6 # put it on your PATH censer.6 # interactive UI: type a phrase, Enter -> safe/unsafe + categories + score censer.6 "how to make a bomb at home" # one-shot ``` Or install the GitHub package — the same `censer.6` command is installed and loads the model from this Hub repo with plain `transformers`: ```bash pip install "git+https://github.com/Jerry256254/censer.6.git" censer.6 ``` ### One-liner with `transformers` ```bash pip install torch transformers python - <<'PY' import torch from transformers import AutoTokenizer, AutoModelForSequenceClassification tok = AutoTokenizer.from_pretrained("KucLab/Censer.6") model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.6").eval() enc = tok("how to make a bomb at home", return_tensors="pt") probs = model(**enc).logits.sigmoid()[0] # 12 multi-label scores id2label = model.config.id2label print({id2label[i]: round(float(p), 3) for i, p in enumerate(probs) if p > 0.5}) PY ``` - Binary unsafe score = index 1 (`unsafe`), safe = index 0. Threshold 0.5. - In the browser: the **Use this model** widget on this page. - Runs on CPU out of the box; for GPU pass `device=0`. ### Long inputs Native context is 4096 tokens. Longer texts (and long documents where unsafe content may be buried anywhere) are split into overlapping 256-token chunks (stride 192) and aggregated by **max** — built into `censer.py` (`predict` / `predict_batch`) and the HTTP API, so effective context is unbounded: ```python from censer import CenserPipeline pipe = CenserPipeline(".") # bundle dir pipe.predict("very long text ...") # {"label": "1", "unsafe": true, "score": 0.99, "categories": [...], "chunks": 9} ``` ### Batch HTTP API + Docker ```bash python serve.py --dir . --port 8000 # stdlib only curl -s localhost:8000/classify -d '{"texts": ["hello", "how to make a bomb"]}' # {"results": [{"label": "0", ...}, {"label": "1", "unsafe": true, "score": 0.99, "categories": [...]}]} docker build -t censer6 . # CPU image, downloads KucLab/Censer.6 docker run -p 8000:8000 censer6 ``` Endpoints: `GET /health`, `POST /classify` (batch), `POST /classify-one`. Rate limit via `RATE_LIMIT_PER_MIN` (default 600). ## Model details | Property | Value | | --- | --- | | Architecture | `LlamaForSequenceClassification` (standard transformers) | | Parameters | 902,016 | | Layers | 3 | | d_model / heads / d_ff | 128 / 4 / 384 | | Context length | **4096 tokens** native (linear RoPE scaling, factor 8); unlimited via chunking | | Input cap | 4000 characters before tokenization | | Tokenizer | byte-level BPE, 2048 tokens, trained on the corpus (multilingual) | | Positions | RoPE (theta 10000) + linear scaling ×8 | | Norm / MLP | RMSNorm (pre-norm, eps 1e-5) / SwiGLU | | Head | `score` linear = embedding rows of the 12 answer tokens `0`,`1`,`a`..`j` | | Output | sigmoid over 12 labels: `safe`, `unsafe`, + 10 categories | | Training | 20 epochs, batch 128, AdamW 3e-3 cosine, bf16 AMP, 15% typo + 10% adversarial augmentation, distillation (alpha=0.65, T=4, curated boost 0.75) from a teacher ensemble (Censer.5×2, Censer.4, Censer.3); + fine-tune passes at lower lr | | Checkpoint size | 3.6 MB fp32 · 1.8 MB fp16 · 0.9 MB int8 · 5.6 MB ONNX | ## Prompt format The tokenizer bakes the template into its post-processor, so plain `tokenizer(text)` already produces: ``` [BOS][USER] [ASSISTANT] -> answer logits over tokens "0" "1" "a".."j" ``` ## Metrics Held-out test split of 6,496 rows — **byte-identical across Censer.1–.6**, binary threshold 0.5: | Metric | Censer.5 | **Censer.6** | | --- | --- | --- | | Test F1 | 0.8490 | **0.8289** | | Test accuracy | 0.876 | **0.8581** | | Precision / recall | 0.864 / 0.834 | **0.833 / 0.825** | | Val F1 | 0.8495 | **0.8358** | | Probes (97 hand-written) | 95/97 | **90/97** | Per-source test F1: aegis (human-annotated) **0.914**, BeaverTails **0.893**, jigsaw **0.851**, civil_comments **0.760**. (.6 trades ~0.02 binary F1 for the multi-label head, multilingual data and 4k context at identical size — it still ranks **#1 of all compared models**: the best public classifier scores 0.741 F1.) ### External benchmarks (never seen in training) | Dataset | n | unsafe | F1@0.5 | best F1 | | --- | --- | --- | --- | --- | | ToxiGen (test-annotated) | 940 | 402 | 0.546 | 0.624 @0.23 | | HateXplain (test) | 1,924 | 1,376 | 0.714 | 0.834 @0.05 | ### Adversarial robustness (detection rate on attacked unsafe texts, n=160) | Attack | detection | | --- | --- | | clean | 91.3% | | typo | 89.4% | | leetspeak | 92.5% | | homoglyph | 98.1% | | zero-width | 95.0% | | case flip | 92.5% | ### Long-context (300 synthetic docs of 25 held-out rows, avg 2,178 tokens) | Inference | F1 | | --- | --- | | truncate at 192 tokens (Censer.5 behaviour) | 0.571 | | native 4096-token single pass | 0.052 (last-token pooling misses content buried mid-document) | | **chunked 256/192, max aggregation (default in censer.py/API)** | **0.995 (acc 0.990)** | ### Cascade pre-filter (gray-zone escalation) On 2,000 held-out rows: 56.1% decided by Censer.6 alone (score <0.2 or >0.8), 43.9% gray-zone / jailbreak-category hits escalated to a validator; cascade accuracy 0.8825. ## Benchmark (latency / throughput / size) AMD Radeon RX 9060 XT (ROCm 7.1, torch 2.13.0+rocm7.1) and a CPU-only run: | Metric | Censer.6 GPU | Censer.6 CPU | Censer.5 GPU | | --- | --- | --- | --- | | Parameters | 902,016 | 902,016 | 902,016 | | Checkpoint | 3.6 MB | 3.6 MB (int8 0.9 MB) | 3.6 MB | | Single-text latency | 1.06 ms | 0.56 ms | 1.43 ms | | Single-text throughput | 939 texts/s | 1,776 texts/s | 823 texts/s | | Throughput (batch 8) | 6,103 texts/s | 4,954 texts/s | 5,709 texts/s | | Peak VRAM | 85.1 MB | — | 105 MB | ## Comparison with other public safety classifiers Same held-out test split (6,496 rows, threshold 0.5, GPU): | Model | Params | Checkpoint | F1 | Latency | Throughput (b=8) | | --- | --- | --- | --- | --- | --- | | **Censer.6 (ours)** | **0.9M** | **3.6 MB** | **0.829** | **1.22 ms** | **1,960/s** | | KucLab/Censer.5 | 0.9M | 3.6 MB | 0.849 | 1.27 ms | 1,978/s | | KucLab/Censer.4 | 0.9M | 3.6 MB | 0.845 | 1.17 ms | 2,229/s | | KucLab/Censer.1 | 1.1M | 4.5 MB | 0.836 | 1.43 ms | 1,853/s | | KucLab/Censer.2 | 7.3M | 29.4 MB | 0.829 | 2.70 ms | 508/s | | KucLab/Censer.3 | 14.4M | 57.7 MB | 0.822 | 4.27 ms | 263/s | | unitary/unbiased-toxic-roberta | 124.7M | 498.7 MB | 0.741 | 14.7 ms | 67/s | | unitary/toxic-bert | 109.5M | 876.0 MB | 0.681 | 14.2 ms | 68/s | | martin-ha/toxic-comment-model | 67.0M | 267.9 MB | 0.566 | 7.2 ms | 133/s | | Hate-speech-CNERG/bert-base-uncased-hatexplain | 109.5M | 438.0 MB | 0.543 | 14.7 ms | 67/s | | Hate-speech-CNERG/dehatebert-mono-english | 167.4M | 669.5 MB | 0.252 | 14.3 ms | 68/s | | mistralai/Shieldstral-1.0-3B | 3.85B | 7.7 GB | 0.318 | 51.6 ms | 24/s | | keyword baseline | — | — | 0.094 | 0.03 ms | 34,784/s | ![Comparison overview](figs/comparison_overview.png) ![F1 per megabyte](figs/f1_vs_size.png) ![Metric bars](figs/metrics_bars.png) ## Usage without transformers The repository is self-contained: `model.safetensors`, `config.json`, `tokenizer.json`, `censer.py`, `censer.6`. Only `torch`, `tokenizers`, `safetensors` are needed: ```bash hf download KucLab/Censer.6 --local-dir Censer.6 cd Censer.6 python censer.py "how to make a bomb at home" --json # {"label": "1", "unsafe": true, "score": 0.99, "categories": ["weapons_explosives"]} ./censer.6 # interactive UI ``` Files: | File | Purpose | | --- | --- | | `model.safetensors` | weights in standard Llama layout, 3.6 MB | | `model_fp16.safetensors` | fp16 variant, 1.8 MB | | `model_int8.safetensors` + `quant_report.json` | int8 per-tensor weights + scales, 0.9 MB | | `censer6.onnx` | ONNX graph: input_ids -> 12 answer logits (dynamic batch/seq), 5.6 MB | | `config.json` | standard `LlamaForSequenceClassification` config (multi-label, RoPE linear ×8) | | `tokenizer.json` / `tokenizer_config.json` | byte-level BPE, prompt template baked in | | `censer.py` | standalone inference incl. long-input chunking | | `censer.6` | executable launcher for the interactive terminal UI | | `serve.py` | batch HTTP API (stdlib only) | | `Dockerfile` | CPU container serving the API | ## Limitations - 902K parameters: nuance, sarcasm and long-tail cases are out of reach; the test split itself contains label noise from scraped sources. - Multilingual coverage is strongest for cs/de/es/fr (synthetic + curated training data); other languages work via the byte-level tokenizer but with lower accuracy. - Categories are multi-label and independent — a text can fire several. Category scores are less calibrated than the binary safe/unsafe score (they fire more liberally); the binary verdict is the primary signal. - Single-pass inference pools the last token, so in very long documents unsafe content can be diluted — use the built-in chunked-max inference (default in `censer.py` / the API) for long documents. - A moderation signal, not a safety guarantee. For production, use the cascade pattern: decide clear cases with Censer.6, escalate gray-zone / jailbreak hits to a larger validator.