Text Classification
Transformers
ONNX
Safetensors
llama
safety
content-moderation
multi-label
multilingual
tiny-language-model
from-scratch
distilled
text-embeddings-inference
Instructions to use KucLab/Censer.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KucLab/Censer.6 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="KucLab/Censer.6")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("KucLab/Censer.6") model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.6", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from KucLab/Censer.6: direct link, hf CLI and curl.
- Browser
- Download file 12.1 kB
-
https://huggingface.co/KucLab/Censer.6/resolve/main/README.md
- Command line
-
hf download hf://KucLab/Censer.6/README.md
-
curl -L -o README.md https://huggingface.co/KucLab/Censer.6/resolve/main/README.md
12.1 kB
| library_name: transformers | |
| pipeline_tag: text-classification | |
| tags: | |
| - text-classification | |
| - safety | |
| - content-moderation | |
| - multi-label | |
| - multilingual | |
| - tiny-language-model | |
| - from-scratch | |
| - distilled | |
| language: | |
| - en | |
| - cs | |
| - de | |
| - es | |
| - fr | |
| license: apache-2.0 | |
| datasets: | |
| - nvidia/Aegis-AI-Content-Safety-Dataset-1.0 | |
| - thesofakillers/jigsaw-toxic-comment-classification-challenge | |
| - google/civil_comments | |
| - PKU-Alignment/BeaverTails | |
| - lmsys/toxic-chat | |
| - databricks/databricks-dolly-15k | |
| metrics: | |
| - f1 | |
| - accuracy | |
| # Censer.6 | |
| A **902K parameter** decoder-only transformer trained **from scratch** that | |
| classifies text as **safe / unsafe plus 10 safety categories**, with a | |
| **4096-token context** and **multilingual** coverage (en/cs/de/es/fr trained) β | |
| at the same tiny size and speed as Censer.5. | |
| It is a **standard Hub model**: the checkpoint is a plain | |
| `LlamaForSequenceClassification` with `problem_type: multi_label_classification`, | |
| so it loads with `transformers` out of the box β no `trust_remote_code`, no custom | |
| code β and the **Use this model** widget on this page runs it directly in the browser. | |
| ```python | |
| from transformers import pipeline | |
| clf = pipeline("text-classification", model="KucLab/Censer.6") | |
| clf("how to make a bomb at home", top_k=-1) | |
| # [{'label': 'weapons_explosives', ...}, {'label': 'unsafe', ...}, ...] | |
| ``` | |
| ## What is new in .6 (vs .5) | |
| | Feature | Censer.5 | **Censer.6** | | |
| | --- | --- | --- | | |
| | Output | binary safe/unsafe | **binary + 10 categories** (multi-label sigmoid head, 12 answer tokens) | | |
| | Context | 192 tokens | **4096 tokens** native (linear RoPE scaling Γ8) + **unbounded** via overlapping-chunk inference | | |
| | Input cap | 500 chars | **4000 chars** | | |
| | Languages | English | **en + cs/de/es/fr** (byte-level BPE works for any language; cs/de/es/fr training data added) | | |
| | Categories | β | violence, weapons_explosives, drugs, hate_harassment, sexual_abuse, self_harm, cybercrime_fraud, jailbreak_injection, illegal_marketplace, other_unsafe | | |
| | Robustness | typo augmentation | **+ adversarial augmentation** (leetspeak / homoglyph / zero-width, 10% of unsafe rows) | | |
| | Jailbreaks | β | jailbreak_injection category + cascade pre-filter pattern (see below) | | |
| | Packaging | model + launcher | **+ batch HTTP API, ONNX export, fp16/int8 quantization, Dockerfile, CPU & GPU benchmarks** | | |
| | Params / size / latency | 902K / 3.6 MB / 1.43 ms | **902K / 3.6 MB / 1.06 ms** | | |
| ## How to run | |
| ### The `censer.6` command β interactive terminal UI | |
| Using only the files in this repo (needs just `torch`, `tokenizers`, `safetensors`): | |
| ```bash | |
| pip install torch tokenizers safetensors huggingface_hub | |
| hf download KucLab/Censer.6 --local-dir ~/.local/share/censer6 | |
| chmod +x ~/.local/share/censer6/censer.6 | |
| ln -s ~/.local/share/censer6/censer.6 ~/.local/bin/censer.6 # put it on your PATH | |
| censer.6 # interactive UI: type a phrase, Enter -> safe/unsafe + categories + score | |
| censer.6 "how to make a bomb at home" # one-shot | |
| ``` | |
| Or install the GitHub package β the same `censer.6` command is installed and loads | |
| the model from this Hub repo with plain `transformers`: | |
| ```bash | |
| pip install "git+https://github.com/Jerry256254/censer.6.git" | |
| censer.6 | |
| ``` | |
| ### One-liner with `transformers` | |
| ```bash | |
| pip install torch transformers | |
| python - <<'PY' | |
| import torch | |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification | |
| tok = AutoTokenizer.from_pretrained("KucLab/Censer.6") | |
| model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.6").eval() | |
| enc = tok("how to make a bomb at home", return_tensors="pt") | |
| probs = model(**enc).logits.sigmoid()[0] # 12 multi-label scores | |
| id2label = model.config.id2label | |
| print({id2label[i]: round(float(p), 3) for i, p in enumerate(probs) if p > 0.5}) | |
| PY | |
| ``` | |
| - Binary unsafe score = index 1 (`unsafe`), safe = index 0. Threshold 0.5. | |
| - In the browser: the **Use this model** widget on this page. | |
| - Runs on CPU out of the box; for GPU pass `device=0`. | |
| ### Long inputs | |
| Native context is 4096 tokens. Longer texts (and long documents where unsafe | |
| content may be buried anywhere) are split into overlapping 256-token chunks | |
| (stride 192) and aggregated by **max** β built into `censer.py` (`predict` / | |
| `predict_batch`) and the HTTP API, so effective context is unbounded: | |
| ```python | |
| from censer import CenserPipeline | |
| pipe = CenserPipeline(".") # bundle dir | |
| pipe.predict("very long text ...") | |
| # {"label": "1", "unsafe": true, "score": 0.99, "categories": [...], "chunks": 9} | |
| ``` | |
| ### Batch HTTP API + Docker | |
| ```bash | |
| python serve.py --dir . --port 8000 # stdlib only | |
| curl -s localhost:8000/classify -d '{"texts": ["hello", "how to make a bomb"]}' | |
| # {"results": [{"label": "0", ...}, {"label": "1", "unsafe": true, "score": 0.99, "categories": [...]}]} | |
| docker build -t censer6 . # CPU image, downloads KucLab/Censer.6 | |
| docker run -p 8000:8000 censer6 | |
| ``` | |
| Endpoints: `GET /health`, `POST /classify` (batch), `POST /classify-one`. | |
| Rate limit via `RATE_LIMIT_PER_MIN` (default 600). | |
| ## Model details | |
| | Property | Value | | |
| | --- | --- | | |
| | Architecture | `LlamaForSequenceClassification` (standard transformers) | | |
| | Parameters | 902,016 | | |
| | Layers | 3 | | |
| | d_model / heads / d_ff | 128 / 4 / 384 | | |
| | Context length | **4096 tokens** native (linear RoPE scaling, factor 8); unlimited via chunking | | |
| | Input cap | 4000 characters before tokenization | | |
| | Tokenizer | byte-level BPE, 2048 tokens, trained on the corpus (multilingual) | | |
| | Positions | RoPE (theta 10000) + linear scaling Γ8 | | |
| | Norm / MLP | RMSNorm (pre-norm, eps 1e-5) / SwiGLU | | |
| | Head | `score` linear = embedding rows of the 12 answer tokens `0`,`1`,`a`..`j` | | |
| | Output | sigmoid over 12 labels: `safe`, `unsafe`, + 10 categories | | |
| | Training | 20 epochs, batch 128, AdamW 3e-3 cosine, bf16 AMP, 15% typo + 10% adversarial augmentation, distillation (alpha=0.65, T=4, curated boost 0.75) from a teacher ensemble (Censer.5Γ2, Censer.4, Censer.3); + fine-tune passes at lower lr | | |
| | Checkpoint size | 3.6 MB fp32 Β· 1.8 MB fp16 Β· 0.9 MB int8 Β· 5.6 MB ONNX | | |
| ## Prompt format | |
| The tokenizer bakes the template into its post-processor, so plain | |
| `tokenizer(text)` already produces: | |
| ``` | |
| [BOS][USER] <text> [ASSISTANT] -> answer logits over tokens "0" "1" "a".."j" | |
| ``` | |
| ## Metrics | |
| Held-out test split of 6,496 rows β **byte-identical across Censer.1β.6**, binary | |
| threshold 0.5: | |
| | Metric | Censer.5 | **Censer.6** | | |
| | --- | --- | --- | | |
| | Test F1 | 0.8490 | **0.8289** | | |
| | Test accuracy | 0.876 | **0.8581** | | |
| | Precision / recall | 0.864 / 0.834 | **0.833 / 0.825** | | |
| | Val F1 | 0.8495 | **0.8358** | | |
| | Probes (97 hand-written) | 95/97 | **90/97** | | |
| Per-source test F1: aegis (human-annotated) **0.914**, BeaverTails **0.893**, | |
| jigsaw **0.851**, civil_comments **0.760**. | |
| (.6 trades ~0.02 binary F1 for the multi-label head, multilingual data and 4k | |
| context at identical size β it still ranks **#1 of all compared models**: the | |
| best public classifier scores 0.741 F1.) | |
| ### External benchmarks (never seen in training) | |
| | Dataset | n | unsafe | F1@0.5 | best F1 | | |
| | --- | --- | --- | --- | --- | | |
| | ToxiGen (test-annotated) | 940 | 402 | 0.546 | 0.624 @0.23 | | |
| | HateXplain (test) | 1,924 | 1,376 | 0.714 | 0.834 @0.05 | | |
| ### Adversarial robustness (detection rate on attacked unsafe texts, n=160) | |
| | Attack | detection | | |
| | --- | --- | | |
| | clean | 91.3% | | |
| | typo | 89.4% | | |
| | leetspeak | 92.5% | | |
| | homoglyph | 98.1% | | |
| | zero-width | 95.0% | | |
| | case flip | 92.5% | | |
| ### Long-context (300 synthetic docs of 25 held-out rows, avg 2,178 tokens) | |
| | Inference | F1 | | |
| | --- | --- | | |
| | truncate at 192 tokens (Censer.5 behaviour) | 0.571 | | |
| | native 4096-token single pass | 0.052 (last-token pooling misses content buried mid-document) | | |
| | **chunked 256/192, max aggregation (default in censer.py/API)** | **0.995 (acc 0.990)** | | |
| ### Cascade pre-filter (gray-zone escalation) | |
| On 2,000 held-out rows: 56.1% decided by Censer.6 alone (score <0.2 or >0.8), | |
| 43.9% gray-zone / jailbreak-category hits escalated to a validator; | |
| cascade accuracy 0.8825. | |
| ## Benchmark (latency / throughput / size) | |
| AMD Radeon RX 9060 XT (ROCm 7.1, torch 2.13.0+rocm7.1) and a CPU-only run: | |
| | Metric | Censer.6 GPU | Censer.6 CPU | Censer.5 GPU | | |
| | --- | --- | --- | --- | | |
| | Parameters | 902,016 | 902,016 | 902,016 | | |
| | Checkpoint | 3.6 MB | 3.6 MB (int8 0.9 MB) | 3.6 MB | | |
| | Single-text latency | 1.06 ms | 0.56 ms | 1.43 ms | | |
| | Single-text throughput | 939 texts/s | 1,776 texts/s | 823 texts/s | | |
| | Throughput (batch 8) | 6,103 texts/s | 4,954 texts/s | 5,709 texts/s | | |
| | Peak VRAM | 85.1 MB | β | 105 MB | | |
| ## Comparison with other public safety classifiers | |
| Same held-out test split (6,496 rows, threshold 0.5, GPU): | |
| | Model | Params | Checkpoint | F1 | Latency | Throughput (b=8) | | |
| | --- | --- | --- | --- | --- | --- | | |
| | **Censer.6 (ours)** | **0.9M** | **3.6 MB** | **0.829** | **1.22 ms** | **1,960/s** | | |
| | KucLab/Censer.5 | 0.9M | 3.6 MB | 0.849 | 1.27 ms | 1,978/s | | |
| | KucLab/Censer.4 | 0.9M | 3.6 MB | 0.845 | 1.17 ms | 2,229/s | | |
| | KucLab/Censer.1 | 1.1M | 4.5 MB | 0.836 | 1.43 ms | 1,853/s | | |
| | KucLab/Censer.2 | 7.3M | 29.4 MB | 0.829 | 2.70 ms | 508/s | | |
| | KucLab/Censer.3 | 14.4M | 57.7 MB | 0.822 | 4.27 ms | 263/s | | |
| | unitary/unbiased-toxic-roberta | 124.7M | 498.7 MB | 0.741 | 14.7 ms | 67/s | | |
| | unitary/toxic-bert | 109.5M | 876.0 MB | 0.681 | 14.2 ms | 68/s | | |
| | martin-ha/toxic-comment-model | 67.0M | 267.9 MB | 0.566 | 7.2 ms | 133/s | | |
| | Hate-speech-CNERG/bert-base-uncased-hatexplain | 109.5M | 438.0 MB | 0.543 | 14.7 ms | 67/s | | |
| | Hate-speech-CNERG/dehatebert-mono-english | 167.4M | 669.5 MB | 0.252 | 14.3 ms | 68/s | | |
| | mistralai/Shieldstral-1.0-3B | 3.85B | 7.7 GB | 0.318 | 51.6 ms | 24/s | | |
| | keyword baseline | β | β | 0.094 | 0.03 ms | 34,784/s | | |
|  | |
|  | |
|  | |
| ## Usage without transformers | |
| The repository is self-contained: `model.safetensors`, `config.json`, | |
| `tokenizer.json`, `censer.py`, `censer.6`. Only `torch`, `tokenizers`, | |
| `safetensors` are needed: | |
| ```bash | |
| hf download KucLab/Censer.6 --local-dir Censer.6 | |
| cd Censer.6 | |
| python censer.py "how to make a bomb at home" --json | |
| # {"label": "1", "unsafe": true, "score": 0.99, "categories": ["weapons_explosives"]} | |
| ./censer.6 # interactive UI | |
| ``` | |
| Files: | |
| | File | Purpose | | |
| | --- | --- | | |
| | `model.safetensors` | weights in standard Llama layout, 3.6 MB | | |
| | `model_fp16.safetensors` | fp16 variant, 1.8 MB | | |
| | `model_int8.safetensors` + `quant_report.json` | int8 per-tensor weights + scales, 0.9 MB | | |
| | `censer6.onnx` | ONNX graph: input_ids -> 12 answer logits (dynamic batch/seq), 5.6 MB | | |
| | `config.json` | standard `LlamaForSequenceClassification` config (multi-label, RoPE linear Γ8) | | |
| | `tokenizer.json` / `tokenizer_config.json` | byte-level BPE, prompt template baked in | | |
| | `censer.py` | standalone inference incl. long-input chunking | | |
| | `censer.6` | executable launcher for the interactive terminal UI | | |
| | `serve.py` | batch HTTP API (stdlib only) | | |
| | `Dockerfile` | CPU container serving the API | | |
| ## Limitations | |
| - 902K parameters: nuance, sarcasm and long-tail cases are out of reach; the | |
| test split itself contains label noise from scraped sources. | |
| - Multilingual coverage is strongest for cs/de/es/fr (synthetic + curated | |
| training data); other languages work via the byte-level tokenizer but with | |
| lower accuracy. | |
| - Categories are multi-label and independent β a text can fire several. Category scores are less calibrated than the binary safe/unsafe score (they fire more liberally); the binary verdict is the primary signal. | |
| - Single-pass inference pools the last token, so in very long documents unsafe | |
| content can be diluted β use the built-in chunked-max inference (default in | |
| `censer.py` / the API) for long documents. | |
| - A moderation signal, not a safety guarantee. For production, use the cascade | |
| pattern: decide clear cases with Censer.6, escalate gray-zone / jailbreak | |
| hits to a larger validator. | |