Text Classification
Transformers
ONNX
Safetensors
llama
safety
content-moderation
multi-label
multilingual
calibrated
tiny-language-model
from-scratch
distilled
text-embeddings-inference
Instructions to use KucLab/Censer.7 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KucLab/Censer.7 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="KucLab/Censer.7")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("KucLab/Censer.7") model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.7", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from KucLab/Censer.7: direct link, hf CLI and curl.
- Browser
- Download file 13.3 kB
-
https://huggingface.co/KucLab/Censer.7/resolve/main/README.md
- Command line
-
hf download hf://KucLab/Censer.7/README.md
-
curl -L -o README.md https://huggingface.co/KucLab/Censer.7/resolve/main/README.md
13.3 kB
| library_name: transformers | |
| pipeline_tag: text-classification | |
| tags: | |
| - text-classification | |
| - safety | |
| - content-moderation | |
| - multi-label | |
| - multilingual | |
| - calibrated | |
| - tiny-language-model | |
| - from-scratch | |
| - distilled | |
| language: | |
| - en | |
| - cs | |
| - de | |
| - es | |
| - fr | |
| - it | |
| - pt | |
| - ru | |
| - ar | |
| - zh | |
| - hi | |
| license: apache-2.0 | |
| datasets: | |
| - nvidia/Aegis-AI-Content-Safety-Dataset-1.0 | |
| - thesofakillers/jigsaw-toxic-comment-classification-challenge | |
| - google/civil_comments | |
| - PKU-Alignment/BeaverTails | |
| - lmsys/toxic-chat | |
| - databricks/databricks-dolly-15k | |
| - Hate-speech-CNERG/hatexplain | |
| - Intuit-GenSRF/jigsaw-multilingual-train-unique | |
| metrics: | |
| - f1 | |
| - accuracy | |
| - ece | |
| # Censer.7 | |
| A **902K parameter** decoder-only transformer trained **from scratch**: binary | |
| **safe / unsafe** (F1 restored to .5 level) **+ 10 calibrated safety categories**, | |
| **4096-token** native context with chunked-max inference for long documents, and | |
| **multilingual** coverage (trained on en + cs/de/es/fr synthetic + Jigsaw-multilingual | |
| fr/it/pt/es/ru/tr) β at the same tiny size and speed. | |
| Standard Hub model: plain `LlamaForSequenceClassification` with | |
| `problem_type: multi_label_classification` β loads with `transformers` out of the box, | |
| no `trust_remote_code`. | |
| ```python | |
| from transformers import pipeline | |
| clf = pipeline("text-classification", model="KucLab/Censer.7") | |
| clf("how to make a bomb at home", top_k=-1) | |
| # [{'label': 'unsafe', ...}, {'label': 'weapons_explosives', ...}, ...] | |
| ``` | |
| ## What is new in .7 (vs .6) | |
| | Fix | .6 | **.7** | | |
| | --- | --- | --- | | |
| | Binary test F1 | 0.829 (regressed from .5's 0.849) | **0.8348** (two-phase training: binary first, then categories at low lr with cat_weight) | | |
| | Probes | 90/97 | **95/97** (diff vs .5: fixed 'Wait, Rockstar really give us fake hope', regressed 'buy a gun') | | |
| | External benchmarks | ToxiGen 0.546, HateXplain 0.714, no head-to-head | **ToxiGen 0.561 / HateXplain 0.738 + direct head-to-head vs unbiased-toxic-roberta on the same sets** (table below); HateXplain-train + Jigsaw-multilingual added to training (test splits stay pure) | | |
| | Category scores | uncalibrated ("fire liberally") | **per-category thresholds tuned on held-out catval + ECE + reliability diagram** (see below); thresholds ship in `config.json` as `cat_thresholds` | | |
| | Multilingual | cs/de/es/fr only, no eval | **per-language F1 on textdetox multilingual toxicity (12 languages) + held-out synthetic** (table below) | | |
| | Long documents | native 4096 pass silently broken (F1 0.052) | **chunked-max (256/192) is the only documented path for long docs; native single-pass is explicitly marked NOT suitable for long documents** (last-token pooling); chunked F1 0.998 (acc 0.997) | | |
| | Training data | 176k rows | **251,655 rows** (+HateXplain train 15k, +Jigsaw-ML 60k); val/test byte-identical to .1β.6 | | |
| ## How to run | |
| ### The `censer.7` command β interactive terminal UI | |
| ```bash | |
| pip install torch tokenizers safetensors huggingface_hub | |
| hf download KucLab/Censer.7 --local-dir ~/.local/share/censer7 | |
| chmod +x ~/.local/share/censer7/censer.7 | |
| ln -s ~/.local/share/censer7/censer.7 ~/.local/bin/censer.7 | |
| censer.7 # interactive UI: phrase in -> safe/unsafe + categories + score | |
| censer.7 "how to make a bomb at home" | |
| ``` | |
| Or via the GitHub package (loads the Hub model with plain `transformers`): | |
| ```bash | |
| pip install "git+https://github.com/Jerry256254/censer.7.git" | |
| censer.7 | |
| ``` | |
| ### One-liner with `transformers` | |
| ```bash | |
| pip install torch transformers | |
| python - <<'PY' | |
| import torch | |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification | |
| tok = AutoTokenizer.from_pretrained("KucLab/Censer.7") | |
| model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.7").eval() | |
| enc = tok("how to make a bomb at home", return_tensors="pt") | |
| probs = model(**enc).logits.sigmoid()[0] | |
| id2label = model.config.id2label | |
| cat_thr = model.config.cat_thresholds or {} | |
| print({id2label[i]: round(float(p), 3) for i, p in enumerate(probs) if p > cat_thr.get(id2label[i], 0.5)}) | |
| PY | |
| ``` | |
| - Binary unsafe score = index 1; threshold 0.5. Category labels use per-category | |
| calibrated thresholds (`config.json: cat_thresholds`). | |
| - Runs on CPU out of the box; GPU via `device=0`. | |
| ### Long inputs β always use the chunked path | |
| Native context is 4096 tokens, but a **single pass pools only the last token**, so in | |
| long documents unsafe content in the middle can be missed (measured F1 0.052 on | |
| 25-row synthetic docs). Therefore `censer.py` / the API **always** split inputs longer | |
| than 256 tokens into overlapping chunks (256 tokens, stride 192) and aggregate by | |
| **max** β this is the default and only recommended path for long documents | |
| (measured F1 0.998 (acc 0.997) on the same docs): | |
| ```python | |
| from censer import CenserPipeline | |
| pipe = CenserPipeline(".") | |
| pipe.predict("very long text ...") # {"label": "1", "unsafe": true, "score": ..., "categories": [...], "chunks": N} | |
| ``` | |
| ### Batch HTTP API + Docker | |
| ```bash | |
| python serve.py --dir . --port 8000 | |
| curl -s localhost:8000/classify -d '{"texts": ["hello", "how to make a bomb"]}' | |
| docker build -t censer7 . && docker run -p 8000:8000 censer7 | |
| ``` | |
| Endpoints: `GET /health`, `POST /classify` (batch), `POST /classify-one`. | |
| Rate limit via `RATE_LIMIT_PER_MIN` (default 600). | |
| ## Model details | |
| | Property | Value | | |
| | --- | --- | | |
| | Architecture | `LlamaForSequenceClassification` (standard transformers) | | |
| | Parameters | 902,016 | | |
| | Layers / d_model / heads / d_ff | 3 / 128 / 4 / 384 | | |
| | Context | 4096 tokens native (linear RoPE Γ8); long docs via chunked-max (default) | | |
| | Input cap | 4000 chars | | |
| | Tokenizer | byte-level BPE, 2048 tokens (multilingual corpus) | | |
| | Head | `score` = embedding rows of the 12 answer tokens `0`,`1`,`a`..`j` | | |
| | Output | sigmoid over 12 labels; per-category calibrated thresholds in config | | |
| | Training | two-phase: (A) 5 epochs binary (cat_weight=0, lr 2e-3) from Censer.6 init, (B) 3 epochs categories (cat_weight=0.5, lr 3e-4); batch 256, AdamW cosine, bf16 AMP, 15% typo + 5% adversarial aug, distillation (alpha=0.65, T=4, boost 0.75) from Censer.5Γ2+.4+.3 ensemble on 251,655 rows | | |
| | Checkpoint | 3.6 MB fp32 Β· 1.8 MB fp16 Β· 0.9 MB int8 Β· ONNX | | |
| ## Prompt format | |
| `tokenizer(text)` produces `[BOS][USER] <text> [ASSISTANT] ->` answer logits over | |
| `"0" "1" "a".."j"` (template baked into the post-processor). | |
| ## Metrics | |
| Internal held-out test split (6,496 rows, byte-identical across Censer.1β.7), binary | |
| threshold 0.5: | |
| | Metric | Censer.5 | Censer.6 | **Censer.7** | | |
| | --- | --- | --- | --- | | |
| | Test F1 | 0.8490 | 0.8289 | **0.8348** | | |
| | Test accuracy | 0.876 | 0.858 | **0.8655** | | |
| | Precision / recall | 0.864 / 0.834 | 0.833 / 0.825 | **0.855 / 0.816** | | |
| | Val F1 | 0.8495 | 0.8358 | **0.8368** | | |
| | Probes (97 hand-written) | 95/97 | 90/97 | **95/97** | | |
| Per-source test F1: aegis 0.912, BeaverTails 0.881, jigsaw 0.856, civil_comments 0.776. | |
| ### External benchmarks (never in training) β with head-to-head vs public models | |
| | Dataset | Model | F1@0.5 | best F1 | latency | | |
| | --- | --- | --- | --- | --- | | |
| | ToxiGen annotated (n=940, 402 unsafe) | **Censer.7** | **0.561** | 0.619 @ 0.27 | 1.24 ms | | |
| | ToxiGen | unitary/unbiased-toxic-roberta | 0.443 | 0.483 | ~14 ms | | |
| | HateXplain test (n=1,924, 1,376 unsafe) | **Censer.7** | **0.738** | 0.834 @ 0.05 | 1.24 ms | | |
| | HateXplain test | unitary/unbiased-toxic-roberta | 0.577 | 0.690 | ~14 ms | | |
| (full numbers in `reports/external_h2h.json`) | |
| ### Adversarial robustness (detection rate on attacked unsafe texts, n=160) | |
| | Attack | .6 | **.7** | | |
| | --- | --- | --- | | |
| | clean | 91.3% | 93.8% | | |
| | typo | 89.4% | 88.8% | | |
| | leetspeak | 92.5% | 93.1% | | |
| | homoglyph | 98.1% | 98.1% | | |
| | zero-width | 95.0% | 97.5% | | |
| | case flip | 92.5% | 91.9% | | |
| ### Long-context (300 synthetic docs of 25 held-out rows, avg 2,178 tokens) | |
| | Inference | F1 | | |
| | --- | --- | | |
| | truncate at 192 tokens | 0.571 | | |
| | native 4096 single pass (NOT recommended for long docs) | 0.052 | | |
| | **chunked 256/192 max (default everywhere)** | **0.998 (acc 0.997)** | | |
| ### Category calibration (held-out HateXplain val, n=1,922) | |
| Binary-score ECE = **0.1798**. Per-category tuned thresholds (F1-maximizing) ship in | |
| `config.json: cat_thresholds`: | |
| | Category | threshold | F1@thr | ECE | | |
| | --- | --- | --- | --- | | |
| | violence | 0.20 | 0.000 | 0.195 | | |
| | weapons_explosives | 0.20 | 0.000 | 0.140 | | |
| | drugs | 0.20 | 0.000 | 0.103 | | |
| | hate_harassment | 0.20 | 0.834 | 0.112 | | |
| | sexual_abuse | 0.20 | 0.000 | 0.124 | | |
| | self_harm | 0.20 | 0.000 | 0.095 | | |
| | cybercrime_fraud | 0.20 | 0.000 | 0.130 | | |
| | jailbreak_injection | 0.20 | 0.000 | 0.088 | | |
| | illegal_marketplace | 0.20 | 0.000 | 0.083 | | |
| | other_unsafe | 0.20 | 0.000 | 0.147 | | |
|  | |
| ### Multilingual β per-language binary F1 (textdetox multilingual toxicity, never in training; + held-out synthetic) | |
| | Language | n | unsafe | F1@0.5 | best F1 | | |
| | --- | --- | --- | --- | --- | | |
| | synth_cs | 300 | 150 | 1.000 | 1.000 | | |
| | synth_de | 300 | 150 | 1.000 | 1.000 | | |
| | synth_es | 300 | 150 | 1.000 | 1.000 | | |
| | synth_fr | 300 | 150 | 1.000 | 1.000 | | |
| | textdetox_am | 800 | 385 | 0.141 | 0.646 | | |
| | textdetox_ar | 800 | 387 | 0.000 | 0.049 | | |
| | textdetox_de | 800 | 397 | 0.596 | 0.690 | | |
| | textdetox_en | 800 | 391 | 0.820 | 0.842 | | |
| | textdetox_es | 800 | 408 | 0.543 | 0.674 | | |
| | textdetox_fr | 800 | 391 | 0.512 | 0.676 | | |
| | textdetox_he | 800 | 332 | 0.063 | 0.305 | | |
| | textdetox_hi | 800 | 409 | 0.000 | 0.000 | | |
| | textdetox_hin | 800 | 394 | 0.543 | 0.639 | | |
| | textdetox_it | 800 | 409 | 0.607 | 0.677 | | |
| | textdetox_ja | 800 | 409 | 0.000 | 0.010 | | |
| | textdetox_ru | 800 | 391 | 0.399 | 0.655 | | |
| | textdetox_tt | 800 | 403 | 0.590 | 0.671 | | |
| | textdetox_uk | 800 | 391 | 0.393 | 0.657 | | |
| | textdetox_zh | 800 | 409 | 0.010 | 0.190 | | |
| ### Cascade pre-filter | |
| On 2,000 held-out rows: 42.8% escalated (gray-zone / jailbreak-category hits), | |
| cascade accuracy 0.8600. | |
| ## Benchmark (latency / throughput / size) | |
| AMD Radeon RX 9060 XT (ROCm) and CPU-only: | |
| | Metric | .7 GPU | .7 CPU | .5 GPU | | |
| | --- | --- | --- | --- | | |
| | Parameters | 902,016 | 902,016 | 902,016 | | |
| | Checkpoint | 3.6 MB | 3.6 MB (int8 0.9 MB) | 3.6 MB | | |
| | Single-text latency | 1.24 ms | 0.61 ms | 1.43 ms | | |
| | Single throughput | 804 texts/s | 1628 texts/s | 823/s | | |
| | Batch-8 throughput | 5348 texts/s | 4603 texts/s | 5,709/s | | |
| | Peak VRAM | 85.1 MB | β | 105 MB | | |
| ## Comparison on the internal test split (6,496 rows, threshold 0.5) | |
| | Model | Params | Checkpoint | F1 | Latency | Throughput (b=8) | | |
| | --- | --- | --- | --- | --- | --- | | |
| | **Censer.7 (ours)** | **0.9M** | **3.6 MB** | **0.8348** | **1.39 ms** | **1761/s** | | |
| | KucLab/Censer.5 | 0.9M | 3.6 MB | 0.849 | 1.27 ms | 1,978/s | | |
| | KucLab/Censer.6 | 0.9M | 3.6 MB | 0.829 | 1.22 ms | 1,960/s | | |
| | KucLab/Censer.4 | 0.9M | 3.6 MB | 0.845 | 1.17 ms | 2,229/s | | |
| | KucLab/Censer.1 | 1.1M | 4.5 MB | 0.836 | 1.43 ms | 1,853/s | | |
| | KucLab/Censer.2 | 7.3M | 29.4 MB | 0.829 | 2.70 ms | 508/s | | |
| | KucLab/Censer.3 | 14.4M | 57.7 MB | 0.822 | 4.27 ms | 263/s | | |
| | unitary/unbiased-toxic-roberta | 124.7M | 498.7 MB | 0.741 | 14.7 ms | 67/s | | |
| | unitary/toxic-bert | 109.5M | 876.0 MB | 0.681 | 14.2 ms | 68/s | | |
| | martin-ha/toxic-comment-model | 67.0M | 267.9 MB | 0.566 | 7.2 ms | 133/s | | |
| | Hate-speech-CNERG/bert-base-uncased-hatexplain | 109.5M | 438.0 MB | 0.543 | 14.7 ms | 67/s | | |
| | Hate-speech-CNERG/dehatebert-mono-english | 167.4M | 669.5 MB | 0.252 | 14.3 ms | 68/s | | |
| | mistralai/Shieldstral-1.0-3B | 3.85B | 7.7 GB | 0.318 | 51.6 ms | 24/s | | |
| | keyword baseline | β | β | 0.094 | 0.03 ms | 34,784/s | | |
|  | |
|  | |
|  | |
| ## Usage without transformers | |
| ```bash | |
| hf download KucLab/Censer.7 --local-dir Censer.7 | |
| cd Censer.7 | |
| python censer.py "how to make a bomb at home" --json | |
| ./censer.7 # interactive UI | |
| ``` | |
| Files: | |
| | File | Purpose | | |
| | --- | --- | | |
| | `model.safetensors` | standard Llama layout, 3.6 MB | | |
| | `model_fp16.safetensors` / `model_int8.safetensors` + `quant_report.json` | 1.8 MB / 0.9 MB quantized | | |
| | `censer7.onnx` | ONNX graph: input_ids -> 12 logits, dynamic batch/seq | | |
| | `config.json` | standard config + `cat_thresholds` (calibrated category cutoffs) | | |
| | `tokenizer.json` / `tokenizer_config.json` | byte-level BPE, prompt template baked in | | |
| | `censer.py` | standalone inference, chunked-max for long inputs (default) | | |
| | `censer.7` | launcher for the interactive terminal UI | | |
| | `serve.py` | batch HTTP API (stdlib only) | | |
| | `Dockerfile` | CPU container serving the API | | |
| ## Limitations | |
| - 902K params: nuance/sarcasm/long-tail out of reach; internal test split has label | |
| noise from scraped sources. | |
| - Native single-pass 4096-token inference pools the last token only β for long | |
| documents ALWAYS use the chunked-max path (default in `censer.py`/API). | |
| - Category scores use calibrated per-category thresholds, but remain less reliable | |
| than the binary verdict. | |
| - Multilingual accuracy is best for the trained languages (en/cs/de/es/fr + Jigsaw-ML | |
| languages); see the per-language table β low-resource languages are weaker. | |
| - A moderation signal, not a safety guarantee; use the cascade pattern (clear cases | |
| decided by Censer.7, gray-zone/jailbreak hits escalated to a larger validator). | |