metadata
language: en
license: apache-2.0
tags:
- text-classification
- toxic-comments
- distilbert
- multi-label-classification
datasets:
- jigsaw-toxic-comment-classification
metrics:
- f1
- roc_auc
Toxic Comment Classifier
Fine-tuned DistilBERT for multi-label toxic comment classification, wrapped in a live FastAPI endpoint with Docker deployment.
Model Details
- Base model: distilbert-base-uncased
- Task: Multi-label text classification (6 labels)
- Labels: toxic, severe_toxic, obscene, threat, insult, identity_hate
- Training data: Jigsaw Toxic Comment Classification (159,571 comments)
- Framework: HuggingFace Transformers + Trainer API
- Training time: ~28 minutes on T4 GPU
Performance
| Metric | Score |
|---|---|
| ROC-AUC (macro) | 0.990 |
| F1 (macro) | 0.652 |
F1 is lower due to severe class imbalance — threat label has only 478 positive examples out of 159k. ROC-AUC of 0.990 reflects true model quality.
Latency Benchmark (GPU, single request)
| Percentile | Latency |
|---|---|
| p50 | 7.3ms |
| p95 | 9.9ms |
| p99 | 19.4ms |
| min | 4.1ms |
| max | 26.2ms |
| mean | 7.3ms |
Live API
- Swagger UI: https://chandkr123-toxic-classifier-api.hf.space/docs
- Predict endpoint: https://chandkr123-toxic-classifier-api.hf.space/predict
Example Predictions
| Comment | Flagged |
|---|---|
| I love this community! | clean |
| You are the most stupid idiot | toxic, insult, obscene |
| I will find you and hurt you | toxic, threat |
| I will beat you in the hotel | toxic, threat |
Limitations
- Trained on Wikipedia comments, may not generalise to all domains
- Severe class imbalance: threat label has only 478 training examples
- Predictions above 0.5 threshold are flagged (adjustable)
- Not suitable for production content moderation without human review