| --- |
| language: en |
| license: apache-2.0 |
| tags: |
| - text-classification |
| - toxic-comments |
| - distilbert |
| - multi-label-classification |
| datasets: |
| - jigsaw-toxic-comment-classification |
| metrics: |
| - f1 |
| - roc_auc |
| --- |
| |
| # Toxic Comment Classifier |
|
|
| Fine-tuned DistilBERT for multi-label toxic comment classification, |
| wrapped in a live FastAPI endpoint with Docker deployment. |
|
|
| ## Model Details |
| - **Base model:** distilbert-base-uncased |
| - **Task:** Multi-label text classification (6 labels) |
| - **Labels:** toxic, severe_toxic, obscene, threat, insult, identity_hate |
| - **Training data:** Jigsaw Toxic Comment Classification (159,571 comments) |
| - **Framework:** HuggingFace Transformers + Trainer API |
| - **Training time:** ~28 minutes on T4 GPU |
|
|
| ## Performance |
| | Metric | Score | |
| |--------|-------| |
| | ROC-AUC (macro) | 0.990 | |
| | F1 (macro) | 0.652 | |
|
|
| > F1 is lower due to severe class imbalance — threat label has only 478 |
| > positive examples out of 159k. ROC-AUC of 0.990 reflects true model quality. |
|
|
| ## Latency Benchmark (GPU, single request) |
| | Percentile | Latency | |
| |------------|---------| |
| | p50 | 7.3ms | |
| | p95 | 9.9ms | |
| | p99 | 19.4ms | |
| | min | 4.1ms | |
| | max | 26.2ms | |
| | mean | 7.3ms | |
|
|
| ## Live API |
| - Swagger UI: https://chandkr123-toxic-classifier-api.hf.space/docs |
| - Predict endpoint: https://chandkr123-toxic-classifier-api.hf.space/predict |
|
|
| ## Example Predictions |
| | Comment | Flagged | |
| |---------|---------| |
| | I love this community! | clean | |
| | You are the most stupid idiot | toxic, insult, obscene | |
| | I will find you and hurt you | toxic, threat | |
| | I will beat you in the hotel | toxic, threat | |
|
|
| ## Limitations |
| - Trained on Wikipedia comments, may not generalise to all domains |
| - Severe class imbalance: threat label has only 478 training examples |
| - Predictions above 0.5 threshold are flagged (adjustable) |
| - Not suitable for production content moderation without human review |
|
|
