---
language: en
license: mit
base_model: distilbert/distilbert-base-uncased
pipeline_tag: text-classification
tags:
- toxicity
- content-moderation
datasets:
- google/jigsaw_unintended_bias
---

# DistilBERT toxicity classifier (soft labels + identity weights)

Fine-tuned `distilbert-base-uncased` that predicts whether an English comment is toxic. Built for the capstone project
*Trust in Automated Moderation of Toxic Comments* (ECE 4424 / CS 4824). Code: https://github.com/martinagg7/ML-ToxicModeration

## Training

- Data: Jigsaw Unintended Bias in Toxicity Classification (`train.csv`, 1.8 M news comments), 80/10/10 split grouped by article.
- Soft labels: the model learns the share of annotators who marked a comment as toxic, not only 0/1.
- Identity weights: comments that mention an identity weigh more in the loss (2 if toxic, 3 if non-toxic).
- 2 epochs, learning rate 2e-5, batch size 16, max length 128 tokens. Epoch chosen on validation PR-AUC.

## Results (held-out test set, 177,635 comments, threshold 0.5)

| Model | Precision | Recall | F1 | ROC AUC | PR-AUC |
|---|---|---|---|---|---|
| Word blocklist (LDNOOBW) | 0.27 | 0.10 | 0.15 | 0.54 | 0.10 |
| TF-IDF + logistic regression | 0.73 | 0.53 | 0.61 | 0.93 | 0.69 |
| **This model** | **0.80** | **0.58** | **0.67** | **0.96** | **0.78** |

## Limitations

- Misses about 4 in 10 toxic comments at threshold 0.5.
- Still blocks more innocent comments when they mention an identity (for example 5.6% for gay/lesbian vs 1.3% overall).
- Fails on negations with insults ("You are not an idiot"), counter-speech and positive swearing.
- Evaluated only on English news comments from the same source as training.

Not intended for fully automatic moderation: uncertain predictions (probability 0.4–0.6) should go to human review.

## Usage

```python
from transformers import pipeline
clf = pipeline("text-classification", model="Martinagg/toxic-distilbert")
clf("You are an idiot")
```