README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
language:
- en
base_model: datalama/mmBERT-small
datasets:
- google/civil_comments
- Heliosoph/Jigsaw-Toxic-Comments
tags:
- toxicity
- content-moderation
- multi-label-classification
- modernbert
- multilingual
metrics:
- roc_auc
- f1
---

# Multilabel Toxic Comment Classifier

This model produces seven independent moderation scores for English comments:
`toxicity`, `severe_toxicity`, `obscene`, `threat`, `insult`,
`identity_attack`, and `sexual_explicit`. It is a fine-tune of
[`datalama/mmBERT-small`](https://huggingface.co/datalama/mmBERT-small), a
multilingual ModernBERT/mmBERT base model.

This is a multi-label classifier, not a mutually exclusive class classifier.
Every output is a sigmoid score and a comment can score highly on multiple
dimensions. `toxicity` is trained as its own target; it is not computed as an
OR of the other labels.

## Intended use

Use the scores as one input to a broader content-moderation workflow. Choose
and validate a separate decision threshold for every label and review decisions
in context. Do not use this model as the sole basis for high-impact decisions
about people.

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "wagesj45/multilabel-toxic-comment-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

inputs = tokenizer(
    "Your text to score goes here.",
    return_tensors="pt",
    truncation=True,
    max_length=256,
)
with torch.no_grad():
    scores = torch.sigmoid(model(**inputs).logits[0]).tolist()

result = {
    model.config.id2label.get(index, model.config.id2label.get(str(index))): score
    for index, score in enumerate(scores)
}
print(result)
```

The model's saved label order is:

1. `toxicity`
2. `severe_toxicity`
3. `obscene`
4. `threat`
5. `insult`
6. `identity_attack`
7. `sexual_explicit`

Scores are model outputs, not calibrated probabilities. Thresholds should be
chosen for the target application and monitored after deployment.

## Training

The model was fine-tuned with `transformers` on
[`google/civil_comments`](https://huggingface.co/datasets/google/civil_comments)
and
[`Heliosoph/Jigsaw-Toxic-Comments`](https://huggingface.co/datasets/Heliosoph/Jigsaw-Toxic-Comments).
Civil Comments supplies all seven continuous annotation-fraction targets.
Jigsaw supplies binary targets for the first six dimensions; its unavailable
`sexual_explicit` label is masked from loss rather than treated as negative.

The combined corpus was shuffled and split deterministically 90/10 with seed
42. Training used three epochs, a learning rate of `2e-5`, batch sizes 16/32,
weight decay `0.01`, and a 256-token input limit. The exported weights are the
best checkpoint, selected by validation macro ROC-AUC at epoch 2.

## Evaluation

Evaluation used the held-out split described above. Metrics threshold soft
annotation targets at 0.5 for F1; ROC-AUC uses the continuous model scores.

| Label | ROC-AUC | F1 at 0.5 |
| --- | ---: | ---: |
| toxicity | 0.9744 | 0.6992 |
| severe_toxicity | 0.9991 | 0.4085 |
| obscene | 0.9936 | 0.7141 |
| threat | 0.9894 | 0.4847 |
| insult | 0.9802 | 0.6959 |
| identity_attack | 0.9873 | 0.3902 |
| sexual_explicit | 0.9967 | 0.5075 |
| **Macro average** | **0.9887** | **0.5571** |

These results are not a measure of performance on arbitrary production
comments or languages outside the evaluation data.

## Limitations and risks

- The training and evaluation comments are English. The multilingual base model
  does not establish validated performance for non-English text.
- Toxicity labels are subjective and noisy, and data may carry historical or
  cultural biases.
- Profanity, reclaimed language, identity terms, quotations, discussions of
  abuse, and strongly worded criticism can cause false positives.
- The model can miss implicit, coded, contextual, or adversarially written
  abuse.
- Use application-specific thresholds, appeals, human review, and monitoring
  for real moderation systems.

## Licensing and provenance

The fine-tuned model artifacts in this repository are released under the
[Apache License 2.0](LICENSE). The listed base model and both listed Hugging
Face training datasets use Apache-2.0 and CC0-1.0 licensing, respectively;
refer to their dataset cards for source attribution and terms. This repository
does not redistribute training rows. Any future hard-negative data must be
documented with its source and license before use in a released model.