Rename to dialect-fair-hate-classifier (drop 'Guardian')
Browse files
README.md
CHANGED
|
@@ -15,11 +15,11 @@ metrics:
|
|
| 15 |
- recall
|
| 16 |
---
|
| 17 |
|
| 18 |
-
#
|
| 19 |
|
| 20 |
A binary hate-speech classifier fine-tuned from [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large), explicitly debiased against **African-American English (AAE) false positives**.
|
| 21 |
|
| 22 |
-
Off-the-shelf toxicity/hate classifiers flag benign AAE text as hateful at **2x+** the rate of benign General-American English (Sap et al. 2019).
|
| 23 |
|
| 24 |
Author: **Guy Grigsby** (Aeryx-ai). License: MIT.
|
| 25 |
|
|
@@ -45,8 +45,8 @@ The dialect FP gap is near-parity across the whole frontier (raw bias is ~5x low
|
|
| 45 |
import torch
|
| 46 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 47 |
|
| 48 |
-
tok = AutoTokenizer.from_pretrained("Aeryx-ai/
|
| 49 |
-
model = AutoModelForSequenceClassification.from_pretrained("Aeryx-ai/
|
| 50 |
|
| 51 |
def hate_prob(text):
|
| 52 |
enc = tok(text, return_tensors="pt", truncation=True, max_length=256)
|
|
@@ -74,11 +74,11 @@ Loss reweighting and DANN-style adversarial debiasing were tried; **data volume
|
|
| 74 |
## Citation
|
| 75 |
|
| 76 |
```
|
| 77 |
-
@misc{
|
| 78 |
-
title = {
|
| 79 |
author = {Grigsby, Guy},
|
| 80 |
year = {2026},
|
| 81 |
-
howpublished = {\url{https://huggingface.co/Aeryx-ai/
|
| 82 |
}
|
| 83 |
```
|
| 84 |
|
|
|
|
| 15 |
- recall
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# Dialect-Fair Hate Classifier (ModernBERT-large)
|
| 19 |
|
| 20 |
A binary hate-speech classifier fine-tuned from [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large), explicitly debiased against **African-American English (AAE) false positives**.
|
| 21 |
|
| 22 |
+
Off-the-shelf toxicity/hate classifiers flag benign AAE text as hateful at **2x+** the rate of benign General-American English (Sap et al. 2019). This classifier targets and measures that gap directly: it catches **93% of hate** while keeping the benign-AAE false-positive rate within **~0.04** of the benign-GAE rate.
|
| 23 |
|
| 24 |
Author: **Guy Grigsby** (Aeryx-ai). License: MIT.
|
| 25 |
|
|
|
|
| 45 |
import torch
|
| 46 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 47 |
|
| 48 |
+
tok = AutoTokenizer.from_pretrained("Aeryx-ai/dialect-fair-hate-classifier")
|
| 49 |
+
model = AutoModelForSequenceClassification.from_pretrained("Aeryx-ai/dialect-fair-hate-classifier").eval()
|
| 50 |
|
| 51 |
def hate_prob(text):
|
| 52 |
enc = tok(text, return_tensors="pt", truncation=True, max_length=256)
|
|
|
|
| 74 |
## Citation
|
| 75 |
|
| 76 |
```
|
| 77 |
+
@misc{grigsby2026dialectfair,
|
| 78 |
+
title = {A Dialect-Fair Hate Classifier},
|
| 79 |
author = {Grigsby, Guy},
|
| 80 |
year = {2026},
|
| 81 |
+
howpublished = {\url{https://huggingface.co/Aeryx-ai/dialect-fair-hate-classifier}}
|
| 82 |
}
|
| 83 |
```
|
| 84 |
|