lilhuang15/bert-ai-text-detector-reddit

BERT AI-Text Detector (HC3 reddit_eli5)

bert-base-uncased fine-tuned to classify a text as human-written (label 0) or AI-generated (label 1). Trained on the HC3 reddit_eli5 subset (human vs ChatGPT/GPT-3.5 answers, 2023), 47.6K samples, class-weighted loss, max_length=256.

Evaluation	Macro-F1
Held-out reddit_eli5 test (n=10,200)	0.9935
finance - cross-domain, never trained on (n=8,436)	0.9812
medicine - cross-domain, never trained on (n=2,582)	0.9821

Known limitations

• Trained on 2023 GPT-3.5 text: newer generators are out of distribution - generator drift is the predicted failure axis, not topic shift.
• The model leans on a length/style prior: abnormally short AI text can evade it, and long, well-structured human writing is what it most often false-flags. Not suitable for high-stakes accusations.
• English only.

Full project - 5-model comparison, cross-domain study, 100%-coverage error audit, live demo: https://github.com/lilhuang15/ai-generated-text-detector