⚠️ DO NOT USE THIS MODEL ⚠️
This model failed its own evaluation. It is kept publicly for research transparency only.
It is an AI-generated-text ("slop") detector fine-tuned from LiquidAI/LFM2.5-Encoder-350M on a 786k-row corpus (v1 data mix). Despite near-perfect benchmark numbers, it learned the wrong thing.
Measured results (proper eval, final weights)
| slice | AUROC | verdict |
|---|---|---|
| blogs/gutenberg/writingprompts vs pile | 1.00 | era+length shortcut, meaningless |
| same vs storyscope | 1.00 | register shortcut, meaningless |
| coai strict (paired, style-isolated) | 0.50 | coin flip — worse than a 415-pattern regex floor (0.8643) |
| heldout laguna | 0.756 | weak |
| heldout local | 0.861 | weak |
| heldout deepseek_eval | 0.752 | weak |
What went wrong
The training mix was 72% The-Pile with a length confound (AI rows avg 2221 chars, human rows ≤984). The model became an era-and-register classifier: it detects "old-fashioned or short prose vs modern long web text," not slop. Mid-training it scored 1.00 on coai's validation slice; by end of training catastrophic forgetting dropped it to chance while the easy registers absorbed all capacity.
Mid-training val AUROC read 0.9998. That number was a lie told by an unstratified metric. Trust nothing without the paired-slice eval above.
Why it's still public
Negative results are results. If you're building an LLM-text detector, this is what register-confounded training data produces: spectacular metrics, coin-flip reality. See the eval harness requirements: paired same-topic slices, unseen-generator holdouts, and a regex baseline you must beat.
- training data: vstalingrady/itais (
train_all.parquet, pile-heavy v1 mix) - eval report shape: cross-register matrix + strict-paired slice + held-out files
Do not use for content moderation, academic integrity, or anything else.