Argus v5: a machine-generated text detector (research release)

Argus v5 is the third Argus submission to the RAID benchmark, released with the paper "Argus: Narrowing the Held-Out-to-Hidden-Test Gap in Machine-Generated Text Detection" (S. V. Arumalla and R. Ramisetty, under review at IEEE Access). It is a ModernBERT-large encoder (28 layers, 395,833,346 parameters) with a two-class sequence-classification head, fine-tuned once with cross-entropy for one epoch on 512-token inputs. Label 1 is machine-generated text; the score of a document is the softmax probability of label 1.

Intended use

Research on the detection of machine-generated English text: reproducing the paper's results, scoring public benchmarks, and studying detector behavior. It is not intended for automated decisions about individuals. A score is evidence for human review, never a finding of misconduct on its own.

Required input canonicalization

The model was trained on canonicalized text, and scores are valid only on text canonicalized the same way. This repository holds the two functions that do it:

  • normalize.py, R1 (normalize_text): NFKC normalization, removal of zero-width and invisible formatting characters, mapping of cross-script confusable characters, typographic quotation marks and dashes to ASCII, and collapsing of whitespace.
  • confusables.py, R2 (normalize_text_r2): R1 followed by a wider ASCII-targeted confusable fold (further Cyrillic and Greek look-alikes, fullwidth and mathematical alphanumeric forms). It imports normalize_text from normalize.py.

Score a document as softmax(model(tokenize(normalize_text_r2(text), truncation=True, max_length=512)))[1]. The canonicalization is meant for English: it folds genuine Cyrillic and Greek look-alike letters to ASCII.

Training data

2,449,630 documents, 1,224,815 per class. Machine class by source: RAID training split, attacked 61.3% and unattacked 9.3%; text the authors generated with current-generation models through their APIs 15.1%; WildChat 6.8%; MAGE 3.4%; M4GT 2.4%; HC3 0.8%; Ghostbuster 0.6%; an essay corpus 0.3%. The human class adds HC3, paired essays, arXiv abstracts from before 2021, open-web prose collected before 2020 (FineWeb), RAID's human documents and learner essays (Write & Improve). RAID's generator labels gpt4, mistral-chat and cohere and its recipes domain were withheld for evaluation; related models (GPT-4 turns in WildChat, mistral, cohere-chat) were not.

Results

RAID hidden test, scored by the benchmark maintainers (leaderboard entry "Argus v5", 672,000 documents, per-domain thresholds fitted by the maintainers):

AUROC TPR at 1% FPR TPR at 5% FPR
All attacks 0.9313 94.47% 97.45%
Unattacked 0.9219 93.42%

The weakest generator label is cohere (72.61% TPR at 1% FPR) and the strongest attack paraphrase (92.02%). On MIRAGE (DIG setting, generate task), a benchmark not used in training, the AUROC was 0.9852 and the TPR at 1% FPR, at a threshold fitted on MIRAGE's own human excerpts, 75.65%, from 35.71% for Claude-3.5-Haiku to 92.52% for DeepSeek-V3.

Limitations

  • Thresholds must be fitted locally. The 99th percentile of human scores ranges from 0.023 to 0.9411 across the human sets the paper measured. Fit the threshold on known-human text from the population being assessed, canonicalized as above, and refit it when that text changes.
  • False positives on student essays. At thresholds fitted on other human text, the model flagged 3.67% to 8.92% of 1,200 learner essays, several times the 1% target. A small control of native students' essays from the same corpus was flagged at similar or higher rates, so the excess is not shown to be specific to non-native writers.
  • False positives on edited human text. Human text passed through word-level edits of the kind that paraphrasing, casing or spelling tools produce was flagged at 8.60% to 63.90%, against 0.60% to 1.20% of the unedited originals, at a threshold that flags about 1% of RAID's human text. This is consistent with a shortcut: such edits reached its training data only in the machine class.
  • Recall at a fixed operating point does not fully transfer. Ranking quality carried over to MIRAGE; recall at 1% FPR fell to 75.65%.
  • RAID's AUROC rank. Its AUROC ranked 29th of 38 entries in a RAID leaderboard snapshot of 2026-07-25, and its TPR at 1% FPR 12th. The AUROC rank reflects, at least in part, the shortcut above, because RAID's AUROC counts attacked human documents; its AUROC was 0.9817 to 0.9982 on the paper's development slices and 0.9852 on MIRAGE.
  • Attacks. Paraphrase remains the most effective RAID attack, and the model is not claimed to be robust to attacks designed with knowledge of it.
  • Scope. English only; inputs are truncated to 512 tokens.

License

The weights are released under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0). The base model, ModernBERT-large, is released under the Apache 2.0 license. The training data come from sources with their own terms.

Citation

@misc{arumalla2026argus,
  author = {Sai Vineeth Arumalla and Rupika Ramisetty},
  title  = {Argus: Narrowing the Held-Out-to-Hidden-Test Gap in Machine-Generated Text Detection},
  year   = {2026},
  note   = {Manuscript under review at IEEE Access}
}
Downloads last month
29
Safetensors
Model size
0.4B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for vineeth98/argus-v5

Finetuned
(437)
this model