Hugging Face's logo

protagonist
/
roberta-eloquent

Text Classification
Transformers
Safetensors
English
roberta
ai-text-detection
voight-kampff
pan-2025
text-embeddings-inference
Model card Files
xet
 Community
roberta-eloquent
File size: 1,955 Bytes
9445fad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
---
license: cc-by-4.0
library_name: transformers
base_model: roberta-base
tags:
- text-classification
- ai-text-detection
- voight-kampff
- pan-2025
datasets:
- pan-webis-de/pan25-generative-ai-detection-task1
language: en
pipeline_tag: text-classification
---

# eloquent26 RoBERTa-base detector — PAN'25/26 Voight-Kampff Subtask 1

Fine-tuned `roberta-base` on the official PAN'25/26 Generative AI Detection
training split (Zenodo DOI 10.5281/zenodo.14962653). Used in the eloquent26
detector panel for the ELOQUENT 2026 Voight-Kampff research paper.

## Training data
- Source: Bevendorff et al., PAN'25/26 Generative AI Detection: Voight-Kampff
  AI Detection Sensitivity (Zenodo, March 2025).
- Split: train.jsonl (n = 23,707; 9,101 human + 14,606 LLM-generated).
- Generator models in train: gpt-3.5-turbo, gpt-4o, gpt-4o-mini, o3-mini,
  gemini-1.5-pro, gemini-2.0-flash, llama-3.1-8b-instruct, llama-3.3-70b-instruct,
  ministral-8b-instruct-2410, deepseek-r1-distill-qwen-32b, falcon3-10b-instruct,
  gpt-4.5-preview, gpt-4-turbo-paraphrase, gemini-pro.
- Genres in train: essays, fiction, news.

## Training config
- Epochs: 2
- Batch size: 16
- Learning rate: 2e-05
- Weight decay: 0.01
- Warmup ratio: 0.1
- Max length: 512
- Seed: 42
- Mixed precision: fp16

## Final val metrics
```
{
  "eval_loss": 0.07029607146978378,
  "eval_accuracy": 0.9877403176372248,
  "eval_f1": 0.9905579399141631,
  "eval_roc_auc": 0.9994946525295825,
  "eval_runtime": 3.7238,
  "eval_samples_per_second": 963.793,
  "eval_steps_per_second": 30.345,
  "epoch": 2.0
}
```

## Usage

```python
from transformers import pipeline
clf = pipeline("text-classification", model="protagonist/roberta-eloquent", top_k=None)
print(clf("Your text here"))
```

Returned label `llm` carries the probability that the text was machine-generated.

## Reproduce

The training script lives at
[`notebooks/train_roberta_a100.py`](https://github.com/...) in the eloquent26 repo.