README.md

---
language: en
license: [mit]
tags:
  - text-classification
  - ai-generated-text-detection
  - deberta-v3
base_model: microsoft/deberta-v3-large
pipeline_tag: text-classification
---

<div align="center">

 <img src="assets/gradient.svg" alt="Gradient logo" width="128"/> 
 
 <h1> Gradient — AI-Generated Text Detector</h1>
</div>


This model is a fine-tuned version of [DeBERTa-v3-large](https://huggingface.co/microsoft/deberta-v3-large) for binary classification of human-written vs. AI-generated text. It outputs a single probability, P(AI), indicating the likelihood that a given input was generated by a language model.

## Model Details

- **Base model:** DeBERTa-v3-large
- **Architecture:** DeBERTa-v3-large with a single classification head (binary, sigmoid output)
- **Output:** A single scalar P(AI) in [0, 1]; a decision threshold of 0.5 is used by default, where P(AI) > 0.5 indicates AI-generated text
- **Language:** English
- **License:** MIT License


## Training Data

This model was trained on approximately 1.1 million texts from three datasets: DACTYL 2.0, LLMTrace, and MAGA-Bench. 
 

## Evaluation

The model was evaluated against two leading open-source AI text detectors, [Fakespot](https://huggingface.co/PLACEHOLDER) and [Desklib](https://huggingface.co/PLACEHOLDER), across ten benchmark datasets. Three of these (dactyl-v2.0, llm-trace-eng, maga) are in-distribution with respect to this model's training data; the remaining seven (beemo, coconuts, detectrl, dolly-cosmopedia, originalityai, realdet, uchicago) are out-of-distribution and were not seen during training.

All F1 scores are macro-averaged and computed at a decision threshold of P(AI) = 0.5.

### Results

| Dataset | AUROC | Macro-F1 |
|---|---|---|
| dactyl-v2.0 † | 0.9846 | 0.9651 |
| llm-trace-eng † | 0.9903 | 0.9674 |
| maga † | 0.9992 | 0.9900 |
| beemo | 0.8780 | 0.7312 |
| coconuts | 0.9819 | 0.8387 |
| detectrl | 0.9465 | 0.8756 |
| dolly-cosmopedia | 0.9952 | 0.9058 |
| originalityai | 0.9213 | 0.7248 |
| realdet | 0.9810 | 0.9417 |
| uchicago | 0.9817 | 0.8685 |

† In-distribution (training data overlap)

**Out-of-distribution averages:** AUROC 0.9551, Macro-F1 0.8409

### Comparison to Baselines (OOD average)

| Model | AUROC | Macro-F1 |
|---|---|---|
| Fakespot | 0.9315 | 0.8029 |
| Desklib | 0.9213 | 0.7837 |
| **DeBERTa-v3-large (this model)** | **0.9551** | **0.8409** |

<img src="assets/deberta_auroc_by_dataset.svg" width="1000">
<img src="assets/deberta_f1_by_dataset.svg" width="1000">

### Notes on Evaluation

- Fakespot and Desklib use a two-head softmax architecture and were evaluated using their native argmax decision rule, which is mathematically equivalent to thresholding P(AI) at 0.5.
- This model outperforms both baselines on most out-of-distribution datasets, with the exception of coconuts (Desklib) and originalityai (Fakespot), where the baselines hold an edge.
- In-distribution performance is substantially higher than out-of-distribution performance, which is expected and should be taken into account when interpreting the headline averages; OOD results are more representative of expected real-world generalization.



### Limitations and Out-of-Scope Use

This model should not be used as the sole basis for high-stakes decisions such as academic penalties or employment actions, given the false positive/negative rates documented below. Performance also degrades on text distributions not represented in training data; see evaluation results.

## Citation

```
@article{thorat2026panclef,
  title={Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection},
  author={Thorat, Shantanu},
  journal={Working Notes of CLEF},
  year={2026}
}
```