File size: 3,964 Bytes
ed76bbe c9f71fd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | ---
library_name: transformers
tags: []
---
# DACTYL Text Detector
## Training Configuration
```json
{
"training_split": "training",
"evaluation_split": "testing",
"results_path": "bce-tinybert-base.csv",
"num_epochs": 5,
"model_path": "huawei-noah/TinyBERT_General_4L_312D",
"tokenizer": "huawei-noah/TinyBERT_General_4L_312D",
"optimizer": "AdamW",
"optimizer_type": "torch",
"optimizer_args": {
"lr": 2e-05,
"weight_decay": 0.01
},
"loss_fn": "BCEWithLogitsLoss",
"reset_classification_head": false,
"loss_type": "torch",
"loss_fn_args": {},
"needs_loss_fn_as_parameter": false,
"save_path": "ShantanuT01/dactyl-tinybert-pretrained",
"training_args": {
"batch_size": 64,
"needs_sampler": false,
"needs_index": false,
"shuffle": true,
"sampling_rate": null,
"apply_sigmoid": false
},
"best_model_path": "best-tinybert-model"
}
```
## Results
| model | AP Score | AUC Score | OPAUC Score | TPAUC Score |
|:--------------------------------------------------------------------------------------|-----------:|------------:|--------------:|--------------:|
| DeepSeek-V3 | 0.989355 | 0.996855 | 0.993696 | 0.938917 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-RedditWritingPrompts-testing | 0.0371425 | 0.749778 | 0.641038 | 0 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-abstracts-testing | 0.157916 | 0.898819 | 0.654391 | 0 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-news-testing | 0.00768129 | 0.591508 | 0.517311 | 0 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-reviews-testing | 0.0827563 | 0.921267 | 0.7135 | 0 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-student_essays-testing | 0.00826 | 0.580281 | 0.506719 | 0 |
| ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-tweets-testing | 0.456347 | 0.976401 | 0.86447 | 0.105054 |
| claude-3-5-haiku-20241022 | 0.973562 | 0.995183 | 0.982616 | 0.830874 |
| claude-3-5-sonnet-20241022 | 0.985813 | 0.99776 | 0.990951 | 0.912414 |
| gemini-1.5-flash | 0.959554 | 0.993995 | 0.973325 | 0.741471 |
| gemini-1.5-pro | 0.906423 | 0.981521 | 0.940218 | 0.434661 |
| gpt-4o-2024-11-20 | 0.971624 | 0.995329 | 0.981673 | 0.82246 |
| gpt-4o-mini | 0.99553 | 0.999309 | 0.997378 | 0.974682 |
| llama-3.2-90b | 0.929239 | 0.98677 | 0.953935 | 0.556386 |
| llama-3.3-70b | 0.96513 | 0.993237 | 0.976893 | 0.775907 |
| mistral-large-latest | 0.988606 | 0.997797 | 0.993021 | 0.932415 |
| mistral-small-latest | 0.987975 | 0.996807 | 0.992217 | 0.924494 |
| overall | 0.97755 | 0.980085 | 0.953644 | 0.550035 |
|