roberta-ai-detector-v2

RoBERTa-based AI text detector fine-tuned for academic writing

Model Description

This model is fine-tuned to detect AI-generated text in academic papers and essays. It distinguishes between human-written and AI-generated content with high accuracy.

• Model type: roberta
• Language(s): EN
• License: Apache 2.0
• Fine-tuned from: roberta-base

Intended Use

This model is intended for:

• Detecting AI-generated content in academic submissions
• Research on AI text detection
• Educational tools for academic integrity

Important: This model should be used as one signal among many when evaluating text authenticity. It should not be the sole basis for academic misconduct decisions.

Performance

Metric	Score
Accuracy	99.04%
F1 Score	99.04%
ROC AUC	99.74%

Training Data

The model was trained on 56,213 samples of paired human and AI-generated academic text, including outputs from:

• Claude (Anthropic)
• GPT models (OpenAI)
• Gemini (Google)

Evaluation

Evaluated on 11,023 held-out test samples.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model
model_name = "coai/roberta-ai-detector-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Predict
text = "Your text to analyze..."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    ai_probability = probs[0][1].item()  # Probability of AI-generated

print(f"AI Probability: {ai_probability:.2%}")

Limitations

• Optimized for academic/formal writing; may be less accurate on casual text
• Performance may vary on text from AI models not in the training set
• Should not be used as the sole determinant of academic misconduct
• May have reduced accuracy on very short texts (<50 words)

Ethical Considerations

• False positives can have serious consequences for students
• Always use human judgment alongside model predictions
• Consider the context and provide opportunities for appeal
• This tool is meant to assist, not replace, human evaluation

Citation

If you use this model, please cite:

@misc{roberta_ai_detector_v2},
  author = {COAI},
  title = {roberta-ai-detector-v2: AI Text Detection Model},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/coai/roberta-ai-detector-v2}
}

Contact

For questions or issues, please open an issue on the model repository.