Update model card to v2.0 (calibrated logit margin; base = distilbert-base-cased)

c59bd84 verified 23 days ago

3.07 kB

license: mit
language: en
library_name: transformers
pipeline_tag: text-classification
tags:
  - ai-text-detection
  - academic-integrity
  - paperguard
  - distilbert
datasets:
  - Rajarshi-Roy-research/Defactify_Text_Dataset
  - Ateeqq/AI-and-Human-Generated-Text
  - artem9k/ai-text-detection-pile

PaperGuard AI Detector (v2.0)

Fine-tuned DistilBERT (distilbert-base-cased) sequence classifier that detects AI-generated text in academic papers, essays, and reports. It is the core AI-detection engine of the PaperGuard multi-agent academic-integrity system.

Labels

0 = ai, 1 = human (see config.json id2label).

Training data

v2.0 ("mega") continues from the v1.5 checkpoint and adds a larger, more diverse mix so the model sees many modern LLM writing styles:

Claude Opus distillation — recent Anthropic-style generations
Ateeqq/AI-and-Human-Generated-Text — academic abstracts (human + AI)
artem9k/ai-text-detection-pile — large open human-vs-AI corpus
(v1.5 lineage) Defactify + Ateeqq, spanning 30+ frontier LLMs (GPT-4o, LLaMA-3, Claude 3, Gemini, Mistral, Qwen, ChatGPT, …)

Important: use the logit margin, not the raw softmax

On easy/separable data the model becomes overconfident — its softmax saturates (it can report 0% AI even on genuine AI text). The discriminative signal lives in the logit margin (human_logit − ai_logit). PaperGuard therefore scores AI-likelihood from a logistic calibration of the margin, not the raw softmax. After calibration it flags clean/academic AI at ~70–90% while keeping human text low (10%).

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tok = AutoTokenizer.from_pretrained("vediumsameer/paperguard-ai-detector")
model = AutoModelForSequenceClassification.from_pretrained("vediumsameer/paperguard-ai-detector")

text = "The rapid advancement of artificial intelligence has transformed modern education..."
inputs = tok(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits[0]

# Recommended: score off the margin (softmax is saturated)
margin = float(logits[model.config.label2id["human"]] - logits[model.config.label2id["ai"]])
# lower margin  -> more AI-like ; higher margin -> more human-like
print("logit margin (human - ai):", margin)

Limitations

Blind spot: slang / style-masked AI (an LLM told to write casually) can still read as human. PaperGuard mitigates this with embedding-based stylometric "patchwork" detection, and a v2.1 trained on adversarial / multi-model / reasoning data is planned to close the gap.
AI detection is a probabilistic indicator, not proof of authorship.

Part of PaperGuard

This model powers the AI-detection layer of PaperGuard, which also does citation claim verification, plagiarism, and writing-quality analysis. See the project repo.