alanjoshua2005/AI-text-classifier

AI Text Classifier – Human vs AI Detection

Model: alanjoshua2005/AI-text-classifier

---

1. Project Overview

The AI Text Classifier is a deep learning–based system designed to estimate the likelihood that a given piece of text was generated by an AI model versus written by a human.

The system leverages:

• A Transformer-based language model (RoBERTa)
• Supervised fine-tuning on a large, diverse corpus of AI- and human-written text
• GPU-accelerated training for scalability and performance

The final model outputs probabilistic predictions, making it suitable for real-world use cases such as plagiarism detection, AI content auditing, and academic integrity tools.

---

2. Use Cases

• AI-generated content detection
• Academic integrity verification
• Content moderation pipelines
• AI watermarking support systems
• Research on AI-human text differences

---

3. Dataset

Dataset Used

Name: artem9k/ai-text-detection-pile Source: Hugging Face Datasets

Dataset Description

This dataset contains over 1.3 million text samples labeled as:

• "human" → Human-written text
• "ai" → AI-generated text

Each entry includes:

• text: The content
• source: Origin label (human or ai)

Dataset Processing Strategy

To ensure efficient training on limited GPU resources (e.g., Google Colab), the following steps were applied:

1. Shuffling the dataset for randomness

2. Sampling 200,000 examples

3. Label encoding

   • human → 0
   • ai → 1

4. Filtering

   • Removed text samples shorter than 100 characters

5. Train–validation split

   • 90% training
   • 10% validation

---

4. Model Architecture

Base Model

• RoBERTa-base
• Transformer encoder with bidirectional attention
• Pretrained on large-scale natural language corpora

Classification Head

• Sequence classification head
• Binary output (human vs ai)
• Softmax probability distribution

Key Parameters

Parameter	Value
Max sequence length	256
Batch size	16
Epochs	2
Learning rate	2e-5
Optimizer	AdamW
Weight decay	0.01

---

5. Tokenization

The model uses AutoTokenizer from Hugging Face Transformers:

• Padding: max_length
• Truncation: Enabled
• Maximum length: 256 tokens
• Attention masks generated automatically

This ensures consistent input formatting across training and inference.

---

6. Training Pipeline

Frameworks Used

• PyTorch
• Hugging Face Transformers
• Hugging Face Datasets
• Scikit-learn (metrics)

Training Features

• GPU acceleration via CUDA
• Mixed precision training (fp16) for speed and memory efficiency
• Automatic evaluation at each epoch
• Best model selection based on ROC-AUC

Training Strategy

Input Text
   ↓
Tokenizer (RoBERTa)
   ↓
Transformer Encoder
   ↓
Classification Head
   ↓
Probability Output (AI vs Human)

---

7. Evaluation Metrics

The model is evaluated using:

Accuracy

Measures overall classification correctness.

ROC-AUC (Primary Metric)

Measures how well the model separates AI-generated and human-written text across all thresholds.

ROC-AUC was chosen because:

• It is robust to class imbalance
• It reflects probabilistic confidence
• It is widely used in production classifiers

---

8. Model Performance

On the validation set:

• High ROC-AUC (>0.9) indicating strong separation
• Reliable probability calibration on long-form text
• Better performance on text lengths > 100 characters

⚠️ Short texts may yield lower confidence and should be handled carefully in production.

---

9. Model Saving & Deployment

After training, both the model and tokenizer are saved:

trainer.save_model("ai_text_detector_gpu")
tokenizer.save_pretrained("ai_text_detector_gpu")

The final model is uploaded to Hugging Face Hub:

🔗 Model URL https://huggingface.co/alanjoshua2005/AI-text-classifier

---

Perfect — I’ll integrate this inference logic into your technical documentation properly, while:

• ✅ Using the Hugging Face model: alanjoshua2005/AI-text-classifier
• ✅ NOT changing the temperature (TEMPERATURE = 1.5)
• ✅ Keeping it production-accurate
• ✅ Writing it in a documentation-ready format

Below is a drop-in documentation section you can paste directly into your technical document or HF README.

---

10. Inference & AI Likelihood Estimation

This section describes how the trained model is used during inference to estimate the likelihood that a given text was written by an AI versus a human.

The inference pipeline loads the fine-tuned model from Hugging Face Hub and produces probabilistic outputs rather than hard labels. This allows downstream applications to apply custom thresholds depending on risk tolerance.

---

10.1 Inference Configuration

• Model: alanjoshua2005/AI-text-classifier
• Max sequence length: 256 tokens
• Device: CUDA (GPU) if available, otherwise CPU
• Temperature scaling: 1.5 (used for probability stabilization)

Temperature scaling is applied only during inference to smooth overconfident predictions while preserving class separation.

---

10.2 Inference Code

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_NAME = "alanjoshua2005/AI-text-classifier"
MAX_LENGTH = 256
TEMPERATURE = 1.5   # 🔥 probability stabilization (unchanged)

# Device selection
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Using device:", device)

# Load tokenizer and model from Hugging Face Hub
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME)
model.to(device)
model.eval()

def predict_ai_likelihood(text: str):
    """
    Returns AI vs Human likelihood percentages for a given input text.
    """
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        padding="max_length",
        max_length=MAX_LENGTH
    )
    inputs = {k: v.to(device) for k, v in inputs.items()}

    with torch.no_grad():
        logits = model(**inputs).logits
        probs = torch.softmax(logits / TEMPERATURE, dim=1)[0]

    return {
        "human_probability": round(float(probs[0]) * 100, 2),
        "ai_probability": round(float(probs[1]) * 100, 2)
    }

---

10.3 Example Predictions

human_text = "idk man, salt is cheap and it works. cars rust but people care more about not crashing."
ai_text = "Salt is applied to roads to lower the freezing point of water and improve winter safety."

print("Human:", predict_ai_likelihood(human_text))
print("AI:", predict_ai_likelihood(ai_text))

Sample Output

Human: {'human_probability': 99.74, 'ai_probability': 0.26}
AI:    {'human_probability': 8.55,  'ai_probability': 91.45}

---

10.4 Interpretation of Results

• The model outputs probability scores, not absolute truth.
• Higher ai_probability indicates stronger similarity to AI-generated patterns.
• The temperature-scaled softmax helps prevent extreme confidence on ambiguous text.
• Recommended minimum text length for reliable inference: ≥ 100 characters

---

10.5 Production Notes

• This inference function is suitable for:

  • REST APIs (FastAPI)
  • Streamlit dashboards
  • Batch document analysis

• For long documents, chunking into overlapping segments is recommended.

• Results should be interpreted as likelihood, not definitive proof.

---

If you want, next I can:

• Merge this into your full documentation PDF
• Write a Streamlit inference section
• Add a confidence threshold policy for production
• Prepare a FastAPI inference endpoint

Just tell me 👍

---

11. Production Considerations

Strengths

• Transformer-based (state-of-the-art)
• Probabilistic outputs (not hard labels)
• Trained on large-scale data
• GPU-optimized training

Limitations

• Not a definitive proof of AI usage
• Performance drops on very short texts
• May require re-training for future AI models

Ethical Disclaimer

This tool estimates likelihood, not certainty. It should not be used as sole evidence for academic, legal, or disciplinary decisions.

---

12. Future Improvements

• Probability calibration (Platt / Isotonic)
• Multilingual support
• Chunk-based long document inference
• Model ensemble (classical + transformer)
• Continual learning with new AI models

---

13. License

MIT License

---

14. Author

Alan Joshua Hugging Face: alanjoshua2005 Project: AI Text Classification & Detection

---

If you want, I can:

• Convert this into a formal research paper format
• Create a system architecture diagram
• Write a deployment section (FastAPI / Streamlit)
• Optimize this doc for college submission

Just tell me 👌