| --- |
| license: mit |
| --- |
| |
| ## AI Text Classifier – Human vs AI Detection |
|
|
| **Model:** `alanjoshua2005/AI-text-classifier` |
|
|
| --- |
|
|
| ## 1. Project Overview |
|
|
| The **AI Text Classifier** is a deep learning–based system designed to estimate the likelihood that a given piece of text was generated by an **AI model** versus written by a **human**. |
|
|
| The system leverages: |
|
|
| * A **Transformer-based language model (RoBERTa)** |
| * Supervised fine-tuning on a large, diverse corpus of AI- and human-written text |
| * GPU-accelerated training for scalability and performance |
|
|
| The final model outputs **probabilistic predictions**, making it suitable for real-world use cases such as plagiarism detection, AI content auditing, and academic integrity tools. |
|
|
| --- |
|
|
| ## 2. Use Cases |
|
|
| * AI-generated content detection |
| * Academic integrity verification |
| * Content moderation pipelines |
| * AI watermarking support systems |
| * Research on AI-human text differences |
|
|
| --- |
|
|
| ## 3. Dataset |
|
|
| ### Dataset Used |
|
|
| **Name:** `artem9k/ai-text-detection-pile` |
| **Source:** Hugging Face Datasets |
|
|
| ### Dataset Description |
|
|
| This dataset contains over **1.3 million text samples** labeled as: |
|
|
| * `"human"` → Human-written text |
| * `"ai"` → AI-generated text |
|
|
| Each entry includes: |
|
|
| * `text`: The content |
| * `source`: Origin label (`human` or `ai`) |
|
|
| ### Dataset Processing Strategy |
|
|
| To ensure efficient training on limited GPU resources (e.g., Google Colab), the following steps were applied: |
|
|
| 1. **Shuffling** the dataset for randomness |
| 2. **Sampling** 200,000 examples |
| 3. **Label encoding** |
|
|
| * `human → 0` |
| * `ai → 1` |
| 4. **Filtering** |
|
|
| * Removed text samples shorter than 100 characters |
| 5. **Train–validation split** |
|
|
| * 90% training |
| * 10% validation |
|
|
| --- |
|
|
| ## 4. Model Architecture |
|
|
| ### Base Model |
|
|
| * **RoBERTa-base** |
| * Transformer encoder with bidirectional attention |
| * Pretrained on large-scale natural language corpora |
|
|
| ### Classification Head |
|
|
| * Sequence classification head |
| * Binary output (`human` vs `ai`) |
| * Softmax probability distribution |
|
|
| ### Key Parameters |
|
|
| | Parameter | Value | |
| | ------------------- | ----- | |
| | Max sequence length | 256 | |
| | Batch size | 16 | |
| | Epochs | 2 | |
| | Learning rate | 2e-5 | |
| | Optimizer | AdamW | |
| | Weight decay | 0.01 | |
|
|
| --- |
|
|
| ## 5. Tokenization |
|
|
| The model uses `AutoTokenizer` from Hugging Face Transformers: |
|
|
| * Padding: `max_length` |
| * Truncation: Enabled |
| * Maximum length: 256 tokens |
| * Attention masks generated automatically |
|
|
| This ensures consistent input formatting across training and inference. |
|
|
| --- |
|
|
| ## 6. Training Pipeline |
|
|
| ### Frameworks Used |
|
|
| * PyTorch |
| * Hugging Face Transformers |
| * Hugging Face Datasets |
| * Scikit-learn (metrics) |
|
|
| ### Training Features |
|
|
| * **GPU acceleration** via CUDA |
| * **Mixed precision training (fp16)** for speed and memory efficiency |
| * Automatic evaluation at each epoch |
| * Best model selection based on **ROC-AUC** |
|
|
| ### Training Strategy |
|
|
| ```text |
| Input Text |
| ↓ |
| Tokenizer (RoBERTa) |
| ↓ |
| Transformer Encoder |
| ↓ |
| Classification Head |
| ↓ |
| Probability Output (AI vs Human) |
| ``` |
|
|
| --- |
|
|
| ## 7. Evaluation Metrics |
|
|
| The model is evaluated using: |
|
|
| ### Accuracy |
|
|
| Measures overall classification correctness. |
|
|
| ### ROC-AUC (Primary Metric) |
|
|
| Measures how well the model separates AI-generated and human-written text across all thresholds. |
|
|
| ROC-AUC was chosen because: |
|
|
| * It is robust to class imbalance |
| * It reflects probabilistic confidence |
| * It is widely used in production classifiers |
|
|
| --- |
|
|
| ## 8. Model Performance |
|
|
| On the validation set: |
|
|
| * **High ROC-AUC (>0.9)** indicating strong separation |
| * Reliable probability calibration on long-form text |
| * Better performance on text lengths > 100 characters |
|
|
| ⚠️ Short texts may yield lower confidence and should be handled carefully in production. |
|
|
| --- |
|
|
| ## 9. Model Saving & Deployment |
|
|
| After training, both the model and tokenizer are saved: |
|
|
| ```python |
| trainer.save_model("ai_text_detector_gpu") |
| tokenizer.save_pretrained("ai_text_detector_gpu") |
| ``` |
|
|
| The final model is uploaded to Hugging Face Hub: |
|
|
| 🔗 **Model URL** |
| [https://huggingface.co/alanjoshua2005/AI-text-classifier](https://huggingface.co/alanjoshua2005/AI-text-classifier) |
|
|
| --- |
|
|
| Perfect — I’ll **integrate this inference logic into your technical documentation properly**, while: |
|
|
| * ✅ Using the **Hugging Face model**: `alanjoshua2005/AI-text-classifier` |
| * ✅ **NOT changing the temperature** (`TEMPERATURE = 1.5`) |
| * ✅ Keeping it **production-accurate** |
| * ✅ Writing it in a **documentation-ready format** |
|
|
| Below is a **drop-in documentation section** you can paste directly into your technical document or HF README. |
|
|
| --- |
|
|
| ## 10. Inference & AI Likelihood Estimation |
|
|
| This section describes how the trained model is used during **inference** to estimate the likelihood that a given text was written by an AI versus a human. |
|
|
| The inference pipeline loads the fine-tuned model from **Hugging Face Hub** and produces **probabilistic outputs** rather than hard labels. This allows downstream applications to apply custom thresholds depending on risk tolerance. |
|
|
| --- |
|
|
| ### 10.1 Inference Configuration |
|
|
| * **Model:** `alanjoshua2005/AI-text-classifier` |
| * **Max sequence length:** 256 tokens |
| * **Device:** CUDA (GPU) if available, otherwise CPU |
| * **Temperature scaling:** `1.5` (used for probability stabilization) |
|
|
| Temperature scaling is applied **only during inference** to smooth overconfident predictions while preserving class separation. |
|
|
| --- |
|
|
| ### 10.2 Inference Code |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| |
| MODEL_NAME = "alanjoshua2005/AI-text-classifier" |
| MAX_LENGTH = 256 |
| TEMPERATURE = 1.5 # 🔥 probability stabilization (unchanged) |
| |
| # Device selection |
| device = torch.device("cuda" if torch.cuda.is_available() else "cpu") |
| print("Using device:", device) |
| |
| # Load tokenizer and model from Hugging Face Hub |
| tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) |
| model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME) |
| model.to(device) |
| model.eval() |
| |
| def predict_ai_likelihood(text: str): |
| """ |
| Returns AI vs Human likelihood percentages for a given input text. |
| """ |
| inputs = tokenizer( |
| text, |
| return_tensors="pt", |
| truncation=True, |
| padding="max_length", |
| max_length=MAX_LENGTH |
| ) |
| inputs = {k: v.to(device) for k, v in inputs.items()} |
| |
| with torch.no_grad(): |
| logits = model(**inputs).logits |
| probs = torch.softmax(logits / TEMPERATURE, dim=1)[0] |
| |
| return { |
| "human_probability": round(float(probs[0]) * 100, 2), |
| "ai_probability": round(float(probs[1]) * 100, 2) |
| } |
| ``` |
|
|
| --- |
|
|
| ### 10.3 Example Predictions |
|
|
| ```python |
| human_text = "idk man, salt is cheap and it works. cars rust but people care more about not crashing." |
| ai_text = "Salt is applied to roads to lower the freezing point of water and improve winter safety." |
| |
| print("Human:", predict_ai_likelihood(human_text)) |
| print("AI:", predict_ai_likelihood(ai_text)) |
| ``` |
|
|
| #### Sample Output |
|
|
| ```text |
| Human: {'human_probability': 99.74, 'ai_probability': 0.26} |
| AI: {'human_probability': 8.55, 'ai_probability': 91.45} |
| ``` |
|
|
| --- |
|
|
| ### 10.4 Interpretation of Results |
|
|
| * The model outputs **probability scores**, not absolute truth. |
| * Higher `ai_probability` indicates stronger similarity to AI-generated patterns. |
| * The temperature-scaled softmax helps prevent extreme confidence on ambiguous text. |
| * Recommended minimum text length for reliable inference: **≥ 100 characters** |
|
|
| --- |
|
|
| ### 10.5 Production Notes |
|
|
| * This inference function is suitable for: |
|
|
| * REST APIs (FastAPI) |
| * Streamlit dashboards |
| * Batch document analysis |
| * For long documents, chunking into overlapping segments is recommended. |
| * Results should be interpreted as **likelihood**, not definitive proof. |
|
|
| --- |
|
|
| If you want, next I can: |
|
|
| * Merge this into your **full documentation PDF** |
| * Write a **Streamlit inference section** |
| * Add a **confidence threshold policy** for production |
| * Prepare a **FastAPI inference endpoint** |
|
|
| Just tell me 👍 |
|
|
|
|
| --- |
|
|
| ## 11. Production Considerations |
|
|
| ### Strengths |
|
|
| * Transformer-based (state-of-the-art) |
| * Probabilistic outputs (not hard labels) |
| * Trained on large-scale data |
| * GPU-optimized training |
|
|
| ### Limitations |
|
|
| * Not a definitive proof of AI usage |
| * Performance drops on very short texts |
| * May require re-training for future AI models |
|
|
| ### Ethical Disclaimer |
|
|
| This tool **estimates likelihood**, not certainty. |
| It should **not** be used as sole evidence for academic, legal, or disciplinary decisions. |
|
|
| --- |
|
|
| ## 12. Future Improvements |
|
|
| * Probability calibration (Platt / Isotonic) |
| * Multilingual support |
| * Chunk-based long document inference |
| * Model ensemble (classical + transformer) |
| * Continual learning with new AI models |
|
|
| --- |
|
|
| ## 13. License |
|
|
| **MIT License** |
|
|
| --- |
|
|
| ## 14. Author |
|
|
| **Alan Joshua** |
| Hugging Face: `alanjoshua2005` |
| Project: AI Text Classification & Detection |
|
|
| --- |
|
|
| If you want, I can: |
|
|
| * Convert this into a **formal research paper format** |
| * Create a **system architecture diagram** |
| * Write a **deployment section (FastAPI / Streamlit)** |
| * Optimize this doc for **college submission** |
|
|
| Just tell me 👌 |
|
|
