stylometric-ai-detector

AI vs Human text detection using stylometric features and a Random Forest classifier.

Model Description

This model classifies text as AI-generated or human-written based on 8 stylometric features:

Feature	Description
char_count	Total number of characters
word_count	Total number of words
avg_word_len	Average word length
punct_count	Number of punctuation characters
sentence_count	Number of sentences
avg_sentence_len	Average sentence length (in words)
upper_case_count	Number of fully uppercase alphabetic words
title_case_count	Number of title-case words

Training

• Algorithm: Random Forest (100 estimators, random_state=42)
• Dataset: AI vs Human Text (~487k samples)
• Train/test split: 80/20 (389,788 train / 97,447 test)
• Accuracy: 96.03%

Classification Report

Class	Precision	Recall	F1-score	Support
Human (0)	0.96	0.98	0.97	61,112
AI (1)	0.97	0.92	0.95	36,335
Weighted avg	0.96	0.96	0.96	97,447

Usage

Install the Python package:

pip install stylometric-ai-detector

from stylometric_ai_detector import extract_stylometric_features, predict

# Extract features
features = extract_stylometric_features("Your text here...")

# Predict AI vs Human
result = predict(text="Your text here...")
# {"label": "AI", "probability": 0.87}

Or load the model directly:

import joblib
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="dinisds/stylometric-ai-detector",
    filename="random_forest_stylometric_model.joblib",
)
model = joblib.load(path)

Feature Order for Direct Inference

FEATURES = [
    "char_count", "word_count", "avg_word_len", "punct_count",
    "sentence_count", "avg_sentence_len", "upper_case_count", "title_case_count",
]
# model.predict([[char_count, word_count, avg_word_len, punct_count,
#                 sentence_count, avg_sentence_len, upper_case_count, title_case_count]])
# 0 = Human, 1 = AI

Package

This model is used by the stylometric-ai-detector Python package:

pip install stylometric-ai-detector

from stylometric_ai_detector import predict
result = predict(text="Your text here...")

Limitations

• Trained on a single English-language dataset; may not generalize to other languages or domains
• Stylometric patterns vary across AI models and versions
• Not a deep learning model — relies on surface-level text statistics

Citation

Dataset: Shanegerami's AI vs Human Text on Kaggle.

License

MIT