Fediverse Bot Detector

A logistic regression classifier that estimates whether a Mastodon / fediverse account is a bot. It uses the account's profile metadata and the text of its bio.

Inputs

Each account is a dict with:

field description
username account handle (without the domain)
note profile bio as returned by the Mastodon API (HTML is stripped)
statuses_count number of posts
followers_count number of followers
following_count number of accounts followed
language (optional) ISO code of the bio language. If omitted, it is detected with fastText.

Features (794 total)

  1. 9 numeric features, standardized with the bundled scaler.joblib: statuses_count, followers_count, following_count, username length, digit/letter/special-character counts, and whether the username starts or ends with a digit.
  2. 17 language features: a one-hot over the 15 most frequent bio languages in training (language_vocab.json), plus unknown (empty bio) and other. The language is detected with fastText lid.176.
  3. 768 text features: the [CLS] embedding of the cleaned bio from bert-base-multilingual-cased, truncated to 128 tokens.

Usage

pip install -r requirements.txt
import sys
from huggingface_hub import snapshot_download

path = snapshot_download("FranUPM/fediverse-bot-detector")
sys.path.insert(0, path)
from predict import BotClassifier

clf = BotClassifier()              # downloads mBERT and fastText lid.176 on first use
clf.predict([{
    "username": "news_bot_2024",
    "note": "<p>Automated headlines posted every hour.</p>",
    "statuses_count": 52000,
    "followers_count": 120,
    "following_count": 0,
}])
# [{'username': 'news_bot_2024', 'prediction': True, 'probability': 0.999...}]

prediction is probability >= 0.5. Pass BotClassifier(threshold=...) to change the cutoff.

Files

file content
logistic_regression_model.joblib trained sklearn.linear_model.LogisticRegression (scikit-learn 1.7.1)
scaler.joblib StandardScaler for the 9 numeric features
language_vocab.json language one-hot vocabulary
bot_features.py feature engineering
predict.py BotClassifier inference wrapper

Load the .joblib files with scikit-learn 1.7.1. Other versions may warn or behave differently. Only load joblib/pickle files from sources you trust.

Training data

The model was trained on labelled accounts from these sources. No training data is included in this repository.

  • Bot Into The Fediverse Dataset: F. Moreno García, Universidad Politécnica de Madrid, 2025. Mastodon accounts collected in January–February 2025, CC-BY-4.0. doi:10.5281/zenodo.17987595. Companion paper: doi:10.1007/s13278-025-01567-z.
  • Cresci-2017: S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi. The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race. WWW '17 Companion, 2017. doi:10.1145/3041021.3055135. Labelled Twitter accounts, distributed for academic research only.

Training set: 44,730 accounts, balanced to 22,365 bots and 22,365 humans. They were split 80/20 (stratified, random_state=42) into 35,784 training and 8,946 test accounts. The scaler and the language vocabulary were fit on the training split only. The model is LogisticRegression(C=1.0, max_iter=2000).

Evaluation

Held-out test split (8,946 accounts, 50% bots), threshold 0.5:

metric value
ROC AUC 0.963
Accuracy 0.910
Precision (bot) 0.910
Recall (bot) 0.910
F1 (bot) 0.910

Confusion matrix: 4,069 true negatives, 404 false positives, 404 false negatives, 4,069 true positives.

Limitations

  • The test set is balanced (50% bots). On real instances bots are usually much rarer, so expect lower precision and consider raising threshold.
  • Part of the training data (Cresci-2017) comes from Twitter, whose account conventions differ from the fediverse.
  • Because Cresci-2017 is restricted to academic research, this model is intended for research use.
  • The model was trained on a specific snapshot of accounts. Bot behaviour and instance conventions change over time.
  • It only looks at profile metadata and the bio, not post content or posting times.
  • Bios in languages outside the top 15 are grouped together as other.
  • Use it as a signal for moderation or research, not as the only basis for actions against an account.

Third-party components (not redistributed here)

Citation

If you use this model, please cite the dataset and paper:

@dataset{moreno_garcia_2025_botfediverse,
  author    = {Moreno García, Francisco},
  title     = {Bot Into The Fediverse Dataset},
  year      = {2025},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.17987595}
}
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support