Instructions to use FranUPM/fediverse-bot-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Scikit-learn
How to use FranUPM/fediverse-bot-detector with Scikit-learn:
from huggingface_hub import hf_hub_download import joblib model = joblib.load( hf_hub_download("FranUPM/fediverse-bot-detector", "sklearn_model.joblib") ) # only load pickle files from sources you trust # read more about it here https://skops.readthedocs.io/en/stable/persistence.html - Notebooks
- Google Colab
- Kaggle
Fediverse Bot Detector
A logistic regression classifier that estimates whether a Mastodon / fediverse account is a bot. It uses the account's profile metadata and the text of its bio.
Inputs
Each account is a dict with:
| field | description |
|---|---|
username |
account handle (without the domain) |
note |
profile bio as returned by the Mastodon API (HTML is stripped) |
statuses_count |
number of posts |
followers_count |
number of followers |
following_count |
number of accounts followed |
language (optional) |
ISO code of the bio language. If omitted, it is detected with fastText. |
Features (794 total)
- 9 numeric features, standardized with the bundled
scaler.joblib:statuses_count,followers_count,following_count, username length, digit/letter/special-character counts, and whether the username starts or ends with a digit. - 17 language features: a one-hot over the 15 most frequent bio languages in training (
language_vocab.json), plusunknown(empty bio) andother. The language is detected with fastTextlid.176. - 768 text features: the
[CLS]embedding of the cleaned bio frombert-base-multilingual-cased, truncated to 128 tokens.
Usage
pip install -r requirements.txt
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("FranUPM/fediverse-bot-detector")
sys.path.insert(0, path)
from predict import BotClassifier
clf = BotClassifier() # downloads mBERT and fastText lid.176 on first use
clf.predict([{
"username": "news_bot_2024",
"note": "<p>Automated headlines posted every hour.</p>",
"statuses_count": 52000,
"followers_count": 120,
"following_count": 0,
}])
# [{'username': 'news_bot_2024', 'prediction': True, 'probability': 0.999...}]
prediction is probability >= 0.5. Pass BotClassifier(threshold=...) to change the cutoff.
Files
| file | content |
|---|---|
logistic_regression_model.joblib |
trained sklearn.linear_model.LogisticRegression (scikit-learn 1.7.1) |
scaler.joblib |
StandardScaler for the 9 numeric features |
language_vocab.json |
language one-hot vocabulary |
bot_features.py |
feature engineering |
predict.py |
BotClassifier inference wrapper |
Load the
.joblibfiles with scikit-learn 1.7.1. Other versions may warn or behave differently. Only load joblib/pickle files from sources you trust.
Training data
The model was trained on labelled accounts from these sources. No training data is included in this repository.
- Bot Into The Fediverse Dataset: F. Moreno García, Universidad Politécnica de Madrid, 2025. Mastodon accounts collected in January–February 2025, CC-BY-4.0. doi:10.5281/zenodo.17987595. Companion paper: doi:10.1007/s13278-025-01567-z.
- Cresci-2017: S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi. The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race. WWW '17 Companion, 2017. doi:10.1145/3041021.3055135. Labelled Twitter accounts, distributed for academic research only.
Training set: 44,730 accounts, balanced to 22,365 bots and 22,365 humans. They were split 80/20 (stratified, random_state=42) into 35,784 training and 8,946 test accounts. The scaler and the language vocabulary were fit on the training split only. The model is LogisticRegression(C=1.0, max_iter=2000).
Evaluation
Held-out test split (8,946 accounts, 50% bots), threshold 0.5:
| metric | value |
|---|---|
| ROC AUC | 0.963 |
| Accuracy | 0.910 |
| Precision (bot) | 0.910 |
| Recall (bot) | 0.910 |
| F1 (bot) | 0.910 |
Confusion matrix: 4,069 true negatives, 404 false positives, 404 false negatives, 4,069 true positives.
Limitations
- The test set is balanced (50% bots). On real instances bots are usually much rarer, so expect lower precision and consider raising
threshold. - Part of the training data (Cresci-2017) comes from Twitter, whose account conventions differ from the fediverse.
- Because Cresci-2017 is restricted to academic research, this model is intended for research use.
- The model was trained on a specific snapshot of accounts. Bot behaviour and instance conventions change over time.
- It only looks at profile metadata and the bio, not post content or posting times.
- Bios in languages outside the top 15 are grouped together as
other. - Use it as a signal for moderation or research, not as the only basis for actions against an account.
Third-party components (not redistributed here)
bert-base-multilingual-cased, Apache-2.0, downloaded from the Hub.- fastText
lid.176.ftz, CC-BY-SA 3.0, © Facebook. Downloaded on first use to~/.cache/fediverse-bot-detector/(override withFASTTEXT_MODEL_PATH).
Citation
If you use this model, please cite the dataset and paper:
@dataset{moreno_garcia_2025_botfediverse,
author = {Moreno García, Francisco},
title = {Bot Into The Fediverse Dataset},
year = {2025},
publisher = {Zenodo},
doi = {10.5281/zenodo.17987595}
}
- Downloads last month
- -