license: apache-2.0
base_model: ProsusAI/finbert
tags:
- financial-sentiment
- social-media
- reddit
- wallstreetbets
- text-classification
pipeline_tag: text-classification
language:
- en
WSB-FinBERT
A FinBERT model fine-tuned on r/wallstreetbets text for three-class sentiment classification (negative / neutral / positive).
Off-the-shelf financial sentiment models are trained on formal financial English — earnings calls, analyst reports, newswire. Retail social-media text is a different register: irony, slang, emoji, and self-deprecation. On the held-out set below, vanilla FinBERT scores below the majority-class floor, i.e. worse than ignoring the text entirely. This model is a domain-adaptation check on that gap.
This is the frozen checkpoint behind the results reported in Beyond the Volume of Attention: Domain-Adapted Sentiment and the Content of Retail Investor Discussion (ICAIF '26) — not a retrained copy. The replication package for that paper is distributed separately; this repository is the model only.
Labels
| id | label |
|---|---|
| 0 | negative |
| 1 | neutral |
| 2 | positive |
Usage
You do not need to download anything by hand. transformers fetches the weights on first use
and caches them under ~/.cache/huggingface/, so the first call takes a moment (~440 MB) and
every call after that is instant.
pip install transformers torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "AnonymousResearchICAIF/wsb-finbert"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
text = "NVDA printing again, loaded calls for next week"
inputs = tok(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
probs = model(**inputs).logits.softmax(-1)[0]
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})
A continuous sentiment score in [-1, 1] is formed as P(positive) - P(negative). This is the
sentiment_wsb variable used throughout the paper — the sign gives the direction and the
magnitude gives the confidence. Use max_length=128: that is what the model was trained
with, and longer inputs are truncated to it.
For many texts at once, batch them rather than looping:
texts = ["...", "...", "..."]
batch = tok(texts, return_tensors="pt", truncation=True, max_length=128, padding=True)
with torch.no_grad():
p = model(**batch).logits.softmax(-1)
scores = (p[:, 2] - p[:, 0]).tolist()
Training data
2,503 r/wallstreetbets posts and comments mentioning a fixed universe of AI-related tickers, labelled for sentiment by an LLM teacher (Claude) and split 70/15/15 stratified by label with seed 42 — 1,752 train / 375 validation / 376 test. The sample is stratified across tickers and balanced 50/50 between the first and second halves of the collection period.
Training procedure
| Base model | ProsusAI/finbert |
| Max sequence length | 128 |
| Learning rate | 2e-5 |
| Epochs | 4 |
| Train batch size | 8 |
| Weight decay | 0.01 |
| Warmup ratio | 0.1 |
| Seed | 42 |
Evaluation
On the 376 held-out texts:
| Model | Accuracy | Macro F1 |
|---|---|---|
| WSB-FinBERT (this model) | 0.524 | 0.495 |
Vanilla ProsusAI/finbert |
0.410 | 0.400 |
| Majority-class baseline ("always positive") | 0.434 | — |
Cohen's κ = 0.252.
Per-class (this model):
| label | precision | recall | f1 | support |
|---|---|---|---|---|
| negative | 0.448 | 0.312 | 0.368 | 96 |
| neutral | 0.513 | 0.513 | 0.513 | 117 |
| positive | 0.557 | 0.656 | 0.603 | 163 |
Limitations
- Absolute accuracy is modest. 52.4% on a three-class problem is only ~9 points above the majority-class floor. The gain over vanilla FinBERT (+11.4 points) is the meaningful result; the model is not a strong standalone classifier.
- Labels are LLM-generated, not human gold standard. They inherit the teacher's biases. They do correlate with past five-day returns, as a text-only annotator should, and show no positive correlation with forward returns (no evidence of look-ahead leakage).
- Narrow domain. Trained on r/wallstreetbets text about a small universe of AI-related tickers over a specific period. Generalisation to other subreddits, other sectors, other periods, or to formal financial text is untested and unlikely.
- Positive skew. The forum's labels run ~1.7 to 1 positive and the model inherits this (51.1% of predictions positive vs 43.4% in truth). Use within-entity demeaning if the level matters for your application.
- Not investment advice; not suitable for trading decisions on its own.