|
|
--- |
|
|
language: en |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- finance |
|
|
- sentiment-analysis |
|
|
- modernbert |
|
|
- financial-nlp |
|
|
datasets: |
|
|
- neoyipeng/financial_reasoning_aggregated |
|
|
metrics: |
|
|
- accuracy |
|
|
widget: |
|
|
- text: "The company reported strong quarterly earnings with revenue beating expectations." |
|
|
example_title: "Positive Example" |
|
|
- text: "Stock prices fell sharply following disappointing guidance from management." |
|
|
example_title: "Negative Example" |
|
|
- text: "The merger is expected to close in Q3 pending regulatory approval." |
|
|
example_title: "Neutral Example" |
|
|
--- |
|
|
|
|
|
# ModernFinBERT: Financial Sentiment Analysis |
|
|
|
|
|
ModernFinBERT is a financial sentiment analysis model based on the ModernBERT architecture, fine-tuned specifically for financial text classification. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model**: answerdotai/ModernBERT-base |
|
|
- **Task**: 3-class sentiment classification (Negative, Neutral, Positive) |
|
|
- **Training Data**: Vanilla sentiment tasks from multiple sources (including FinancialPhraseBank) |
|
|
- **Parameters**: 149,607,171 |
|
|
|
|
|
## Performance |
|
|
|
|
|
### Overall Accuracy |
|
|
|
|
|
| Split | Accuracy | |
|
|
|-------|----------| |
|
|
| Validation | 85.3% | |
|
|
| Test | 83.1% | |
|
|
|
|
|
### Test Accuracy by Source |
|
|
|
|
|
| Source | Accuracy | Correct/Total | |
|
|
|--------|----------|---------------| |
|
|
| 4.0 | 89.5% | 77/86 | |
|
|
| 9.0 | 88.0% | 205/233 | |
|
|
| 5.0 | 84.4% | 205/243 | |
|
|
| 3.0 | 80.0% | 20/25 | |
|
|
| 8.0 | 69.1% | 94/136 | |
|
|
|
|
|
## Usage |
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
import torch |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("neoyipeng/ModernFinBERT-base") |
|
|
model = AutoModelForSequenceClassification.from_pretrained("neoyipeng/ModernFinBERT-base") |
|
|
|
|
|
text = "The company's quarterly results exceeded analyst expectations." |
|
|
inputs = tokenizer(text, return_tensors="pt") |
|
|
outputs = model(**inputs) |
|
|
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1) |
|
|
|
|
|
labels = ["NEGATIVE", "NEUTRAL", "POSITIVE"] |
|
|
predicted_class = labels[predictions.argmax().item()] |
|
|
confidence = predictions.max().item() |
|
|
|
|
|
print(f"Sentiment: {predicted_class} ({confidence:.2f})") |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **Epochs**: 3 |
|
|
- **Batch Size**: 32 |
|
|
- **Learning Rate**: 5e-05 |
|
|
- **Optimizer**: AdamW |
|
|
- **Scheduler**: Cosine |
|
|
- **Framework**: Unsloth + Transformers |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model, please cite: |
|
|
```bibtex |
|
|
@misc{modernfinbert2025, |
|
|
title={ModernFinBERT: A Modern Approach to Financial Sentiment Analysis}, |
|
|
author={Neo Yi Peng}, |
|
|
year={2025}, |
|
|
howpublished={HuggingFace Model Hub}, |
|
|
url={https://huggingface.co/neoyipeng/ModernFinBERT-base} |
|
|
} |
|
|
``` |
|
|
|
