FinBERT LoRA Sentiment
FinBERT fine-tuned with LoRA adapters for financial sentiment classification. Built on ProsusAI/finbert, with LoRA adapters trained on FinancialPhraseBank.
Training Data
FinancialPhraseBank (sentences_allagree subset) — 1584 training sentences, 680 validation sentences.
Labels: 0 = positive, 1 = negative, 2 = neutral
Results
| Model | Macro F1 | Accuracy |
|---|---|---|
| FinBERT + LoRA (this model) | 0.97 | 98% |
| Base FinBERT (no fine-tuning) | 0.01 | 1% |
Training Details
- LoRA rank: 8, alpha: 32, target modules: query and value
- Batch size: 16, learning rate: 2e-4, epochs: 3
- Trained on Google Colab T4 GPU in ~90 seconds
How to Load
from transformers import AutoModelForSequenceClassification, AutoTokenizer from peft import PeftModel
base = AutoModelForSequenceClassification.from_pretrained("ProsusAI/finbert") model = PeftModel.from_pretrained(base, "Pranshu1104/finbert-lora-sentiment") tokenizer = AutoTokenizer.from_pretrained("ProsusAI/finbert")
- Downloads last month
- 43
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Pranshu1104/finbert-lora-sentiment
Base model
ProsusAI/finbert