language:
- en
license: mit
library_name: peft
tags:
- finance
- sentiment
- finbert
- lora
- efficient-tuning
datasets:
- financial_phrasebank
- zeroshot/twitter-financial-news-sentiment
- TheFinAI/fiqa-sentiment-classification
metrics:
- accuracy
base_model: ProsusAI/finbert
widget:
- text: The company reported a record 20% increase in revenue this quarter.
example_title: Strong Earnings
- text: Analysts are worried about the looming debt crisis.
example_title: Negative Outlook
Financial-Sentiment-LLM (Multi-Task LoRA)
A production-ready financial sentiment classifier fine-tuned on FinBERT. This model utilizes a Multi-Task Architecture (Classification + Regression) to achieve state-of-the-art performance across diverse financial text sources, including professional news, social media, and forum discussions.
- 🚀 Live Demo: Hugging Face Space
- 💻 Source Code: GitHub Repository
- 👨💻 Author: Pablo Matorras-Cuevas
Efficiency vs. Performance
We benchmarked this LoRA adapter against the fully fine-tuned baseline. It matches the full model on professional news but trades off slight accuracy on complex forum discussions for massive efficiency gains.
| Metric / Dataset | FinBERT (LoRA) | FinBERT (Full) |
|---|---|---|
| Model Size | ~5 MB | ~420 MB |
| Overall Accuracy | 83.2% | 85.4% |
| Financial PhraseBank (News) | 97.1% | 95.9% |
| Twitter Financial News | 80.5% | 83.3% |
| FiQA (Forums) | 72.6% | 81.5% |
Recommendation: Use this LoRA version for analyzing News headlines (where it actually beats the full model) or whenever model size is a critical constraint. For maximum accuracy on messy social media data, use the Full Multi-Task Model.
Architecture
Unlike standard sentiment classifiers, this model shares a bert-base backbone with two task-specific heads:
- Classification Head: Predicts
Negative/Neutral/Positive(Optimized for News & Twitter). - Regression Head: Predicts a continuous sentiment score (Optimized for FiQA forum discussions).
This approach yielded a +6.1% accuracy boost on Twitter data compared to single-task training, proving that learning continuous sentiment intensity helps the model understand noisy social text better.
Training Details
- Base Model:
ProsusAI/finbert - Technique: LoRA (Low-Rank Adaptation)
- Rank (r): 16 (matches the experiment config)
- Trainable Parameters: ~0.6% of full model
- Hardware: Trained on NVIDIA RTX 4050 in ~10 mins.
Usage
To use this adapter, you need the peft library alongside transformers:
from peft import PeftModel, PeftConfig
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
# Load the specific LoRA configuration
# (Make sure this matches your REPO NAME exactly)
adapter_id = "pmatorras/financial-sentiment-analysis-lora"
config = PeftConfig.from_pretrained(adapter_id)
# Load the Base Model (ProsusAI/finbert)
# The config automatically knows the base model path
base_model = AutoModelForSequenceClassification.from_pretrained(config.base_model_name_or_path)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
# Load the LoRA Adapter on top
model = PeftModel.from_pretrained(base_model, adapter_id)
# Inference
text = "The company's cost-cutting measures are expected to boost margins significantly."
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
probabilities = torch.nn.functional.softmax(outputs.logits, dim=-1)
print(probabilities)