File size: 6,581 Bytes

054c79a

# Financial Sentiment Classifier 📈

A fine-tuned DistilBERT model for financial text sentiment analysis, capable of classifying financial news and statements into three categories: **positive**, **negative**, and **neutral**.

## Model Description

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) specifically trained on financial text data for sentiment classification. It achieves **97.5% accuracy** on the validation set and is optimized for analyzing financial news, earnings reports, market commentary, and other finance-related text.

### Key Features
- **High Performance**: 97.5% accuracy on financial sentiment classification
- **Fast Inference**: Built on DistilBERT for efficient processing
- **Domain-Specific**: Trained specifically on financial text data
- **Balanced Classes**: Handles positive, negative, and neutral sentiments effectively

## Model Details

- **Base Model**: distilbert-base-uncased
- **Task**: Text Classification (Sentiment Analysis)
- **Language**: English
- **Domain**: Financial Text
- **Classes**: 3 (positive, negative, neutral)
- **Training Data**: ~36K financial text samples (original + synthetic data)

### Performance Metrics

| Metric | Score |
|--------|-------|
| Accuracy | 97.52% |
| F1-Score | 97.51% |
| Precision | 97.52% |
| Recall | 97.52% |

## Quick Start

### Installation

```bash
pip install transformers torch
```

### Usage

```python
from transformers import pipeline

# Load the classifier
classifier = pipeline(
    "text-classification", 
    model="AdityaAI9/distilbert_finance_sentiment_analysis"
)

# Analyze financial text
result = classifier("The company reported strong quarterly earnings with 15% revenue growth.")
print(result)
# Output: [{'label': 'positive', 'score': 0.9845}]

# Multiple examples
texts = [
    "Stock prices fell sharply due to disappointing earnings.",
    "The company maintained steady performance this quarter.",
    "Revenue exceeded expectations with record-breaking profits."
]

results = classifier(texts)
for text, result in zip(texts, results):
    print(f"Text: {text}")
    print(f"Sentiment: {result['label']} (confidence: {result['score']:.3f})")
    print("-" * 50)
```

### Advanced Usage

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("AdityaAI9/distilbert_finance_sentiment_analysis")
model = AutoModelForSequenceClassification.from_pretrained("AdityaAI9/distilbert_finance_sentiment_analysis")

# Manual prediction
text = "The merger is expected to create significant shareholder value."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=256)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(predictions, dim=-1)

# Map to labels
label_mapping = {0: "negative", 1: "neutral", 2: "positive"}
sentiment = label_mapping[predicted_class.item()]
confidence = predictions.max().item()

print(f"Sentiment: {sentiment} (confidence: {confidence:.3f})")
```

## Training Data

The model was trained on a combination of:

1. **Original Financial News Dataset**: ~5K labeled financial news sentences
2. **Synthetic Financial Data**: ~31K synthetic financial statements generated using state-of-the-art language models

The synthetic data generation approach helps address class imbalance and provides diverse financial vocabulary coverage. You can find the synthetic data generation code [here](https://github.com/aditya699/Common-Challenges-in-LLMS-/blob/main/synthetic_data_generator/syndata.py).

### Data Distribution
- **Neutral**: 14,638 samples (40.2%)
- **Negative**: 11,272 samples (31.0%)
- **Positive**: 10,539 samples (28.8%)

## Training Details

### Training Hyperparameters
- **Epochs**: 5
- **Batch Size**: 16 (training), 32 (validation)
- **Learning Rate**: Default AdamW
- **Max Sequence Length**: 256 tokens
- **Optimizer**: AdamW
- **Warmup**: Linear warmup

### Training Infrastructure
- **GPU**: CUDA-enabled training
- **Framework**: Hugging Face Transformers
- **Evaluation Strategy**: Every 500 steps

## Use Cases

This model is particularly useful for:

- **Financial News Analysis**: Classify sentiment of news articles affecting stock prices
- **Earnings Report Processing**: Analyze quarterly and annual reports
- **Market Research**: Sentiment analysis of financial commentary and analyst reports
- **Trading Signals**: Generate sentiment-based trading indicators
- **Risk Assessment**: Evaluate sentiment trends for investment decisions
- **Social Media Monitoring**: Analyze financial discussions on social platforms

## Limitations and Considerations

- **Domain Specificity**: Optimized for financial text; may not perform well on general sentiment tasks
- **Language**: Currently supports English only
- **Context Window**: Limited to 256 tokens; longer texts will be truncated
- **Temporal Bias**: Trained on contemporary financial language; may need updates for evolving terminology
- **Market Context**: Does not consider broader market conditions or temporal context

## Ethical Considerations

- This model should not be the sole basis for financial decisions
- Always combine with fundamental analysis and professional financial advice
- Be aware of potential biases in training data
- Consider market volatility and external factors not captured in text

## Citation

If you use this model in your research or applications, please cite:

```bibtex
@misc{distilbert-finance-sentiment-analysis,
  title={Financial Sentiment Classifier: A Fine-tuned DistilBERT Model},
  author={AdityaAI9},
  year={2024},
  howpublished={\url{https://huggingface.co/AdityaAI9/distilbert_finance_sentiment_analysis}},
}
```

## License

This model is released under the MIT License. See LICENSE for more details.

## Acknowledgments

- Built using [Hugging Face Transformers](https://huggingface.co/transformers/)
- Base model: [DistilBERT](https://huggingface.co/distilbert-base-uncased)
- Synthetic data generation techniques for improved model performance

## Contact

For questions, suggestions, or collaboration opportunities, please open an issue in the [GitHub repository](https://github.com/aditya699/Common-Challenges-in-LLMS-) or reach out through Hugging Face.

---

**Note**: This model is for research and educational purposes. Always consult with financial professionals before making investment decisions.