|
|
---
|
|
|
license: apache-2.0
|
|
|
language:
|
|
|
- en
|
|
|
library_name: transformers
|
|
|
tags:
|
|
|
- finance
|
|
|
- aspect-classification
|
|
|
- absa
|
|
|
- finbert
|
|
|
- text-classification
|
|
|
datasets:
|
|
|
- pauri32/fiqa-2018
|
|
|
base_model: ProsusAI/finbert
|
|
|
metrics:
|
|
|
- accuracy
|
|
|
- f1
|
|
|
pipeline_tag: text-classification
|
|
|
---
|
|
|
|
|
|
# ABSA-FinBERT: Aspect Classification for Financial Text
|
|
|
|
|
|
This model classifies financial headlines and tweets into four aspect categories: **Corporate**, **Economy**, **Market**, and **Stock**.
|
|
|
|
|
|
## Model Description
|
|
|
|
|
|
ABSA-FinBERT is a fine-tuned version of [ProsusAI/finbert](https://huggingface.co/ProsusAI/finbert) for Level-1 aspect classification on the FiQA dataset. The model was trained with class-weighted cross-entropy loss to address extreme class imbalance in the training data.
|
|
|
|
|
|
This work is motivated by [Yang et al. (2018)](https://arxiv.org/abs/1808.07931), "Financial Aspect-Based Sentiment Analysis using Deep Representations," which demonstrated that financial text often contains multi-dimensional information requiring aspect-level analysis.
|
|
|
|
|
|
## Intended Use
|
|
|
|
|
|
- Classifying financial news headlines by topic/aspect
|
|
|
- Preprocessing step for aspect-based sentiment analysis pipelines
|
|
|
- Financial text categorization
|
|
|
|
|
|
## Training Data
|
|
|
|
|
|
Trained on the [FiQA dataset](https://huggingface.co/datasets/pauri32/fiqa-2018) (WWW'18 Open Challenge), with Level-1 aspect labels extracted from hierarchical annotations.
|
|
|
|
|
|
| Aspect | Training Examples | Percentage |
|
|
|
|--------|-------------------|------------|
|
|
|
| Stock | 562 | 58.5% |
|
|
|
| Corporate | 367 | 38.2% |
|
|
|
| Market | 26 | 2.7% |
|
|
|
| Economy | 4 | 0.4% |
|
|
|
|
|
|
### Class Weights Applied
|
|
|
Due to extreme imbalance, inverse frequency weights were used: Corporate (0.65), Economy (59.94), Market (9.22), Stock (0.43).
|
|
|
|
|
|
## Performance
|
|
|
|
|
|
| Metric | Score |
|
|
|
|--------|-------|
|
|
|
| Accuracy | 88.59% |
|
|
|
| Macro-F1 | 0.5429 |
|
|
|
| Weighted-F1 | 0.8688 |
|
|
|
|
|
|
### Per-Class Results
|
|
|
|
|
|
| Aspect | Precision | Recall | F1-Score | Support |
|
|
|
|--------|-----------|--------|----------|---------|
|
|
|
| Corporate | 0.91 | 0.94 | 0.92 | 64 |
|
|
|
| Economy | 0.00 | 0.00 | 0.00 | 3 |
|
|
|
| Market | 0.50 | 0.25 | 0.33 | 8 |
|
|
|
| Stock | 0.89 | 0.95 | 0.92 | 74 |
|
|
|
|
|
|
**Note:** The model performs well on majority classes but fails on Economy due to having only 4 training examples. Class weighting cannot overcome severe data scarcity.
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
```python
|
|
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
|
|
import torch
|
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("your-username/absa-finbert")
|
|
|
model = AutoModelForSequenceClassification.from_pretrained("your-username/absa-finbert")
|
|
|
|
|
|
# Label mapping
|
|
|
id2label = {0: "Corporate", 1: "Economy", 2: "Market", 3: "Stock"}
|
|
|
|
|
|
# Example inference
|
|
|
text = "How Kraft-Heinz Merger Came Together in Speedy 10 Weeks"
|
|
|
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
|
|
|
outputs = model(**inputs)
|
|
|
prediction = torch.argmax(outputs.logits, dim=-1).item()
|
|
|
|
|
|
print(f"Aspect: {id2label[prediction]}") # Output: Corporate
|
|
|
```
|
|
|
|
|
|
## Training Procedure
|
|
|
|
|
|
- **Base model:** ProsusAI/finbert
|
|
|
- **Learning rate:** 3e-5
|
|
|
- **Batch size:** 16 (effective 32 with gradient accumulation)
|
|
|
- **Epochs:** 10 (early stopping patience: 3)
|
|
|
- **Loss:** Weighted cross-entropy
|
|
|
- **Optimizer:** AdamW with warmup (10%)
|
|
|
- **Mixed precision:** FP16
|
|
|
|
|
|
## Limitations
|
|
|
|
|
|
- Economy class is effectively unlearnable with only 4 training examples
|
|
|
- Market class has limited representation (26 examples)
|
|
|
- Model is optimized for short financial headlines/tweets, not long-form text
|
|
|
|
|
|
## Citation
|
|
|
|
|
|
If you use this model, please cite:
|
|
|
|
|
|
```bibtex
|
|
|
@misc{absa-finbert-2025,
|
|
|
title={ABSA-FinBERT: Aspect Classification for Financial Text},
|
|
|
author={Cirillo, Nick and Memon, Suha and Truong, Kalen and Zhang, Bruce},
|
|
|
year={2025},
|
|
|
howpublished={\url{https://huggingface.co/your-username/absa-finbert}}
|
|
|
}
|
|
|
```
|
|
|
|
|
|
## References
|
|
|
|
|
|
- Yang, S., Rosenfeld, J., & Makutonin, J. (2018). Financial Aspect-Based Sentiment Analysis using Deep Representations. arXiv:1808.07931.
|
|
|
- Araci, D. (2019). FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. arXiv:1908.10063.
|
|
|
- Maia, M., et al. (2018). WWW'18 Open Challenge: Financial Opinion Mining and Question Answering. |
