Upload README-finlora.md

finlora_hf_submission/README-finlora.md

@@ -0,0 +1,410 @@
+---
+license: mit
+tags:
+- finance
+- llm
+- lora
+- sentiment-analysis
+- named-entity-recognition
+- xbrl
+pipeline_tag: text-generation
+---
+
+# FinLoRA: Financial Large Language Models with LoRA Adaptation
+
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-red.svg)](https://pytorch.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+## Overview
+
+FinLoRA is a comprehensive framework for fine-tuning large language models on financial tasks using Low-Rank Adaptation (LoRA). This repository contains trained LoRA adapters for various financial NLP tasks including sentiment analysis, named entity recognition, headline classification, XBRL processing, and **RAG-enhanced models** for CFA knowledge and FinTagging tasks.
+
+## Model Architecture
+
+- **Base Model**: Meta-Llama-3.1-8B-Instruct
+- **Adaptation Method**: LoRA (Low-Rank Adaptation)
+- **Quantization**: 8-bit and 4-bit quantization support
+- **Tasks**: Financial sentiment analysis, NER, classification, XBRL processing, CFA knowledge, FinTagging
+
+## Available Models
+
+### 8-bit Quantized Models (Recommended)
+- `sentiment_llama_3_1_8b_8bits_r8` - Financial sentiment analysis
+- `ner_llama_3_1_8b_8bits_r8` - Named entity recognition
+- `headline_llama_3_1_8b_8bits_r8` - Financial headline classification
+- `xbrl_extract_llama_3_1_8b_8bits_r8` - XBRL tag extraction
+- `xbrl_term_llama_3_1_8b_8bits_r8` - XBRL terminology processing
+- `financebench_llama_3_1_8b_8bits_r8` - Comprehensive financial benchmark
+- `finer_llama_3_1_8b_8bits_r8` - Financial NER
+- `formula_llama_3_1_8b_8bits_r8` - Financial formula processing
+
+### RAG-Enhanced Models (New!)
+- `cfa_rag_llama_3_1_8b_8bits_r8` - CFA knowledge-enhanced model with RAG
+- `fintagging_combined_rag_llama_3_1_8b_8bits_r8` - Combined FinTagging RAG model
+- `fintagging_fincl_rag_llama_3_1_8b_8bits_r8` - FinCL RAG-enhanced model
+- `fintagging_finni_rag_llama_3_1_8b_8bits_r8` - FinNI RAG-enhanced model
+
+### 4-bit Quantized Models (Memory Efficient)
+- `sentiment_llama_3_1_8b_4bits_r4` - Financial sentiment analysis
+- `ner_llama_3_1_8b_4bits_r4` - Named entity recognition
+- `headline_llama_3_1_8b_4bits_r4` - Financial headline classification
+- `xbrl_extract_llama_3_1_8b_4bits_r4` - XBRL tag extraction
+- `xbrl_term_llama_3_1_8b_4bits_r4` - XBRL terminology processing
+- `financebench_llama_3_1_8b_4bits_r4` - Comprehensive financial benchmark
+- `finer_llama_3_1_8b_4bits_r4` - Financial NER
+- `formula_llama_3_1_8b_4bits_r4` - Financial formula processing
+
+## Quick Start
+
+### 1. Installation
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+```
+
+### 2. Basic Usage
+
+```python
+from inference import FinLoRAPredictor
+
+# Initialize predictor with 8-bit model (recommended)
+predictor = FinLoRAPredictor(
+    model_name="sentiment_llama_3_1_8b_8bits_r8",
+    use_4bit=False
+)
+
+# Financial sentiment analysis
+sentiment = predictor.classify_sentiment(
+    "The company's quarterly earnings exceeded expectations by 20%."
+)
+print(f"Sentiment: {sentiment}")
+
+# Entity extraction
+entities = predictor.extract_entities(
+    "Apple Inc. reported revenue of $394.3 billion in 2022."
+)
+print(f"Entities: {entities}")
+
+# Use 4-bit model for memory efficiency (if you have limited GPU memory)
+predictor_4bit = FinLoRAPredictor(
+    model_name="sentiment_llama_3_1_8b_4bits_r4",
+    use_4bit=True
+)
+
+# CPU-only mode (if no GPU available)
+predictor_cpu = FinLoRAPredictor(
+    model_name="sentiment_llama_3_1_8b_8bits_r8",
+    use_4bit=False
+)
+# The script will automatically detect CPU and adjust accordingly
+```
+
+### 3. Run Complete Test
+
+```bash
+# Test all models (this will download the base Llama model if not present)
+python inference.py
+
+# Test specific model
+python -c "
+from inference import FinLoRAPredictor
+predictor = FinLoRAPredictor('sentiment_llama_3_1_8b_8bits_r8')
+print('Model loaded successfully!')
+"
+```
+
+## Usage Examples
+
+### Financial Sentiment Analysis
+
+```python
+predictor = FinLoRAPredictor("sentiment_llama_3_1_8b_8bits_r8")
+
+# Test cases
+test_texts = [
+    "Stock prices are soaring to new heights.",
+    "Revenue declined by 15% this quarter.",
+    "The company maintained stable performance."
+]
+
+for text in test_texts:
+    sentiment = predictor.classify_sentiment(text)
+    print(f"Text: {text}")
+    print(f"Sentiment: {sentiment}\n")
+```
+
+### Named Entity Recognition
+
+```python
+predictor = FinLoRAPredictor("ner_llama_3_1_8b_8bits_r8")
+
+text = "Apple Inc. reported revenue of $394.3 billion in 2022."
+entities = predictor.extract_entities(text)
+print(f"Entities: {entities}")
+```
+
+### XBRL Processing
+
+```python
+predictor = FinLoRAPredictor("xbrl_extract_llama_3_1_8b_8bits_r8")
+
+text = "Total assets: $1,234,567,890. Current assets: $456,789,123."
+xbrl_tags = predictor.extract_xbrl_tags(text)
+print(f"XBRL Tags: {xbrl_tags}")
+```
+
+### RAG-Enhanced Models
+
+```python
+# CFA RAG-enhanced model for financial knowledge
+predictor = FinLoRAPredictor("cfa_rag_llama_3_1_8b_8bits_r8")
+
+# Enhanced financial analysis with CFA knowledge
+response = predictor.generate_response(
+    "Explain the concept of discounted cash flow valuation"
+)
+print(f"CFA Response: {response}")
+
+# FinTagging RAG models for financial information extraction
+fintagging_predictor = FinLoRAPredictor("fintagging_combined_rag_llama_3_1_8b_8bits_r8")
+
+# Extract financial information with enhanced context
+entities = fintagging_predictor.extract_entities(
+    "Apple Inc. reported revenue of $394.3 billion in 2022."
+)
+print(f"Enhanced Entities: {entities}")
+```
+
+### Memory-Efficient 4-bit Models
+
+```python
+# For users with limited GPU memory
+predictor = FinLoRAPredictor(
+    model_name="sentiment_llama_3_1_8b_4bits_r4",
+    use_4bit=True
+)
+
+# Same API as 8-bit models
+sentiment = predictor.classify_sentiment("The market is performing well.")
+```
+
+## Evaluation
+
+### For Competition Organizers
+
+This section provides guidance for evaluating the submitted models:
+
+#### 1. Quick Model Test
+```bash
+# Test if all models can be loaded successfully
+python test_submission.py
+```
+
+#### 2. Comprehensive Evaluation
+```bash
+# Run full evaluation on all models and datasets
+python comprehensive_evaluation.py
+
+# Check results
+cat comprehensive_evaluation_results.json
+```
+
+#### 3. Incremental Evaluation
+```bash
+# Run evaluation on missing tasks
+python incremental_evaluation.py
+
+# Check results
+cat incremental_evaluation_results.json
+```
+
+#### 4. Evaluation Results
+The evaluation results are provided in:
+- `comprehensive_evaluation_results.json` - Complete evaluation results
+- `incremental_evaluation_results.json` - Missing task evaluation results
+
+#### 5. Model Performance Summary
+All models have been evaluated on multiple financial datasets. See the Performance Results section below for detailed metrics.
+
+### For Researchers
+
+Run comprehensive evaluation on financial datasets:
+
+```bash
+# Run full evaluation
+python comprehensive_evaluation.py
+
+# Run incremental evaluation
+python incremental_evaluation.py
+
+# Run robust evaluation
+python robust_incremental.py
+```
+
+## Performance Results
+
+The models have been evaluated on multiple financial datasets:
+
+| Task | Dataset | F1 Score | Accuracy |
+|------|---------|----------|----------|
+| Sentiment Analysis | Financial Phrasebank | 0.333 | 0.500 |
+| NER | Financial NER | 0.889 | 0.800 |
+| Classification | Headline Classification | 0.697 | 0.700 |
+| XBRL Processing | XBRL Tag Extraction | - | 0.200 |
+| Sentiment Analysis | FIQA SA | 0.727 | 0.700 |
+
+## Project Structure
+
+```
+finlora_hf_submission/
+├── models/                          # 8-bit LoRA model adapters (13 models)
+│   ├── sentiment_llama_3_1_8b_8bits_r8/
+│   ├── ner_llama_3_1_8b_8bits_r8/
+│   ├── headline_llama_3_1_8b_8bits_r8/
+│   ├── xbrl_extract_llama_3_1_8b_8bits_r8/
+│   ├── xbrl_term_llama_3_1_8b_8bits_r8/
+│   ├── financebench_llama_3_1_8b_8bits_r8/
+│   ├── finer_llama_3_1_8b_8bits_r8/
+│   ├── formula_llama_3_1_8b_8bits_r8/
+│   ├── cfa_rag_llama_3_1_8b_8bits_r8/          # NEW: CFA RAG model
+│   ├── fintagging_combined_rag_llama_3_1_8b_8bits_r8/  # NEW: Combined RAG
+│   ├── fintagging_fincl_rag_llama_3_1_8b_8bits_r8/     # NEW: FinCL RAG
+│   ├── fintagging_finni_rag_llama_3_1_8b_8bits_r8/     # NEW: FinNI RAG
+│   └── xbrl_train.jsonl-meta-llama-Llama-3.1-8B-Instruct-8bits_r8/
+├── models_4bit/                     # 4-bit LoRA model adapters (8 models)
+│   ├── sentiment_llama_3_1_8b_4bits_r4/
+│   ├── ner_llama_3_1_8b_4bits_r4/
+│   ├── headline_llama_3_1_8b_4bits_r4/
+│   ├── xbrl_extract_llama_3_1_8b_4bits_r4/
+│   ├── xbrl_term_llama_3_1_8b_4bits_r4/
+│   ├── financebench_llama_3_1_8b_4bits_r4/
+│   ├── finer_llama_3_1_8b_4bits_r4/
+│   └── formula_llama_3_1_8b_4bits_r4/
+├── testdata/                        # Evaluation datasets
+│   ├── FinCL-eval-subset.csv
+│   └── FinNI-eval-subset.csv
+├── rag_system/                      # RAG system components
+├── inference.py                     # Main inference script
+├── comprehensive_evaluation.py      # Full evaluation script
+├── incremental_evaluation.py        # Incremental evaluation
+├── robust_incremental.py           # Robust evaluation
+├── missing_tests.py                # Missing test detection
+├── requirements.txt                 # Python dependencies
+└── README.md                       # This file
+```
+
+## Environment Requirements
+
+### Minimum Requirements (CPU Mode)
+- Python 3.8+
+- PyTorch 2.0+
+- 8GB RAM
+- No GPU required
+
+### Recommended Requirements (GPU Mode)
+- Python 3.9+
+- PyTorch 2.1+
+- CUDA 11.8+ (for NVIDIA GPUs)
+- 16GB+ GPU memory
+- 32GB+ RAM
+
+### Installation Instructions
+
+```bash
+# 1. Clone or download this repository
+# 2. Install dependencies
+pip install -r requirements.txt
+
+# 3. For GPU support (optional but recommended)
+pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
+
+# 4. Verify installation
+python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
+```
+
+### Troubleshooting
+
+**If you encounter memory issues:**
+- Use 4-bit models instead of 8-bit models
+- Reduce batch size in inference
+- Use CPU mode if GPU memory is insufficient
+
+**If models fail to load:**
+- Ensure all model files are present in the correct directories
+- Check that the base model (Llama-3.1-8B-Instruct) can be downloaded from HuggingFace
+- Verify internet connection for model downloads
+
+**Important Notes for Competition Organizers:**
+- The base model (Llama-3.1-8B-Instruct) will be automatically downloaded from HuggingFace on first use (~15GB)
+- All LoRA adapters are included in this submission and do not require additional downloads
+- Models work in both CPU and GPU modes, with automatic device detection
+- RAG-enhanced models require the same base model as regular models
+
+## Model Details
+
+### Training Configuration
+- **LoRA Rank**: 8
+- **LoRA Alpha**: 16
+- **Learning Rate**: 1e-4
+- **Batch Size**: 4
+- **Epochs**: 3-5
+- **Quantization**: 8-bit (BitsAndBytes) / 4-bit (NF4)
+
+### Training Data
+- Financial Phrasebank
+- FinGPT datasets (NER, Headline, XBRL)
+- BloombergGPT financial datasets
+- Custom financial text datasets
+
+## Citation
+
+If you use this work in your research, please cite:
+
+```bibtex
+@article{finlora2024,
+  title={FinLoRA: Financial Large Language Models with LoRA Adaptation},
+  author={Your Name},
+  journal={Financial AI Conference},
+  year={2024}
+}
+```
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## Contact
+
+For questions and support, please open an issue or contact [your-email@example.com](mailto:your-email@example.com).
+
+## Submission Summary
+
+### What's Included
+- **21 Total Models**: 13 8-bit models (9 original + 4 RAG-enhanced) + 8 4-bit models
+- **Complete Evaluation Results**: Comprehensive and incremental evaluation results
+- **RAG-Enhanced Models**: CFA and FinTagging models with enhanced knowledge
+- **Cross-Platform Support**: Works on CPU, GPU, and various memory configurations
+- **Ready-to-Use**: All dependencies specified, automatic device detection
+
+### Quick Start for Competition Organizers
+1. Install dependencies: `pip install -r requirements.txt`
+2. Test submission: `python test_submission.py`
+3. Run evaluation: `python comprehensive_evaluation.py`
+4. Check results: `cat comprehensive_evaluation_results.json`
+
+### Model Categories
+- **Financial NLP**: Sentiment, NER, Classification, XBRL processing
+- **RAG-Enhanced**: CFA knowledge and FinTagging with retrieval augmentation
+- **Memory Options**: Both 8-bit and 4-bit quantized versions available
+
+## Acknowledgments
+
+- Meta for the Llama-3.1-8B-Instruct base model
+- Hugging Face for the transformers and PEFT libraries
+- The financial NLP community for datasets and benchmarks