Delete finlora_hf_submission/README——finlora.md

finlora_hf_submission/README——finlora.md

@@ -1,651 +0,0 @@
-# FinLoRA: Financial Large Language Models with LoRA Adaptation
-
-## Overview
-
-FinLoRA is a comprehensive framework for fine-tuning large language models on financial tasks using Low-Rank Adaptation (LoRA). This project provides trained LoRA adapters for various financial NLP tasks including sentiment analysis, named entity recognition, headline classification, XBRL processing, and CFA knowledge integration.
-
-## Model Architecture
-
-- **Base Model**: Meta-Llama-3.1-8B-Instruct
-- **Adaptation Method**: LoRA (Low-Rank Adaptation)
-- **Quantization**: 8-bit and 4-bit quantization support
-- **Tasks**: Financial sentiment analysis, NER, classification, XBRL processing, CFA knowledge integration
-
-## Available Models
-
-### Core Financial Models
-- `sentiment_llama_3_1_8b_8bits_r8` - Financial sentiment analysis
-- `ner_llama_3_1_8b_8bits_r8` - Named entity recognition
-- `headline_llama_3_1_8b_8bits_r8` - Financial headline classification
-- `xbrl_extract_llama_3_1_8b_8bits_r8` - XBRL tag extraction
-- `xbrl_term_llama_3_1_8b_8bits_r8` - XBRL terminology processing
-
-### Advanced Models
-- `financebench_llama_3_1_8b_8bits_r8` - Comprehensive financial benchmark
-- `finer_llama_3_1_8b_8bits_r8` - Financial NER
-- `formula_llama_3_1_8b_8bits_r8` - Financial formula processing
-
-### RAG Knowledge Base
-- CFA RAG knowledge base (FAISS index + JSONL data)
-- FinTagging RAG knowledge base (FAISS index + JSONL data)
-- RAG system scripts and configuration files
-
-## Quick Start (5 minutes)
-
-### 1. Environment Setup
-```bash
-# Clone the repository
-git clone <repository-url>
-cd FinLora——RAG
-
-# Create and activate environment
-conda env create -f FinLoRA/environment.yml
-conda activate finenv
-```
-
-### 2. Test a Single Model
-```python
-# Quick test script
-from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
-from peft import PeftModel
-import torch
-
-# Check if CUDA is available
-device = "cuda" if torch.cuda.is_available() else "cpu"
-print(f"Using device: {device}")
-
-# Load model (replace with your model path)
-model_path = "FinLoRA/lora_adapters/8bits_r8/sentiment_llama_3_1_8b_8bits_r8"
-base_model = "meta-llama/Llama-3.1-8B-Instruct"
-
-# Load tokenizer
-tokenizer = AutoTokenizer.from_pretrained(base_model)
-if tokenizer.pad_token is None:
-    tokenizer.pad_token = tokenizer.eos_token
-
-# Configure quantization based on device
-if device == "cuda":
-    bnb_config = BitsAndBytesConfig(load_in_8bit=True)
-    base_model = AutoModelForCausalLM.from_pretrained(
-        base_model, quantization_config=bnb_config, device_map="auto"
-    )
-else:
-    # CPU mode - no quantization
-    base_model = AutoModelForCausalLM.from_pretrained(
-        base_model, device_map="cpu", torch_dtype=torch.float32
-    )
-
-# Load LoRA adapter
-model = PeftModel.from_pretrained(base_model, model_path)
-
-# Test inference
-def quick_test(text):
-    inputs = tokenizer(text, return_tensors="pt")
-    with torch.no_grad():
-        outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
-    return tokenizer.decode(outputs[0], skip_special_tokens=True)
-
-# Test
-result = quick_test("Classify sentiment: 'The stock market is performing well today.'")
-print(result)
-```
-
-### 3. Run Full Evaluation
-```bash
-cd testdata
-python comprehensive_evaluation.py
-```
-
-## Environment Setup
-
-### Quest Cluster Environment (Original Development)
-
-The original development was done on Northwestern University's Quest cluster with:
-- **OS**: Linux 4.18.0-553.64.1.el8_10.x86_64
-- **GPU**: NVIDIA H100 80GB HBM3
-- **CUDA**: Version 12.8
-- **Environment**: `finenv` conda environment
-
-### Option 1: Using Conda (Recommended)
-
-```bash
-# Create environment from provided environment.yml
-conda env create -f FinLoRA/environment.yml
-
-# Activate environment
-conda activate finenv
-
-# Install additional requirements
-pip install -r FinLoRA/requirements.txt
-```
-
-### Option 2: Manual Installation
-
-#### For GPU Users:
-```bash
-# Create new conda environment
-conda create -n finlora python=3.11
-
-# Activate environment
-conda activate finlora
-
-# Install PyTorch with CUDA support
-conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
-
-# Install core dependencies
-pip install transformers==4.45.2
-pip install datasets==2.19.1
-pip install peft==0.13.2
-pip install bitsandbytes==0.44.1
-pip install accelerate==1.0.0
-pip install deepspeed==0.15.2
-pip install sentence-transformers
-pip install faiss-cpu
-pip install scikit-learn
-pip install pandas numpy
-```
-
-#### For CPU-Only Users:
-```bash
-# Create new conda environment
-conda create -n finlora python=3.11
-
-# Activate environment
-conda activate finlora
-
-# Install PyTorch CPU version
-conda install pytorch torchvision torchaudio cpuonly -c pytorch
-
-# Install core dependencies (CPU-compatible versions)
-pip install transformers==4.45.2
-pip install datasets==2.19.1
-pip install peft==0.13.2
-pip install accelerate==1.0.0
-pip install sentence-transformers
-pip install faiss-cpu
-pip install scikit-learn
-pip install pandas numpy
-```
-
-### Option 3: Alternative Platforms
-
-#### Google Colab
-```python
-# Install dependencies
-!pip install transformers==4.45.2
-!pip install datasets==2.19.1
-!pip install peft==0.13.2
-!pip install bitsandbytes==0.44.1
-!pip install accelerate==1.0.0
-!pip install sentence-transformers
-!pip install faiss-cpu
-!pip install scikit-learn
-
-# Check GPU availability
-import torch
-print(f"CUDA available: {torch.cuda.is_available()}")
-if torch.cuda.is_available():
-    print(f"GPU: {torch.cuda.get_device_name(0)}")
-    print(f"GPU memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
-```
-
-#### AWS EC2 / Azure / Local GPU
-```bash
-# Install NVIDIA drivers and CUDA toolkit
-# Then follow Option 1 or 2 above
-```
-
-#### CPU-Only Mode
-```python
-# Complete CPU-only model loading example
-from transformers import AutoTokenizer, AutoModelForCausalLM
-from peft import PeftModel
-import torch
-
-# Force CPU usage
-device = "cpu"
-torch.set_default_device(device)
-
-# Load tokenizer
-tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
-if tokenizer.pad_token is None:
-    tokenizer.pad_token = tokenizer.eos_token
-
-# Load base model for CPU (no quantization)
-base_model = AutoModelForCausalLM.from_pretrained(
-    "meta-llama/Llama-3.1-8B-Instruct",
-    device_map="cpu",
-    torch_dtype=torch.float32,
-    low_cpu_mem_usage=True
-)
-
-# Load LoRA adapter
-model = PeftModel.from_pretrained(base_model, "path/to/lora/adapter")
-
-# Test inference
-def cpu_predict(text):
-    inputs = tokenizer(text, return_tensors="pt")
-    with torch.no_grad():
-        outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
-    return tokenizer.decode(outputs[0], skip_special_tokens=True)
-
-# Test
-result = cpu_predict("Classify sentiment: 'The market is performing well.'")
-print(result)
-```
-
-## Usage Instructions
-
-### 1. Basic Model Loading and Inference
-
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
-from peft import PeftModel
-import torch
-
-# Check device availability
-device = "cuda" if torch.cuda.is_available() else "cpu"
-print(f"Using device: {device}")
-
-# Load tokenizer
-tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
-if tokenizer.pad_token is None:
-    tokenizer.pad_token = tokenizer.eos_token
-
-# Configure model loading based on device
-if device == "cuda":
-    # GPU mode with quantization
-    bnb_config = BitsAndBytesConfig(
-        load_in_8bit=True,
-        llm_int8_threshold=6.0
-    )
-    base_model = AutoModelForCausalLM.from_pretrained(
-        "meta-llama/Llama-3.1-8B-Instruct",
-        quantization_config=bnb_config,
-        device_map="auto",
-        torch_dtype=torch.float16,
-        trust_remote_code=True
-    )
-else:
-    # CPU mode without quantization
-    base_model = AutoModelForCausalLM.from_pretrained(
-        "meta-llama/Llama-3.1-8B-Instruct",
-        device_map="cpu",
-        torch_dtype=torch.float32,
-        low_cpu_mem_usage=True
-    )
-
-# Load LoRA adapter
-model = PeftModel.from_pretrained(base_model, "path/to/lora/adapter")
-
-# Example inference
-def predict(text, max_length=256):
-    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
-    with torch.no_grad():
-        outputs = model.generate(
-            **inputs,
-            max_new_tokens=max_length,
-            temperature=0.7,
-            do_sample=True,
-            pad_token_id=tokenizer.eos_token_id
-        )
-    return tokenizer.decode(outputs[0], skip_special_tokens=True)
-
-# Test the model
-result = predict("Classify the sentiment of this financial text: 'The company's revenue increased by 15% this quarter.'")
-print(result)
-```
-
-### 2. Comprehensive Evaluation
-
-For testing all models on financial datasets:
-
-```bash
-# Navigate to testdata directory
-cd testdata
-
-# Run comprehensive evaluation (works on any platform)
-python comprehensive_evaluation.py
-
-# For Quest cluster users only:
-# sbatch submit_comprehensive_evaluation.sh
-```
-
-**Note**: The evaluation script automatically detects your environment and adjusts accordingly:
-- **GPU available**: Uses CUDA with quantization
-- **CPU only**: Uses CPU mode without quantization
-- **Memory constraints**: Automatically reduces batch size
-
-### 3. Individual Model Testing
-
-```python
-# Test specific financial tasks
-from testdata.comprehensive_evaluation import FinLoRAPredictor
-
-# Initialize predictor
-predictor = FinLoRAPredictor("path/to/model")
-
-# Load model
-predictor.load_model()
-
-# Test sentiment analysis
-result = predictor.predict("Analyze the sentiment of: 'Stock prices are declining rapidly.'", max_length=50)
-print(result)
-```
-
-### 4. RAG System Usage
-
-The project includes RAG knowledge bases for enhanced financial understanding:
-
-```python
-# Load RAG system
-from FinLoRA.rag.cfa_rag_system import CFARAGSystem
-
-# Initialize RAG system
-rag_system = CFARAGSystem()
-
-# Query CFA knowledge base
-query = "What are the key principles of portfolio management?"
-results = rag_system.query(query, top_k=5)
-
-# Use with LoRA models for enhanced responses
-enhanced_response = rag_system.generate_enhanced_response(query, model)
-```
-
-## Data Input Formats for Testing
-
-### 1. Financial Sentiment Analysis
-**Input Format:**
-```python
-text = "The company's quarterly earnings exceeded expectations by 20%."
-prompt = f"Classify the sentiment of this financial text as positive, negative, or neutral:\n\nText: {text}\n\nSentiment:"
-```
-
-**Expected Output:**
-- `"positive"` - for positive financial sentiment
-- `"negative"` - for negative financial sentiment  
-- `"neutral"` - for neutral financial sentiment
-
-**Test Examples:**
-- "Stock prices are soaring to new heights." → `positive`
-- "Revenue declined by 15% this quarter." → `negative`
-- "The company maintained stable performance." → `neutral`
-
-### 2. Named Entity Recognition
-**Input Format:**
-```python
-text = "Apple Inc. reported revenue of $394.3 billion in 2022."
-prompt = f"Extract financial entities from the following text:\n\nText: {text}\n\nEntities:"
-```
-
-**Expected Output:**
-- Company names, financial figures, dates, and financial terms
-- Structured entity extraction with context
-
-### 3. XBRL Processing
-**Input Format:**
-```python
-text = "Total assets: $1,234,567,890. Current assets: $456,789,123."
-prompt = f"Extract XBRL tags from the following financial statement:\n\nStatement: {text}\n\nXBRL Tags:"
-```
-
-**Expected Output:**
-- Structured XBRL tag extraction
-- Financial statement element identification
-
-### 4. CFA Knowledge Integration
-**Input Format:**
-```python
-question = "Explain the concept of weighted average cost of capital (WACC)."
-prompt = f"Answer this CFA-related question using your knowledge base:\n\nQuestion: {question}\n\nAnswer:"
-```
-
-**Expected Output:**
-- Comprehensive explanation with CFA knowledge
-- Structured financial concepts and formulas
-
-### 5. Headline Classification
-**Input Format:**
-```python
-headline = "Federal Reserve announces interest rate cut"
-prompt = f"Classify this financial headline:\n\nHeadline: {headline}\n\nClassification:"
-```
-
-**Expected Output:**
-- Financial news category classification
-- Market impact assessment
-
-## Running Without Quest GPU
-
-### Option 1: Local GPU Setup
-```bash
-# Check GPU availability
-nvidia-smi
-
-# Install CUDA toolkit (if not already installed)
-conda install cudatoolkit=11.8
-
-# Run evaluation with GPU
-cd testdata
-python comprehensive_evaluation.py
-```
-
-### Option 2: CPU-Only Mode
-```bash
-# Run evaluation on CPU (slower but works without GPU)
-cd testdata
-python comprehensive_evaluation.py
-```
-
-The evaluation script will automatically detect CPU mode and adjust settings accordingly.
-
-### Option 3: Cloud Platforms
-
-#### Google Colab
-```python
-# Upload the project files to Colab
-# Then run:
-!cd testdata && python comprehensive_evaluation.py
-```
-
-#### AWS EC2 / Azure / Local GPU
-```bash
-# Install NVIDIA drivers and CUDA toolkit first
-# Then follow the environment setup above
-cd testdata
-python comprehensive_evaluation.py
-```
-
-#### Hugging Face Spaces
-```python
-# Deploy as a web application
-# The model will run on Hugging Face's infrastructure
-```
-
-### Option 4: Docker with GPU Support
-```bash
-# Build Docker image
-docker build -t finlora .
-
-# Run with GPU support
-docker run --gpus all -it finlora python comprehensive_evaluation.py
-
-# Run without GPU (CPU mode)
-docker run -it finlora python comprehensive_evaluation.py
-```
-
-### Performance Expectations
-
-| Environment | Expected Speed | Memory Usage | Notes |
-|-------------|----------------|--------------|-------|
-| Quest H100 | Fastest | ~16GB | Original development environment |
-| Local GPU (RTX 4090) | Fast | ~12GB | High-end consumer GPU |
-| Google Colab T4 | Medium | ~8GB | Free tier available |
-| Google Colab V100 | Fast | ~16GB | Pro tier required |
-| CPU Only | Slow | ~32GB | Requires significant RAM |
-| AWS/Azure GPU | Fast | Variable | Depends on instance type |
-
-## Evaluation Results
-
-The models have been evaluated on multiple financial datasets:
-
-### Performance Metrics
-- **Financial Phrasebank**: F1=0.333, Accuracy=0.500
-- **NER Classification**: F1=0.889, Accuracy=0.800
-- **Headline Classification**: F1=0.697, Accuracy=0.700
-- **XBRL Tag Extraction**: Accuracy=0.200
-- **FIQA Sentiment Analysis**: F1=0.727, Accuracy=0.700
-
-### Dataset Coverage
-- BloombergGPT tasks: Financial Phrasebank, FIQA SA, Headline, NER, ConvFinQA
-- XBRL tasks: Tag extraction, Value extraction, Formula construction, Formula calculation
-- CFA integration: Level 1 and Level 2 knowledge base
-
-## File Structure
-
-```
-FinLoRA/
-├── lora_adapters/          # Trained LoRA adapters
-│   ├── 8bits_r8/          # 8-bit quantized models
-│   ├── 4bits_r4/          # 4-bit quantized models
-│   └── fp16_r8/           # Full precision models
-├── testdata/              # Evaluation scripts and data
-│   ├── comprehensive_evaluation.py
-│   ├── incremental_evaluation.py
-│   └── submit_*.sh       # SLURM submission scripts
-├── rag/                   # RAG system components
-├── data/                  # Training and test data
-├── environment.yml        # Conda environment specification
-└── requirements.txt       # Python dependencies
-```
-
-## Environment Verification
-
-Before running the models, verify your environment setup:
-
-```python
-# Environment verification script
-import torch
-import transformers
-import peft
-import datasets
-import sys
-
-print("=== Environment Verification ===")
-print(f"Python version: {sys.version}")
-print(f"PyTorch version: {torch.__version__}")
-print(f"CUDA available: {torch.cuda.is_available()}")
-print(f"CUDA version: {torch.version.cuda}")
-print(f"Transformers version: {transformers.__version__}")
-print(f"PEFT version: {peft.__version__}")
-print(f"Datasets version: {datasets.__version__}")
-
-if torch.cuda.is_available():
-    print(f"GPU count: {torch.cuda.device_count()}")
-    for i in range(torch.cuda.device_count()):
-        print(f"GPU {i}: {torch.cuda.get_device_name(i)}")
-        print(f"GPU {i} memory: {torch.cuda.get_device_properties(i).total_memory / 1e9:.1f} GB")
-else:
-    print("Running in CPU mode")
-
-print("=== Model Path Verification ===")
-import os
-model_paths = [
-    "FinLoRA/lora_adapters/8bits_r8/sentiment_llama_3_1_8b_8bits_r8",
-    "FinLoRA/lora_adapters/8bits_r8/ner_llama_3_1_8b_8bits_r8",
-    "FinLoRA/lora_adapters/8bits_r8/headline_llama_3_1_8b_8bits_r8"
-]
-
-for path in model_paths:
-    exists = os.path.exists(path)
-    print(f"{path}: {'✓' if exists else '✗'}")
-```
-
-## Troubleshooting
-
-### Common Issues
-
-1. **CUDA Out of Memory**
-   ```python
-   # Reduce batch size or use gradient checkpointing
-   model.gradient_checkpointing_enable()
-   
-   # Or use CPU mode
-   device = "cpu"
-   ```
-
-2. **Model Loading Errors**
-   ```python
-   # Check model path and permissions
-   import os
-   print(os.path.exists("path/to/model"))
-   
-   # Check if base model can be loaded
-   from transformers import AutoTokenizer
-   tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
-   ```
-
-3. **Dependency Conflicts**
-   ```bash
-   # Create fresh environment
-   conda create -n finlora_new python=3.11
-   conda activate finlora_new
-   pip install -r requirements.txt
-   ```
-
-4. **CPU Mode Issues**
-   ```python
-   # Ensure CPU mode is properly configured
-   import torch
-   torch.set_default_device("cpu")
-   
-   # Use low memory mode
-   base_model = AutoModelForCausalLM.from_pretrained(
-       "meta-llama/Llama-3.1-8B-Instruct",
-       device_map="cpu",
-       torch_dtype=torch.float32,
-       low_cpu_mem_usage=True
-   )
-   ```
-
-### Performance Optimization
-
-1. **Memory Optimization**
-   - Use 8-bit or 4-bit quantization
-   - Enable gradient checkpointing
-   - Use DeepSpeed for large models
-
-2. **Speed Optimization**
-   - Use GPU acceleration
-   - Batch processing
-   - Model caching
-
-## Citation
-
-If you use this work, please cite:
-
-```bibtex
-@article{finlora2024,
-  title={FinLoRA: Financial Large Language Models with LoRA Adaptation},
-  author={Your Name},
-  journal={Financial AI Conference},
-  year={2024}
-}
-```
-
-## License
-
-This project is licensed under the MIT License - see the LICENSE file for details.
-
-## Contact
-
-For questions and support, please contact:
-- Email: your.email@domain.com
-- GitHub Issues: [Project Repository](https://github.com/your-repo/finlora)
-
-## Acknowledgments
-
-- Meta AI for the Llama-3.1-8B-Instruct base model
-- Hugging Face for the transformers library
-- Microsoft for the LoRA adaptation technique
-- Quest cluster at Northwestern University for computational resources