Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,115 @@
+---
+license: mit
+language:
+- en
+tags:
+- finance
+- nlp
+- lora
+- llama
+- sentiment-analysis
+- named-entity-recognition
+- xbrl
+- financial-analysis
+pipeline_tag: text-generation
+---
+
+# FinLoRA: Financial Large Language Models with LoRA Adaptation
+
+## Overview
+
+FinLoRA is a comprehensive framework for fine-tuning large language models on financial tasks using Low-Rank Adaptation (LoRA). This repository contains trained LoRA adapters for various financial NLP tasks including sentiment analysis, named entity recognition, headline classification, XBRL processing, and CFA knowledge integration.
+
+## Model Architecture
+
+- **Base Model**: Meta-Llama-3.1-8B-Instruct
+- **Adaptation Method**: LoRA (Low-Rank Adaptation)
+- **Quantization**: 8-bit quantization for efficient inference
+- **Tasks**: Financial sentiment analysis, NER, classification, XBRL processing, CFA knowledge integration
+
+## Available Models
+
+### Core Financial Models
+- `sentiment_llama_3_1_8b_8bits_r8` - Financial sentiment analysis
+- `ner_llama_3_1_8b_8bits_r8` - Named entity recognition
+- `headline_llama_3_1_8b_8bits_r8` - Financial headline classification
+- `xbrl_extract_llama_3_1_8b_8bits_r8` - XBRL tag extraction
+- `financebench_llama_3_1_8b_8bits_r8` - Comprehensive financial benchmark
+- `finer_llama_3_1_8b_8bits_r8` - Financial NER
+- `formula_llama_3_1_8b_8bits_r8` - Financial formula processing
+
+## Quick Start
+
+### 1. Install Dependencies
+```bash
+pip install transformers torch peft bitsandbytes accelerate datasets
+```
+
+### 2. Load and Use a Model
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
+from peft import PeftModel
+import torch
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
+if tokenizer.pad_token is None:
+    tokenizer.pad_token = tokenizer.eos_token
+
+# Load model with quantization
+bnb_config = BitsAndBytesConfig(load_in_8bit=True)
+base_model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Llama-3.1-8B-Instruct",
+    quantization_config=bnb_config,
+    device_map="auto"
+)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "models/sentiment_llama_3_1_8b_8bits_r8")
+
+# Generate prediction
+def predict(text):
+    inputs = tokenizer(text, return_tensors="pt")
+    with torch.no_grad():
+        outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
+    return tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+# Test
+result = predict("Classify sentiment: 'The stock market is performing well today.'")
+print(result)
+```
+
+### 3. Run Evaluation
+```bash
+python comprehensive_evaluation.py
+```
+
+### 4. Run Inference
+```bash
+python inference.py
+```
+
+## Performance
+
+The models have been evaluated on multiple financial datasets:
+
+- **Financial Phrasebank**: F1=0.333, Accuracy=0.500
+- **NER Classification**: F1=0.889, Accuracy=0.800
+- **Headline Classification**: F1=0.697, Accuracy=0.700
+- **XBRL Tag Extraction**: Accuracy=0.200
+- **FIQA Sentiment Analysis**: F1=0.727, Accuracy=0.700
+
+## Citation
+
+```bibtex
+@article{finlora2024,
+  title={FinLoRA: Financial Large Language Models with LoRA Adaptation},
+  author={Your Name},
+  journal={Financial AI Conference},
+  year={2024}
+}
+```
+
+## License
+
+This project is licensed under the MIT License.