---
language: 
- en
- zh
tags:
- llama
- finance
- lora
- instruction-tuning
license: apache-2.0
---

# Llama for Finance

A financial domain instruction-tuned Llama-3 model using LoRA on the Finance-Instruct-500k dataset.

## Model Details

- **Base Model:** meta-llama/Meta-Llama-3.1-8B-Instruct
- **Training:** LoRA fine-tuning
- **Domain:** Finance, Economics, Investment
- **Language:** English, Chinese
- **Context Length:** 4096 tokens
- **Training Data:** Finance-Instruct-500k
- **Evaluation:** Held-out test + FinanceBench

## Usage

This is a LoRA adapter for Llama-3. You need access to the base model.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
model = PeftModel.from_pretrained(base_model, "TimberGu/Llama_for_Finance")
tokenizer = AutoTokenizer.from_pretrained("TimberGu/Llama_for_Finance")
```

## Evaluation

See Evaluation.ipynb for:
- BLEU/ROUGE/Perplexity on held-out data
- FinanceBench benchmark results