metadata
license: mit
language:
- en
metrics:
- rouge
base_model:
- meta-llama/Meta-Llama-3-8B
pipeline_tag: text-generation
tags:
- finance
AIPI 590 Large Language Models
Project 1 - Fine Tuning LLM
Files:
- model.ipynb
- notebook containing the code for fine tuning the Llama 3 model using QLoRa
- data/train.json
- json file containing the training set provided in the FINQA paper
- data/test.json
- json file containing the validation set provided in the FINQA paper
Process:
The focal property of interest is analysis financial documents for numerical reasoning. Specifically numerical reasoning over quarterly financial filings with the SEC. The Llama-3-8B model was chosen to fine tune using the QLoRa approach. This approach was chosen due to the paper's findings of a performance increase while utilizing minimal memory and hardware. The aggressive quantization seemed to significantly decreased training time while offering increased performance on financial analysis.
Evaluation:
Rouge Score
| ROUGE Score | Base Model | QLoRa Fine Tuned Model |
|---|---|---|
| ROUGE-1 | 0.05104785 | 0.25257307 |
| ROUGE-2 | 0.01158752 | 0.10479990 |
| ROUGE-L | 0.05104785 | 0.25175429 |
Collaborators:
- Keese Phillips