README.md

---
base_model: Qwen/Qwen2.5-7B-Instruct
library_name: peft
pipeline_tag: text-generation
license: mit
datasets:
  - Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-Dataset
tags:
  - peft
  - lora
  - finqa
  - finance
  - numerical-reasoning
---

# FinQA Qwen2.5-7B LoRA

A LoRA adapter for structured financial question answering over Natively Extended FinQA.

Base model:

`Qwen/Qwen2.5-7B-Instruct`

This repository contains PEFT adapter weights, not a standalone copy of the base model.

## Official Result

| Metric | Final Test |
|---|---:|
| Execution Accuracy | **66.17%** |
| Program Accuracy | **61.64%** |
| Parse Success | **97.82%** |
| Average Latency | 0.4793 s/example |

Selected checkpoint:

`epoch_1_adapter`

The checkpoint was selected using development-set performance before final test evaluation.

## Intended Input

This adapter was trained on **full expanded FinQA context**.

Training mapping:

```text
question
+ pre_text
+ table
+ post_text
        ↓
FinQA program
````

Training target:

`qa.program`

The selected prompt is included in this repository as:

`S2_financial_analyst_operation_reader.json`

Do not replace the full-document input with RAG chunks when attempting to reproduce the reported LoRA result.

## Adapter Configuration

```text
rank:        64
alpha:       32
dropout:     0.05
bias:        none
task type:   CAUSAL_LM
```

Target modules:

```text
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
```

Additional confirmed training settings include:

```text
learning rate:         1e-4
effective batch size:  32
training dtype:        bfloat16
```

## Official Training Burden

For the study's controlled practical comparison:

```text
Training time: 14.7519 hours
Training cost: approximately $27.88
```

## Installation

```bash
pip install torch transformers peft accelerate safetensors
```

## Load the Adapter

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"
ADAPTER = "Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-LoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)

model.eval()
```

## Prepare the Input

Use the exact released `S2_financial_analyst_operation_reader` prompt.

The prompt should receive:

```text
qa.question
pre_text
table
post_text
```

Do not expose:

```text
qa.program
qa.exe_ans
qa.gold_inds
```

during normal dev or test inference.

## Generate

Once `rendered_prompt` has been constructed with the official prompt:

```python
inputs = tokenizer(
    rendered_prompt,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

generated = output[0, inputs["input_ids"].shape[-1]:]

text = tokenizer.decode(
    generated,
    skip_special_tokens=True,
)

print(text)
```

## Expected Output

```json
["subtract(", "5829", "5735", ")", "EOF"]
```

## Evaluation

Parsed predictions should be converted to:

```json
[
  {
    "id": "example-id",
    "predicted": [
      "subtract(",
      "5829",
      "5735",
      ")",
      "EOF"
    ]
  }
]
```

Then evaluate them with the original FinQA evaluator.

## Related Repositories

Dataset:

`Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-Dataset`

Prompts, evaluator, results, and paper materials:

`MarkPaulRosenthal/Accuracy-Is-Not-Enough-Practical-Financial-QA`

## Limitations

This adapter was trained specifically for FinQA-style numerical program generation.

It is not a general financial-advice model, and its reported accuracy should not be assumed to transfer directly to unrelated financial documents or tasks.

## License

The adapter is released under the MIT License.

The Qwen base model remains subject to its own license.

The FinQA-derived training dataset is separately released under CC BY 4.0.