FinQA Qwen2.5-7B LoRA

A LoRA adapter for structured financial question answering over Natively Extended FinQA.

Base model:

Qwen/Qwen2.5-7B-Instruct

This repository contains PEFT adapter weights, not a standalone copy of the base model.

Official Result

Metric	Final Test
Execution Accuracy	66.17%
Program Accuracy	61.64%
Parse Success	97.82%
Average Latency	0.4793 s/example

Selected checkpoint:

epoch_1_adapter

The checkpoint was selected using development-set performance before final test evaluation.

Intended Input

This adapter was trained on full expanded FinQA context.

Training mapping:

question
+ pre_text
+ table
+ post_text
        ↓
FinQA program

Training target:

qa.program

The selected prompt is included in this repository as:

S2_financial_analyst_operation_reader.json

Do not replace the full-document input with RAG chunks when attempting to reproduce the reported LoRA result.

Adapter Configuration

rank:        64
alpha:       32
dropout:     0.05
bias:        none
task type:   CAUSAL_LM

Target modules:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

Additional confirmed training settings include:

learning rate:         1e-4
effective batch size:  32
training dtype:        bfloat16

Official Training Burden

For the study's controlled practical comparison:

Training time: 14.7519 hours
Training cost: approximately $27.88

Installation

pip install torch transformers peft accelerate safetensors

Load the Adapter

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"
ADAPTER = "Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-LoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)

model.eval()

Prepare the Input

Use the exact released S2_financial_analyst_operation_reader prompt.

The prompt should receive:

qa.question
pre_text
table
post_text

Do not expose:

qa.program
qa.exe_ans
qa.gold_inds

during normal dev or test inference.

Generate

Once rendered_prompt has been constructed with the official prompt:

inputs = tokenizer(
    rendered_prompt,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

generated = output[0, inputs["input_ids"].shape[-1]:]

text = tokenizer.decode(
    generated,
    skip_special_tokens=True,
)

print(text)

Expected Output

["subtract(", "5829", "5735", ")", "EOF"]

Evaluation

Parsed predictions should be converted to:

[
  {
    "id": "example-id",
    "predicted": [
      "subtract(",
      "5829",
      "5735",
      ")",
      "EOF"
    ]
  }
]

Then evaluate them with the original FinQA evaluator.

Related Repositories

Dataset:

Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-Dataset

Prompts, evaluator, results, and paper materials:

MarkPaulRosenthal/Accuracy-Is-Not-Enough-Practical-Financial-QA

Limitations

This adapter was trained specifically for FinQA-style numerical program generation.

It is not a general financial-advice model, and its reported accuracy should not be assumed to transfer directly to unrelated financial documents or tasks.