ncfad/qwen2.5-coder-1.5b-finance-sql-lora

Qwen2.5-Coder-1.5B Finance SQL (LoRA)

A rank-16 QLoRA adapter for schema-grounded text-to-SQL, tuned for personal-finance questions and supplemented with general SQL structure and schema grounding.

• Base model: Qwen/Qwen2.5-Coder-1.5B-Instruct
• Adapter type: PEFT LoRA (r=16, alpha=32, dropout 0.05, all-linear)
• Prompt format: Qwen ChatML through tokenizer.apply_chat_template
• Project code: https://github.com/fadnc/Finance-Text-to-SQL

What it does

The model receives a natural-language question and a database schema, and returns one read-only SQL SELECT statement. Every training example carried its own DDL, so the adapter is not tied to the demo schema.

system:      You are a precise text-to-SQL assistant.
user:        Write one read-only SQL SELECT query for the following question
             using only the provided schema. Return SQL only.

             Question: What is the total spending by category?
             Schema: CREATE TABLE expenses (...);
assistant:   SELECT category, SUM(amount) ...;

Training data

Source	Rows used	Role
Gretel synthetic_text_to_sql, finance domain (repeated 3x)	2,835	Target domain
Spider (GRAST-SQL-Spider, pinned revision)	600	Joins, nesting, cross-domain structure
WikiSQL (pinned GitHub release, SHA-256 verified)	8,000	Single-table schema grounding

After removing 309 non-SELECT targets and 4 overlength rows, 11,122 completion-only examples were used. Validation used 1,976 rows; the Gretel finance test split (72 rows) was never trained on.

WikiSQL is the largest source by row count, but it is not the target domain: the finance rows are oversampled 3x, and every metric below is measured on Gretel finance. WikiSQL was fetched from the pinned Salesforce GitHub release (data.tar.bz2, SHA-256 755c728a…) rather than a Hub dataset repo, so the YAML datasets: list names only Gretel and Spider; the table above and the licensing section record the WikiSQL source.

Training procedure

Setting	Value
Method	QLoRA + TRL SFTTrainer, completion-only loss
Base quantization	4-bit NF4, bf16 compute
LoRA	r=16, alpha=32, dropout 0.05, all-linear
Trainable parameters	18,464,768 / 1,562,179,072 (1.18%)
Effective batch	4 (1 × gradient accumulation 4)
Epochs / steps	1 / 2,781
Learning rate	2e-4, cosine schedule, 3% warmup
Optimizer	Paged AdamW 8-bit
Max sequence length	512
Seed	42
Hardware	NVIDIA RTX 3050 Laptop GPU (4.3 GB), 5 h 20 min
Validation loss	0.1321
Validation token accuracy	95.88%

All prompts were rendered with the tokenizer chat template. Prompt tokens were masked from the loss; only SQL tokens were optimized. When a schema exceeded the context budget, DDL for tables referenced by the target SQL was kept first and optional tables were added only while the full target still fit.

Evaluation

Held-out Gretel finance test split: 72 examples, of which 48 have reference SQL that executes in their SQLite fixture. The other 24 are reported as gold execution errors and excluded from the execution denominator. Greedy decoding, 256 new tokens.

Model	Exact match	Valid SQL	Execution (n=48)	Test loss
Qwen2.5-Coder 1.5B base	12.5%	62.5%	52.1%	0.476
This adapter	22.2%	68.1%	56.2%	0.235

Execution accuracy compares result rows on each sample's own SQLite fixture: ordered when either query has a top-level ORDER BY, otherwise a multiset. Exact match is token-level normalization, not semantic equivalence. Valid SQL means EXPLAIN QUERY PLAN accepts the query against a schema-only database.

By complexity (adapter): basic SQL 30.2% exact / 57.1% execution; aggregation 10.0% / 62.5%; single join 11.1% / 57.1%; window functions 20.0% / 33.3%; subqueries and CTEs remain weak.

The 72-example test set is small and synthetic; one example is about 2.1 execution points. Treat these numbers as a development signal, not a production accuracy claim.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
adapter_id = "ncfad/qwen2.5-coder-1.5b-finance-sql-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
    base_id, dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

system = "You are a precise text-to-SQL assistant."
instruction = (
    "Write one read-only SQL SELECT query for the following question using only "
    "the provided schema. Return SQL only."
)
schema = (
    "CREATE TABLE expenses (expense_id INT PRIMARY KEY, user_id INT, "
    "category TEXT, amount DECIMAL, date DATE, description TEXT);"
)
question = "What is the total spending by category?"

messages = [
    {"role": "system", "content": system},
    {
        "role": "user",
        "content": f"{instruction}\n\nQuestion: {question}\nSchema: {schema}",
    },
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Always validate generated SQL before execution. The project code includes a read-only guard (EXPLAIN QUERY PLAN against the schema), write-statement rejection, and a conservative demo fallback.

Files

File	Purpose
adapter_config.json	LoRA configuration and base-model reference
adapter_model.safetensors	Trained adapter weights (36 MB)
tokenizer.json, tokenizer_config.json, chat_template.jinja	Tokenizer and ChatML template
README.md	This model card

Intended use and limitations

Intended for research and prototyping on schema-grounded SQL generation, especially personal-finance questions.

• The training data is synthetic; results do not establish real-world accuracy.
• The holdout is 72 rows with 48 executable reference queries.
• 68.1% schema-valid does not mean 68.1% correct; execution accuracy is 56.2%.
• CTEs, subqueries, and window functions are underrepresented and perform poorly.
• The adapter returns SQL text only. It does not execute queries or enforce authorization, tenant isolation, timeouts, or row limits.
• Do not use generated SQL on production data without review and a read-only execution sandbox.

Licensing and attribution

No additional license is asserted over the adapter weights. Users must comply with the upstream terms:

• Base model: Qwen2.5-Coder-1.5B-Instruct (Apache-2.0).
• Gretel synthetic_text_to_sql: Apache-2.0.
• Spider / GRAST-SQL redistribution: CC-BY-SA-4.0. The share-alike term may apply to derivatives; review before redistribution or commercial use.
• WikiSQL: Wikipedia-derived data; the Salesforce code is BSD-3-Clause.

Project code is MIT. See the repository for evaluation artifacts, training configuration, and the serving stack.