amer003100/qwen2.5-coder-3b-text-to-sql-lora

Qwen2.5-Coder-3B text-to-SQL (LoRA adapter)

LoRA adapter fine-tuned on gretelai/synthetic_text_to_sql. It takes a table schema and a question and returns only the SQL query. This is a demo/portfolio project.

Results

Execution accuracy on 200 questions from the dataset's test split (gold queries that run on SQLite and return rows). A prediction counts as correct if its result set matches the gold result set.

Model	Execution accuracy	Valid SQL
Base Qwen2.5-Coder-3B-Instruct	57.0%	89.5%
+ this adapter	69.5%	97.5%

Training

• 6,330 examples (gold SQL executable on SQLite), 1 epoch, 396 steps, on a Colab L4 (bf16)
• LoRA r=16, alpha=32, dropout 0.05, target_modules=all-linear
• Learning rate 1e-4 (cosine), effective batch size 16, loss on the SQL answer only

Prompt format

System: You are a SQL assistant. Given a table schema and a question, reply with ONLY the SQL query, nothing else. User: Schema:\n<CREATE TABLE statements>\n\nQuestion: <question>

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "Qwen/Qwen2.5-Coder-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "amer003100/qwen2.5-coder-3b-text-to-sql-lora")

Limitations

• Evaluated on SQLite only, on a small test set (200 questions); not tested on other dialects.
• Never run generated SQL with write permissions; use a read-only database user.
• Follow the license of the base model.