metadata
license: mit
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
- text-to-sql
- lora
- peft
- sql
- fine-tuned
datasets:
- b-mc2/sql-create-context
language:
- en
pipeline_tag: text-generation
Text-to-SQL: Fine-tuned Qwen2.5-0.5B
A 0.5B parameter language model fine-tuned with LoRA to convert natural language questions into SQL queries — outperforming a 70B general-purpose model on the same benchmark despite being 140x smaller.
🔥 Benchmark Results
| Model | Exact-Match Accuracy |
|---|---|
| Llama-3.3-70B (zero-shot, via Groq) | 32.0% |
| Qwen2.5-0.5B (this model, fine-tuned) | 84.0% |
Evaluated on a held-out set of 100 text-to-SQL questions, scored by exact match after SQL normalization.
🧠 Model Details
- Base model: Qwen/Qwen2.5-0.5B-Instruct
- Fine-tuning method: LoRA (r=16, alpha=32, target modules: q_proj, v_proj)
- Training data: b-mc2/sql-create-context (78,477 examples)
- Epochs: 3
- Hardware: Free Google Colab T4 GPU
🚀 Usage
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "javeria163/text-to-sql")
schema = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary INT)"
question = "What is the average salary in the Engineering department?"
prompt = (
"Task: Convert the question to SQL using the table schema.\n\n"
"Schema:\n" + schema + "\n\n"
"Question:\n" + question + "\n\n"
"SQL:\n"
)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True).split("SQL:")[-1].strip())
🎮 Try It Live
Interactive demo on Hugging Face Spaces — no setup required.
📊 Training
Loss decreased from 1.73 → 0.69 over 3 epochs, with final mean token accuracy of 82.9%.
🔗 Links
- GitHub Repository — full training code, eval scripts, and benchmark
- Live Demo