qwen35-text2sql
Fine-tuned Qwen3.5-9B for Text-to-SQL generation.
Model Details
| Property |
Value |
| Base Model |
Qwen/Qwen3.5-9B (9B params, hybrid DeltaNet+Attention) |
| Fine-tuning Method |
QLoRA (4-bit NF4 + LoRA rank=16, alpha=32) |
| Training Data |
Custom synthetic (81 examples, 8 domains) + b-mc2/sql-create-context (3,000 examples) |
| Training Hardware |
Nvidia L4 GPU (24GB VRAM) via Colab / Modal |
| Epochs |
3 |
| Effective Batch Size |
16 |
| Precision |
BFloat16 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"yesabhishek/qwen35-text2sql", torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("yesabhishek/qwen35-text2sql")
prompt = """<|im_start|>user
Given the following SQL schema:
CREATE TABLE employees (id INT, name VARCHAR, department VARCHAR, salary DECIMAL)
Generate a SQL query for: What is the average salary by department?<|im_end|>
<|im_start|>assistant
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Domains Covered
| Domain |
Examples |
Key SQL Features |
| E-Commerce |
15 |
JOINs, aggregations, window functions |
| HR |
12 |
Subqueries, HAVING, date arithmetic |
| Healthcare |
10 |
GROUP BY, CASE WHEN, completion rates |
| Education |
10 |
Capacity calculations, grade distributions |
| Finance |
10 |
EXISTS, defaults, transaction analysis |
| Logistics |
8 |
Inventory management, delivery metrics |
| Social Media |
8 |
Engagement metrics, trending analysis |
| Real Estate |
8 |
Price analytics, market time calculations |
Limitations
- Optimized for PostgreSQL syntax; may need adaptation for MySQL/SQLite
- Best with schemas structurally similar to training domains
- May struggle with very complex queries involving 5+ JOINs or recursive CTEs
- Not tested on non-English natural language inputs
- Thinking mode (
<think> tags) is disabled in this fine-tune for clean SQL output
