Kakarot2812/qwen2.5-coder-1.5b-text2sql-qlora

Qwen2.5-Coder-1.5B Text-to-SQL QLoRA (Full Dataset)

A parameter-efficient fine-tuning model for translating natural language questions and database schemas into SQL queries, trained on the full dipanjanS/text2sql-dataset.

Base model

Qwen/Qwen2.5-Coder-1.5B-Instruct

Dataset

dipanjanS/text2sql-dataset

Trained on 78,077 examples and evaluated on 500 held-out examples using seed 42.

Training Configuration & Anti-Overfitting Measures

• Method: QLoRA (4-bit quantization) with Assistant-Only Loss (train_on_responses_only)
• Framework: Unsloth and TRL
• Epochs: 1 (with Early Stopping patience = 3)
• Effective Batch Size: 32 (per_device_train_batch_size=8, gradient_accumulation_steps=4)
• LoRA Rank: 16 (lora_alpha=32, use_rslora=True)
• Learning Rate: 1e-4 with Cosine Scheduler & 0.01 Weight Decay
• Maximum Sequence Length: 1,024 tokens

Evaluation

Exact-match comparison on 500 held-out examples:

Model	Exact matches	Exact-match accuracy
Base model	20/500	4.00%
Fine-tuned model	405/500	81.00%