metadata
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- qwen2.5
- lora
- text-to-sql
- sql
- natural-language-to-sql
- fine-tuned
language:
- en
datasets:
- YOUR_USERNAME/text-to-sql-dataset
metrics:
- perplexity
pipeline_tag: text-generation
Qwen2.5-7B-Instruct Fine-tuned for Text-to-SQL
This is a LoRA fine-tuned version of Qwen/Qwen2.5-7B-Instruct trained on natural language to SQL conversion task.
Model Details
- Base Model: Qwen/Qwen2.5-7B-Instruct
- Fine-tuning Method: LoRA (Low-Rank Adaptation)
- Task: Natural Language to SQL conversion
- Training Time: 6 minutes 15 seconds
- Hardware: 8x NVIDIA A100-80GB GPUs
Performance
Test Set Results (54 examples)
| Metric | Base Model | Fine-tuned | Improvement |
|---|---|---|---|
| Test Loss | 2.1301 | 0.4098 | 80.76% ⬆️ |
| Perplexity | 8.4155 | 1.5064 | 82.10% ⬆️ |
Training Details
- Training Epochs: 3
- Batch Size: 2 (per device)
- Gradient Accumulation: 8 steps
- Effective Batch Size: 16
- Learning Rate: 2e-4
- LoRA Rank (r): 16
- LoRA Alpha: 32
- Optimizer: paged_adamw_8bit
- LR Scheduler: cosine
Dataset
- Training Examples: 425
- Validation Examples: 54
- Test Examples: 54
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-7B-Instruct",
device_map="auto",
trust_remote_code=True
)
# Load LoRA adapter
model = PeftModel.from_pretrained(
base_model,
"YOUR_USERNAME/qwen2.5-7B-text-to-sql"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
# Generate SQL from natural language
prompt = "Show me all customers who made purchases last month"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_length=200)
sql = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(sql)
Training Code
The model was trained using the Hugging Face Transformers library with PEFT for LoRA fine-tuning.
Limitations
- Optimized for SQL generation tasks similar to the training distribution
- May require additional fine-tuning for domain-specific SQL dialects
- Best performance on queries similar to training examples
Citation
If you use this model, please cite:
@misc{qwen2.5-7B-text-to-sql,
author = {Your Name},
title = {Qwen2.5-7B-Instruct Fine-tuned for Text-to-SQL},
year = {2025},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/YOUR_USERNAME/qwen2.5-7B-text-to-sql}},
}
License
This model inherits the Apache 2.0 license from the base Qwen2.5 model.