base_model: Qwen/Qwen3-4B-Base
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- base_model:adapter:Qwen/Qwen3-4B-Base
- peft
- lora
- text-to-sql
- sql
- grpo
- rlvr
- curriculum-learning
- trl
SQL-R1-GRPO-Hard
SQL-R1-GRPO-Hard is a curriculum-learning ablation of SQL-R1. It continues GRPO training from SQL-R1-GRPO on BIRD moderate and challenging examples mixed with stratified replay data.
This repository contains a LoRA adapter for
Qwen/Qwen3-4B-Base.
For the best overall balance across BIRD and Spider, use SQL-R1-GRPO; this
checkpoint is published primarily for analysis and reproducibility.
Curriculum data
- Total records: 2,522
- BIRD moderate/challenging focus records: 1,891
- Stratified replay records: 631
- Spider replay records: 316
- Validation database overlap: 0
The replay subset preserves easier examples and cross-dataset coverage while the curriculum emphasizes harder BIRD queries.
Training
- Starting checkpoint: SQL-R1-GRPO
- Optimization steps: 300
- Candidates per prompt: 4
- Approximate generated rollouts: 1,200
- Learning rate: 2e-6
- Precision: BF16
- Reward: SQL validity, read-only safety, executability, and result equivalence under parallel SQLite execution
Gold SQL is used only to compute verifiable execution rewards and is never included in the model prompt.
Evaluation
| Checkpoint | BIRD Dev EX | BIRD execution-valid | Spider Dev EX | Spider execution-valid |
|---|---|---|---|---|
| SQL-R1-GRPO | 42.37% | 87.35% | 79.59% | 97.00% |
| SQL-R1-GRPO-Hard | 42.50% | 88.79% | 78.92% | 97.20% |
Hard-sample curriculum training slightly improves BIRD execution accuracy and execution validity, but Spider EX decreases mildly. This result illustrates the specialization–retention trade-off and motivates the replay ablation.
These are project-side measurements on the complete public development splits, not hidden-test leaderboard submissions.
Usage
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model_id = "Qwen/Qwen3-4B-Base"
repo_id = "j2521402/SQL-R1"
adapter_subfolder = "grpo-hard"
tokenizer = AutoTokenizer.from_pretrained(
repo_id,
subfolder=adapter_subfolder,
)
model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(
model,
repo_id,
subfolder=adapter_subfolder,
)
model.eval()
Use the same SQL-only ChatML prompt format documented in the SQL-R1 repository.
Limitations
- This checkpoint deliberately specializes toward harder BIRD examples and shows mild Spider forgetting.
- Training and evaluation focus on English Text-to-SQL and SQLite.
- Database contents are not automatically retrieved for the prompt.
- Generated SQL should be executed only in a read-only sandbox with resource limits.
- This is an ablation checkpoint rather than the recommended general model.
Project
Training, curriculum-data construction, reward, and evaluation code: j2521402/SQL-R1.
Core libraries: PyTorch 2.8.0, Transformers 5.14.1, PEFT 0.19.1, TRL 1.8.0, Datasets 5.0.0, and Accelerate 1.14.0.