grpo-hard/README.md

---
base_model: Qwen/Qwen3-4B-Base
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- base_model:adapter:Qwen/Qwen3-4B-Base
- peft
- lora
- text-to-sql
- sql
- grpo
- rlvr
- curriculum-learning
- trl
---

# SQL-R1-GRPO-Hard

SQL-R1-GRPO-Hard is a curriculum-learning ablation of SQL-R1. It continues
GRPO training from SQL-R1-GRPO on BIRD moderate and challenging examples mixed
with stratified replay data.

This repository contains a LoRA adapter for
[`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base).
For the best overall balance across BIRD and Spider, use SQL-R1-GRPO; this
checkpoint is published primarily for analysis and reproducibility.

## Curriculum data

- Total records: 2,522
- BIRD moderate/challenging focus records: 1,891
- Stratified replay records: 631
- Spider replay records: 316
- Validation database overlap: 0

The replay subset preserves easier examples and cross-dataset coverage while
the curriculum emphasizes harder BIRD queries.

## Training

- Starting checkpoint: SQL-R1-GRPO
- Optimization steps: 300
- Candidates per prompt: 4
- Approximate generated rollouts: 1,200
- Learning rate: 2e-6
- Precision: BF16
- Reward: SQL validity, read-only safety, executability, and result equivalence
  under parallel SQLite execution

Gold SQL is used only to compute verifiable execution rewards and is never
included in the model prompt.

## Evaluation

| Checkpoint | BIRD Dev EX | BIRD execution-valid | Spider Dev EX | Spider execution-valid |
|---|---:|---:|---:|---:|
| SQL-R1-GRPO | 42.37% | 87.35% | **79.59%** | 97.00% |
| **SQL-R1-GRPO-Hard** | **42.50%** | **88.79%** | 78.92% | **97.20%** |

Hard-sample curriculum training slightly improves BIRD execution accuracy and
execution validity, but Spider EX decreases mildly. This result illustrates the
specialization–retention trade-off and motivates the replay ablation.

These are project-side measurements on the complete public development splits,
not hidden-test leaderboard submissions.

## Usage

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "Qwen/Qwen3-4B-Base"
repo_id = "j2521402/SQL-R1"
adapter_subfolder = "grpo-hard"

tokenizer = AutoTokenizer.from_pretrained(
    repo_id,
    subfolder=adapter_subfolder,
)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    model,
    repo_id,
    subfolder=adapter_subfolder,
)
model.eval()
```

Use the same SQL-only ChatML prompt format documented in the
[SQL-R1 repository](https://github.com/j2521402/SQL-R1).

## Limitations

- This checkpoint deliberately specializes toward harder BIRD examples and
  shows mild Spider forgetting.
- Training and evaluation focus on English Text-to-SQL and SQLite.
- Database contents are not automatically retrieved for the prompt.
- Generated SQL should be executed only in a read-only sandbox with resource
  limits.
- This is an ablation checkpoint rather than the recommended general model.

## Project

Training, curriculum-data construction, reward, and evaluation code:
[j2521402/SQL-R1](https://github.com/j2521402/SQL-R1).

Core libraries: PyTorch 2.8.0, Transformers 5.14.1, PEFT 0.19.1,
TRL 1.8.0, Datasets 5.0.0, and Accelerate 1.14.0.