grpo/README.md

---
base_model: Qwen/Qwen3-4B-Base
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- base_model:adapter:Qwen/Qwen3-4B-Base
- peft
- lora
- text-to-sql
- sql
- grpo
- rlvr
- trl
---

# SQL-R1-GRPO

SQL-R1-GRPO is the main SQL-R1 checkpoint: a LoRA adapter trained on
Qwen3-4B-Base with completion-only supervised fine-tuning followed by GRPO
using verifiable SQLite execution feedback.

The model receives a database schema, optional evidence, and a natural-language
question, then generates one read-only SQLite query. This repository contains
the adapter only; load it together with
[`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base).

## Training

GRPO starts from the SQL-R1-SFT adapter and optimizes generated SQL with a
layered execution reward:

| Outcome | Reward |
|---|---:|
| Empty output | -1.00 |
| Invalid or non-read-only SQL | -0.75 |
| SQL execution error | -0.25 |
| Truncated execution result | -0.10 |
| Executable but incorrect result | +0.25 |
| Execution-equivalent to the gold query | +1.00 |

A low-weight format reward additionally encourages concise SQL-only output.
Gold SQL and gold execution results are used only by the reward function and
are never included in the model prompt.

- RL training records: 12,885
- Optimization steps: 300
- Candidates per prompt: 4
- Approximate generated rollouts: 1,200
- Learning rate: 5e-6
- GRPO beta: 0.001
- Clipping epsilon: 0.2
- Precision: BF16
- Execution backend: parallel read-only SQLite workers

## Evaluation

Execution accuracy (EX) and execution-valid rate were measured on the complete
BIRD Dev and Spider Dev splits with the same greedy decoding and execution
pipeline at every stage.

| Checkpoint | BIRD Dev EX | BIRD execution-valid | Spider Dev EX | Spider execution-valid |
|---|---:|---:|---:|---:|
| SQL-R1-SFT | 42.37% | 87.29% | 77.27% | 96.62% |
| **SQL-R1-GRPO** | **42.37%** | **87.35%** | **79.59%** | **97.00%** |

GRPO improves Spider Dev EX by 2.32 percentage points over SFT while preserving
BIRD Dev EX. This is the recommended SQL-R1 checkpoint.

These numbers are project-side development-set measurements, not hidden-test
leaderboard submissions. Results from systems using prompting, database
contents, self-consistency, reranking, or larger proprietary models are not
directly comparable.

## Usage

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "Qwen/Qwen3-4B-Base"
repo_id = "j2521402/SQL-R1"
adapter_subfolder = "grpo"

tokenizer = AutoTokenizer.from_pretrained(
    repo_id,
    subfolder=adapter_subfolder,
)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    model,
    repo_id,
    subfolder=adapter_subfolder,
)
model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are a Text-to-SQL assistant. Given a SQLite database schema, "
            "optional evidence, and a question, return exactly one read-only "
            "SQLite query. Do not include explanations or Markdown fences."
        ),
    },
    {
        "role": "user",
        "content": (
            'Database schema:\nTABLE "singer" ("Singer_ID" INT, "Name" TEXT);\n\n'
            "Question:\nHow many singers do we have?"
        ),
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )

completion = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(completion, skip_special_tokens=True).strip())
```

## Limitations

- Training and evaluation focus on English Text-to-SQL and SQLite.
- Prompts contain database schemas and optional evidence, but do not perform
  automatic database-content retrieval.
- Execution equivalence is a strong but imperfect semantic correctness signal.
- The model may still generate incorrect or expensive queries; execute outputs
  in a read-only sandbox with time and row limits.
- This is a direct Text-to-SQL model, not a general tool-calling agent.

## Project

Training, data processing, reward, and evaluation code:
[j2521402/SQL-R1](https://github.com/j2521402/SQL-R1).

Core libraries: PyTorch 2.8.0, Transformers 5.14.1, PEFT 0.19.1,
TRL 1.8.0, Datasets 5.0.0, and Accelerate 1.14.0.