sft/README.md

---
base_model: Qwen/Qwen3-4B-Base
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- base_model:adapter:Qwen/Qwen3-4B-Base
- peft
- lora
- text-to-sql
- sql
- supervised-fine-tuning
---

# SQL-R1-SFT

SQL-R1-SFT is a LoRA adapter for
[Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base), trained to
translate a SQLite schema, optional evidence, and a natural-language question
into exactly one read-only SQL query.

This is an adapter-only repository. The Qwen3-4B-Base weights are required for
inference.

## Training

- **Method:** completion-only supervised fine-tuning
- **Training data:** filtered BIRD and Spider training examples
- **Records:** 12,976 train / 608 database-disjoint validation
- **LoRA:** rank 32, alpha 64, dropout 0.05
- **Target modules:** all attention and MLP projections
- **Trainable parameters:** 66.1M (1.62%)
- **Precision:** BF16
- **Epochs:** 1
- **Learning rate:** 1e-4
- **Effective batch size:** 16

Only target SQL tokens contribute to the loss. Schema, evidence, and question
tokens are masked. The final assistant completion uses Qwen3-4B-Base's native
`<|endoftext|>` EOS rather than the ChatML turn separator.

## Evaluation

Execution accuracy (EX) and execution-valid rate were measured with a common
read-only SQLite evaluator on the public development sets.

| Model | BIRD Dev EX | BIRD valid | Spider Dev EX | Spider valid |
|---|---:|---:|---:|---:|
| Qwen3-4B-Base | 23.21% | 55.61% | 55.51% | 77.66% |
| SQL-R1-SFT | **42.37%** | **87.29%** | **77.27%** | **96.62%** |

These are public Dev results produced by the SQL-R1 evaluator, not official
hidden-test leaderboard submissions.

## Usage

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-4B-Base"
repo_id = "j2521402/SQL-R1"
adapter_subfolder = "sft"

tokenizer = AutoTokenizer.from_pretrained(
    repo_id,
    subfolder=adapter_subfolder,
)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    base_model,
    repo_id,
    subfolder=adapter_subfolder,
).eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are a Text-to-SQL assistant. Given a SQLite database schema, "
            "optional evidence, and a question, return exactly one read-only "
            "SQLite query. Do not include explanations or Markdown fences."
        ),
    },
    {
        "role": "user",
        "content": (
            'Database schema:\nTABLE "singer" ("id" INT, "name" TEXT);\n\n'
            "Question:\nHow many singers are there?"
        ),
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        eos_token_id=[
            tokenizer.eos_token_id,
            tokenizer.convert_tokens_to_ids("<|im_end|>"),
        ],
    )

sql = tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
).strip()
print(sql)
```

## Limitations

- Training and evaluation target SQLite and English Text-to-SQL benchmarks.
- Prompts contain schema metadata but not database row contents.
- The adapter does not implement tool calling or an autonomous SQL agent.
- Generated SQL should be validated and executed through a read-only,
  resource-limited database connection.

## Project

Training code, preprocessing, evaluator, and reproducibility details:
[j2521402/SQL-R1](https://github.com/j2521402/SQL-R1).

### Framework versions

- PyTorch 2.8.0+cu128
- Transformers 5.14.1
- PEFT 0.19.1