File size: 3,859 Bytes
3b4b411 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 | ---
base_model: Qwen/Qwen3-4B-Base
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- base_model:adapter:Qwen/Qwen3-4B-Base
- peft
- lora
- text-to-sql
- sql
- supervised-fine-tuning
---
# SQL-R1-SFT
SQL-R1-SFT is a LoRA adapter for
[Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base), trained to
translate a SQLite schema, optional evidence, and a natural-language question
into exactly one read-only SQL query.
This is an adapter-only repository. The Qwen3-4B-Base weights are required for
inference.
## Training
- **Method:** completion-only supervised fine-tuning
- **Training data:** filtered BIRD and Spider training examples
- **Records:** 12,976 train / 608 database-disjoint validation
- **LoRA:** rank 32, alpha 64, dropout 0.05
- **Target modules:** all attention and MLP projections
- **Trainable parameters:** 66.1M (1.62%)
- **Precision:** BF16
- **Epochs:** 1
- **Learning rate:** 1e-4
- **Effective batch size:** 16
Only target SQL tokens contribute to the loss. Schema, evidence, and question
tokens are masked. The final assistant completion uses Qwen3-4B-Base's native
`<|endoftext|>` EOS rather than the ChatML turn separator.
## Evaluation
Execution accuracy (EX) and execution-valid rate were measured with a common
read-only SQLite evaluator on the public development sets.
| Model | BIRD Dev EX | BIRD valid | Spider Dev EX | Spider valid |
|---|---:|---:|---:|---:|
| Qwen3-4B-Base | 23.21% | 55.61% | 55.51% | 77.66% |
| SQL-R1-SFT | **42.37%** | **87.29%** | **77.27%** | **96.62%** |
These are public Dev results produced by the SQL-R1 evaluator, not official
hidden-test leaderboard submissions.
## Usage
```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_id = "Qwen/Qwen3-4B-Base"
repo_id = "j2521402/SQL-R1"
adapter_subfolder = "sft"
tokenizer = AutoTokenizer.from_pretrained(
repo_id,
subfolder=adapter_subfolder,
)
base_model = AutoModelForCausalLM.from_pretrained(
base_id,
dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(
base_model,
repo_id,
subfolder=adapter_subfolder,
).eval()
messages = [
{
"role": "system",
"content": (
"You are a Text-to-SQL assistant. Given a SQLite database schema, "
"optional evidence, and a question, return exactly one read-only "
"SQLite query. Do not include explanations or Markdown fences."
),
},
{
"role": "user",
"content": (
'Database schema:\nTABLE "singer" ("id" INT, "name" TEXT);\n\n'
"Question:\nHow many singers are there?"
),
},
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
eos_token_id=[
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|im_end|>"),
],
)
sql = tokenizer.decode(
outputs[0, inputs["input_ids"].shape[1]:],
skip_special_tokens=True,
).strip()
print(sql)
```
## Limitations
- Training and evaluation target SQLite and English Text-to-SQL benchmarks.
- Prompts contain schema metadata but not database row contents.
- The adapter does not implement tool calling or an autonomous SQL agent.
- Generated SQL should be validated and executed through a read-only,
resource-limited database connection.
## Project
Training code, preprocessing, evaluator, and reproducibility details:
[j2521402/SQL-R1](https://github.com/j2521402/SQL-R1).
### Framework versions
- PyTorch 2.8.0+cu128
- Transformers 5.14.1
- PEFT 0.19.1
|