README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
language:
  - en
pipeline_tag: text-generation
tags:
  - text-to-sql
  - sql
  - bird
  - spider
  - agentic
  - qwen3.5
library_name: transformers
---

# sqrl-4b

**sqrl-4b** is a 4B-parameter agentic text-to-SQL model built on
[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). It answers natural-language
questions over SQLite databases by optionally *probing the database first* — running
read-only exploration queries to check value formats, join paths, and filters — before
committing to a final SQL answer.

It was trained in two stages:

1. **SFT distillation** — fine-tuned on ~10.2k execution-verified agentic trajectories
   sampled from a much stronger RL-trained 35B teacher (Qwen3.6-35B-A3B, 70.3% BIRD-dev EX)
   on cleaned BIRD-train + Spider-train questions. Only trajectories whose final SQL
   execution-matched the gold answer were kept.
2. **CISPO RL** — online reinforcement learning (group-relative advantage, group size 8)
   with execution-match reward on the same cleaned BIRD+Spider question pool, with
   async dynamic sampling. Best checkpoint at step 80.

## Results

All numbers measured with the agentic harness at temperature 0.7. BIRD-dev is the full
1534-question set; Spider-test is 2147 questions. `pass@1` is the mean single-sample
exec-accuracy; `vote@k` is execution-clustered majority voting over k samples (group
candidates by identical execution result set, return the largest cluster's query).

| Benchmark | pass@1 | vote@8 | vote@16 | vote@32 |
|---|---|---|---|---|
| BIRD-dev (1534) | **64.6%** | **68.8%** | 69.4% | 69.2% |
| Spider-test (2147) | **84.0%** | **85.9%** | 85.9% | 86.1% |

Family comparison on identical seeds (BIRD-dev pass@1 / vote@8):
[sqrl-4b](https://huggingface.co/feyninc/sqrl-4b) 64.6 / 68.8 ·
[sqrl-9b](https://huggingface.co/feyninc/sqrl-9b) 66.6 / 69.8 ·
[sqrl-35b-a3b](https://huggingface.co/feyninc/sqrl-35b-a3b) 68.7 / 70.6.

Notes from the measurement campaign:
- Voting buys the 4B **+4.3 points** over single-pass on BIRD (the largest voting gain
  in the family — smaller models keep more sample diversity). vote-of-4 captures ~85% of
  that gain at half the cost.
- **Voting saturates by k≈8–16.** Past that, more samples add nothing: BIRD peaks at
  vote@16 (69.4) and Spider is flat from vote@8 onward (~86%). Extra test-time compute is
  better spent on a smarter selector than on more votes.
- Base Qwen3.5-4B scores ~52% on the same harness; the pre-RL SFT checkpoint ~63.6% on
  BIRD (80.3% on Spider-test — RL lifts Spider single-shot to 84.0%).
- Vote unanimity is a strong free confidence signal: when all 8 samples agree the answer
  is right ~87% of the time; narrow/tied votes are right ~19-43% — useful for routing
  or escalation.

## How it works — the agentic protocol

The model is trained to emit **exactly one action block per turn**, after a brief
reasoning summary:

- `<sql> SELECT ... </sql>` — a read-only *exploration* query. Execute it against the
  database and feed the result back as a user turn wrapped in `<observation>` tags.
  The model continues (up to a step budget).
- `<answer> SELECT ... </answer>` — the final SQL. Execute and return.

Many questions are answered directly (zero exploration turns); the model learned to
explore only when seeing real data would change its answer (value-format checks,
ambiguous joins).

## Usage

### Serve with vLLM

```bash
vllm serve feyninc/sqrl-4b --served-model-name sqrl-4b \
  --data-parallel-size 4 --gpu-memory-utilization 0.90 --max-model-len 32768
```

> **Important:** do **not** enable a reasoning parser (e.g. `--reasoning-parser qwen3`).
> The model's post-`</think>` content carries the action protocol; stripping/rerouting
> the think block breaks it (measured ~12-point EX drop). Parse `message.content` —
> if it contains a `</think>`, take everything after it.

Recommended sampling: `temperature 0.7, top_p 0.95` (matches training); greedy also works.

### Prompt format

System prompt (fill `{schema}`, `{evidence}`, `{max_steps}`):

````text
<role>
You are an expert data analyst, fluent in SQL, with a meticulous eye for
matching a question's intent to the exact tables, columns, and stored value formats of
a database.
</role>

<task>
Translate the user's natural-language question into a SQL query that answers
it, using the database schema in <schema> and any domain hints in <evidence>. You can
run read-only queries against the database to inspect it before giving your final
answer.
</task>

<database_engine>
SQLite
</database_engine>

<schema>
{schema}
</schema>

<evidence>
{evidence}
</evidence>

<protocol>
Think through the problem internally first. Then, in your response, write a
BRIEF summary of your reasoning — 2-4 sentences stating which tables and columns are
relevant, the joins and filters, and any exact value-format detail. Be decisive: state
the plan once, do not second-guess or restate. End your message with EXACTLY ONE action
block (and nothing after it):

  <sql> a read-only query to inspect the database </sql>
     Use this when uncertain and you want to see real data before answering — e.g.
     confirm a value's exact stored format, check a filter actually matches rows,
     sanity-check an intermediate result, or verify a join. You will see the result
     rows and then continue.

  <answer> your final SQL query </answer>
     Use this once you are confident. It is executed and scored, and the task ends.

Examples:
  I'll check the exact county value before filtering, since the format may vary.
  <sql> SELECT DISTINCT `County Name` FROM frpm LIMIT 10 </sql>

  The schools are in the frpm table; I'll select `School Name` and filter `County Name`
  to 'Alameda'. The schema is clear, so I can answer directly.
  <answer> SELECT `School Name` FROM frpm WHERE `County Name` = 'Alameda' </answer>
</protocol>

<rules>
- Use only the tables and columns defined in <schema>.
- Quote identifiers containing spaces or special characters with backticks.
- Return exactly the columns the question asks for — no more, no fewer.
- Use the hints in <evidence> to resolve ambiguous terms and value encodings.
- If you already know the correct query, go straight to <answer> — investigating is
  optional; only run <sql> when seeing the data would actually change your answer.
- The action block must be the LAST thing in your message; do not discuss the tags
  themselves in your reasoning.
- You have at most {max_steps} <sql> steps; after that you must give <answer>.
</rules>
````

First user turn:

```text
Question: {question}

Reason about it, then give <sql> to investigate or <answer> to finish.
```

After executing an exploration `<sql>`, feed the result back as a **user** message:

```text
<observation>
{tab-separated result rows, or the error message}
</observation>
Continue with <sql> or <answer>.
```

`{schema}` is a readable dump of the SQLite schema (CREATE-TABLE-like listing of
tables/columns/types). `{evidence}` is the BIRD external-knowledge hint, or
`(none provided)`. `max_steps=5` was used in training. When the step budget is
exhausted, send `You must finish now. Give your <answer>.` as a user turn.

### Minimal driver loop

```python
import re
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

messages = [{"role": "system", "content": system_prompt},
            {"role": "user", "content": user0}]
for step in range(6):
    r = client.chat.completions.create(model="sqrl-4b", messages=messages,
                                       temperature=0.7, top_p=0.95, max_tokens=8192)
    content = r.choices[0].message.content
    if "</think>" in content:                       # strip inline think scratchpad
        content = content.split("</think>")[-1].strip()
    messages.append({"role": "assistant", "content": content})
    ans = re.findall(r"<answer>(.*?)</answer>", content, re.S)
    if ans:
        final_sql = ans[-1].strip(); break
    sql = re.findall(r"<sql>(.*?)</sql>", content, re.S)
    if not sql:
        break
    obs = run_readonly(sql[-1].strip())             # your sqlite executor
    messages.append({"role": "user",
                     "content": f"<observation>\n{obs}\n</observation>\n"
                                "Continue with <sql> or <answer>."})
```

## Checkpoint notes

- This repo contains the **full merged HF checkpoint** (LoRA rank 64 merged into the
  base). Qwen3.5-4B ships with tied embeddings; training touched only the output head,
  so this checkpoint **unties** them (`tie_word_embeddings=false`): `embed_tokens` is
  the pristine base matrix and `lm_head` carries the trained head. Loads with standard
  `transformers` / vLLM — no custom code beyond `trust_remote_code` if your
  transformers version requires it for Qwen3.5.
- The vision tower is inherited from the base model unchanged. The model is usable as a
  standard VL checkpoint, but SQL training was text-only.
- Text-to-SQL training data: BIRD-train and Spider-train, execution-filtered and
  semantically cleaned (3-model LLM-judge panel). BIRD-dev and Spider-test were never
  trained on.

## Intended use & limitations

Built for SQLite text-to-SQL with schema + optional evidence in context. Works best
with the exact prompt protocol above (it was both SFT'd and RL'd under it). Not tuned
for other SQL dialects; identifier quoting follows SQLite backtick conventions. As with
all text-to-SQL models, execute generated SQL read-only and validate before acting on
results.