README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
- reinforcement-learning
- grpo
- text-to-sql
- multi-turn
- agent
- skyrl
language:
- en
pipeline_tag: text-generation
---

# Qwen2.5-Coder-7B-SkyRL-SQL

Qwen2.5-Coder-7B-Instruct trained as a **multi-turn SQL agent** with GRPO using [SkyRL](https://github.com/NovaSky-AI/SkyRL)'s SkyRL-SQL recipe. The model interacts with a real SQLite database over up to 6 turns: it probes the schema with exploratory queries, observes actual execution results (or errors), refines its understanding, and then commits a final answer.

## Method

- **Recipe**: [SkyRL-SQL](https://novasky-ai.notion.site/skyrl-sql) (`examples/train/text_to_sql`), GRPO with dual-clip policy loss, no KL
- **Training data**: [SkyRL-SQL-653](https://huggingface.co/datasets/NovaSky-AI/SkyRL-SQL-653-data-newfmt) — only 653 examples, executed against [OmniSQL](https://huggingface.co/datasets/seeklhy/OmniSQL-datasets) databases
- **Reward**: execution-result match against the gold query on the final answer (sparse, outcome-only)
- **Training**: 10 optimizer steps (2 epochs), batch 128 prompts × 5 samples, max 16k context
- **Hardware**: 4× NVIDIA L40 (48GB), FSDP + vLLM via SkyRL

## Results (held-out Spider, execution accuracy pass@1)

| step 0 (base model) | step 5 | **step 10 (this model)** |
|---|---|---|
| 38.4% | 57.9% | **69.8%** |

Average response length also dropped 528 → 364 tokens — the model learned to probe the database decisively rather than ramble.

## Prompt format

The model expects the SkyRL-SQL interaction format: a system/user prompt containing the database schema and question, with `<sql>...</sql>` blocks for exploratory queries (results are returned in `<observation>` messages) and a final `<solution>...</solution>` block. See the [SkyRL-SQL recipe](https://github.com/NovaSky-AI/SkyRL/tree/main/examples/train/text_to_sql) for the exact template and a runnable environment.