| --- |
| base_model: Qwen/Qwen3.5-4B |
| library_name: peft |
| pipeline_tag: text-generation |
| datasets: |
| - spider |
| tags: |
| - base_model:adapter:Qwen/Qwen3.5-4B |
| - lora |
| - text-to-sql |
| - tool-calling |
| - unsloth |
| --- |
| |
| # Qwen3.5-4B Spider text-to-SQL LoRA |
|
|
| A LoRA adapter for [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) that writes SQLite queries |
| from English questions. It was trained on tool-calling traces built from the |
| [Spider](https://yale-lily.github.io/spider) training set. |
|
|
| Code to rebuild the data, retrain, and run the benchmarks: |
| [Self-healing-Text2SQL-Agent](https://github.com/mehul79/Self-healing-Text2SQL-Agent) (`training/` and `backend/eval/`). |
|
|
| ## Results |
|
|
| Spider dev, execution accuracy (%), all 1,034 questions, official Spider evaluator. |
| Both models run inside the Self-healing Text2SQL harness (LangGraph: generate, validate, execute, repair), temperature 0. |
|
|
| | Model | Easy | Medium | Hard | Extra | All | |
| |---|---|---|---|---|---| |
| | Qwen3.5-4B base, first attempt | 87.1 | 78.0 | 56.9 | 45.2 | 71.4 | |
| | Qwen3.5-4B base, with repair | 86.7 | 80.5 | 57.5 | 50.0 | 73.2 | |
| | **This adapter, first attempt** | 89.9 | 77.4 | 74.1 | 51.8 | **75.7** | |
| | **This adapter, with repair** | 90.3 | 80.0 | 74.7 | 53.0 | **77.3** | |
|
|
| Most of the gain is on hard questions (+17 points). |
|
|
| ## Training |
|
|
| Every training example has the same three-step tool-calling shape: |
|
|
| 1. `get_schema` returns the database's `CREATE TABLE` statements. |
| 2. `execute_sql` runs the gold query and returns up to 10 rows. |
| 3. `submit_answer` submits the gold query. |
|
|
| The traces come from Spider train. Gold queries that failed to run were dropped, leaving 6,797 traces. |
| 200 were held out for validation (seed 42). Loss is computed only on the assistant turns, after an empty |
| `<think></think>` block, so the model is trained with thinking off. |
|
|
| | Setting | Value | |
| |---|---| |
| | Method | LoRA, bf16 (Unsloth) | |
| | Rank / alpha / dropout | 64 / 128 / 0 | |
| | Target modules | q, k, v, o, gate, up, down projections | |
| | Learning rate | 1e-4, cosine schedule, 3% warmup | |
| | Batch | 4 per device x 4 accumulation = 16 | |
| | Epochs | 2 (850 steps) | |
| | Max sequence length | 4096 | |
| | Hardware | 1 GPU, 13.7 GB peak, about 75 minutes | |
| | Final losses | train 0.009, validation 0.0165 | |
|
|
| ## Usage |
|
|
| Serve the base model with the adapter in vLLM, with thinking off: |
|
|
| ```bash |
| vllm serve Qwen/Qwen3.5-4B --enable-lora \ |
| --lora-modules text2sql-sft=mehul79/Text2SQL-4B-Model \ |
| --enable-auto-tool-choice --tool-call-parser qwen3_coder \ |
| --reasoning-parser qwen3 \ |
| --default-chat-template-kwargs '{"enable_thinking": false}' \ |
| --language-model-only --max-model-len 16384 |
| ``` |
|
|
| Or merge it into the base model with PEFT (`PeftModel.from_pretrained(base, "mehul79/Text2SQL-4B-Model").merge_and_unload()`) |
| and serve the merged weights. |
|
|
| ## Limitations |
|
|
| - Trained only on Spider and SQLite. Other SQL dialects and real-world schemas are untested. |
| - The validation split comes from Spider train, so the validation loss says little about new databases; |
| the Spider dev results above are the real measure. |
| - Every training trace has one fixed shape and contains no errors, so the model never saw a failed query |
| being fixed during training. |
| - It sometimes writes Spider-style double-quoted string literals (`"value"`), which strict SQL parsers reject. |
|
|
