Hugging Face's logo

NafishZaldinanda
/
Qwen3.5-0.8B-Text2SQL

Safetensors
English
qwen3_5
Model card Files
xet
 Community
Qwen3.5-0.8B-Text2SQL
File size: 5,040 Bytes
9896a4e
 
 
 
 
 
 
 
 
4780089
 
 
 
 
 
 
b25e1a1
 
4780089
326f832
 
 
4780089
 
 
 
 
a5ba1d1
4780089
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
---
datasets:
- xlangai/spider
- birdsql/bird23-train-filtered
- seeklhy/SynSQL-2.5M
language:
- en
base_model:
- Qwen/Qwen3.5-0.8B
---
# Qwen3.5-0.8B Text2SQL

Supervised Fine-Tuning (SFT) for Natural Language to SQL Generation

Fine-tuning **Qwen3.5-0.8B** using **Spider**, **BIRD23**, and **SynSQL-2.5M** datasets with **QLoRA + Unsloth**.

Repository Project: https://github.com/MuhammadNafishZaldinanda/finetuning-text2sql

## Dataset

Dialect: SQLite

| Dataset | Source Paper | Samples Used | Notes | Links |
| :--- | :--- | :---: | :--- | :--- |
| **Spider** | [Spider: A Large-Scale Human-Labeled Dataset...](https://arxiv.org/abs/1809.08887) | 7,000 | All training split. | [Link Google Drive Donwload](https://drive.google.com/file/d/1403EGqzIDoHMdQF4c9Bkyl7dZLZ5Wt6J/view?usp=sharing) |
| **BIRD23-Train-Filtered** | [A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://arxiv.org/abs/2305.03111) | 6,626 | Used subset `bird23-train-filtered`. | [HuggingFace Dataset](https://huggingface.co/datasets/birdsql/bird23-train-filtered) |
| **SynSQL-2.5M (Filtered)** | [OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale](https://arxiv.org/abs/2503.02240) | 7,000 |  Filtering by *question style* dan *SQL complexity*. | [HuggingFace Dataset](https://huggingface.co/datasets/seeklhy/SynSQL-2.5M)<br>[OmniSQL Official Repo](https://github.com/RUCKBReasoning/OmniSQL) |
| **Total** | | **20,626** |  | [NafishZaldinanda/text2sql-omnisql-style](https://huggingface.co/datasets/NafishZaldinanda/text2sql-omnisql-style) |


#### SynSQL-2.5M Filtering Configuration

| Criteria | Value |
|-----------|--------|
| Question Style | Formal, Colloquial, Imperative, Interrogative, Descriptive, Concise |
| Simple | 700 |
| Moderate | 2,800 |
| Complex | 2,800 |
| Highly Complex | 700 |
| Total Samples | 7,000 |

#### Instruction Prompt

````TEXT
Task Overview:
You are a data science expert. Below, you are provided with a database schema and a natural language question. Your task is to understand the schema and generate a valid SQL query to answer the question.

Database Engine:
SQLite

Database Schema:
{db_details}
This schema describes the database's structure, including tables, columns, primary keys, foreign keys, and any relevant relationships or constraints.

Question:
{evidence}{question}

Instructions:
- Make sure you only output the information that is asked in the question. If the question asks for a specific column, make sure to only include that column in the SELECT clause, nothing more.
- The generated query should return all of the information asked in the question without any missing or extra information.
- Before generating the final SQL query, please think through the steps of how to write the query.

Output Format:
In your answer, please enclose the generated SQL query in a code block:
```sql
-- Your SQL query
```

Take a deep breath and think step by step to find the correct SQL query.
````

### LoRA Configuration

| Parameter | Value |
| :--- | :--- |
| **Quantization** | 4-bit |
| **LoRA Rank (r)** | 32 |
| **LoRA Alpha** | 64 |
| **LoRA Dropout** | 0.0 |
| **Bias** | none |
| **Trainable Parameters** | 12.78M |
| **Percentage of Trainable Parameters** | 2.22% |
| **Target Modules** | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |

### Training Configuration

| Parameter | Value |
|------------|--------|
| Base Model | Qwen3.5-0.8B |
| Total Dataset | 20626 |
| Epoch | 1 |
| Max Sequence Length | 8704 |
| Learning Rate | 1e-5 |
| Scheduler | Cosine |
| Warmup Ratio | 10% |
| Optimizer | adam_torch_fused |
| Max Gradient Norm | 0.5 |
| Batch Size | 1 |
| Gradient Accumulation Steps | 8 |
| Training Time | 7 Hours 36 Minutes |

### Training Results

| Metric | Value |
|---------|-------:|
| Final Train Loss | 0.262 |
| Final Validation Loss | 0.218 |

## Model Performance Evaluation: Base vs. Fine-Tuned (Qwen3.5-0.8B)

## 1. Base Model (Qwen3.5-0.8B)

### Overall Performance
| Metric | Value |
| :--- | ---: |
| **Accuracy** | **21.3%** |
| Correct | 106 |
| Wrong | 152 |
| Execution Error | 240 |

### Performance by Difficulty
| Difficulty | Correct / Total | Accuracy |
| :--- | :---: | :---: |
| Simple | 51 / 148 | 34.5% |
| Moderate | 47 / 250 | 18.8% |
| Challenging | 8 / 102 | 7.8% |

---

## 2. Fine-Tuned Model (QLoRA)

### Overall Performance
| Metric | Value |
| :--- | ---: |
| **Accuracy** | **18.3%** |
| Correct | 91 |
| Wrong | 171 |
| Execution Error | 236 |

### Performance by Difficulty
| Difficulty | Correct / Total | Accuracy |
| :--- | :---: | :---: |
| Simple | 57 / 148 | 38.5% |
| Moderate | 26 / 250 | 10.4% |
| Challenging | 8 / 102 | 7.8% |

---

## 3. Head-to-Head Comparison

| Metric | Base Model | Fine-Tuned (QLoRA) | Selisih |
| :--- | :---: | :---: | :---: |
| **Overall Accuracy** | **21.3%** | 18.3% | -3.0% |
| **Simple** | 34.5% | **38.5%** | +4.0% |
| **Moderate** | **18.8%** | 10.4% | -8.4% |
| **Challenging** | 7.8% | 7.8% | 0.0% |
| **Execution Error** | 240 | **236** | -4 |