Improve model card: Add pipeline tag, library, comprehensive content, and correct GitHub links

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +158 -6
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
  license: apache-2.0
 
 
3
  ---
4
 
5
  <div align="center">
@@ -25,18 +27,35 @@ license: apache-2.0
25
  </div>
26
 
27
  <div align="center" style="display: flex; gap: 5px; justify-content: center;">
28
- <a href="https://idea-finai.github.io/SQL-R1/"><img src="https://img.shields.io/badge/🏠_Homepage-SQL--R1-4B4B77?style=flat-square"/></a>
29
- <a href="https://arxiv.org/abs/2504.08600"><img src="https://img.shields.io/badge/πŸ“‘_arXiv-2504.08600-00A98F?style=flat-square"/></a>
30
- <a href="https://github.com/MPX0222/SQL-R1"><img src="https://img.shields.io/badge/⭐_GitHub-SQL--R1-2F80ED?style=flat-square"/></a>
31
- <a href="https://huggingface.co/MPX0222forHF/SQL-R1-7B"><img src="https://img.shields.io/badge/πŸ€—_HuggingFace-SQL--R1-FF9D00?style=flat-square"/></a>
32
- <a href="https://github.com/MPX0222/SQL-R1/stargazers"><img src="https://img.shields.io/github/stars/MPX0222/SQL-R1?style=flat-square&color=946CE6"/></a>
 
 
33
  </div>
 
34
 
 
 
 
 
 
35
 
36
  ## πŸ“– Overview
37
 
38
  Natural Language to SQL (NL2SQL) enables intuitive interactions with databases by transforming natural language queries into structured SQL statements. Despite recent advancements in enhancing human-computer interaction within database applications, significant challenges persist, particularly regarding the inference performance in complex scenarios involving multi-table joins and nested queries. Current methodologies primarily utilize supervised fine-tuning (SFT) to train the NL2SQL model, which may limit adaptability and interpretability in new environments (e.g., finance and healthcare). In order to enhance the reasoning performance of the NL2SQL model in the above complex situations, we introduce SQL-R1, a novel NL2SQL reasoning model trained by the reinforcement learning (RL) algorithms. We design a specialized RL-based reward function tailored for NL2SQL tasks and discussed the impact of cold start on the effectiveness of intensive training. In addition, we achieve competitive accuracy using only a tiny amount of synthetic NL2SQL data for augmented training and further explore data engineering for RL. In existing experiments, SQL-R1 achieves execution accuracy of 88.6\% and 67.1\% on the benchmark Spider and BIRD, respectively.
39
 
 
 
 
 
 
 
 
 
 
40
  ## πŸ“š Citations
41
 
42
  ```bibtex
@@ -46,4 +65,137 @@ Natural Language to SQL (NL2SQL) enables intuitive interactions with databases b
46
  journal={arXiv preprint arXiv:2504.08600},
47
  year={2025}
48
  }
49
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: text-generation
4
+ library_name: transformers
5
  ---
6
 
7
  <div align="center">
 
27
  </div>
28
 
29
  <div align="center" style="display: flex; gap: 5px; justify-content: center;">
30
+ <a href="https://dataarctech.github.io/SQL-R1/"><img src="https://img.shields.io/badge/🏠_Homepage-blue?style=for-the-badge"/></a>
31
+ <a href="https://arxiv.org/abs/2504.08600"><img src="https://img.shields.io/badge/arXiv-red?style=for-the-badge&logo=arxiv"/></a>
32
+ <a href="https://huggingface.co/papers/2504.08600"><img src="https://img.shields.io/badge/Paper-HuggingFace-ffbd45.svg?style=for-the-badge&logo=huggingface"/></a>
33
+ <a href="https://github.com/IDEA-FinAI/SQL-R1"><img src="https://img.shields.io/badge/GitHub-black?style=for-the-badge&logo=github"/></a>
34
+ <a href="https://huggingface.co/MPX0222forHF/SQL-R1-7B"><img src="https://img.shields.io/badge/HuggingFace-FF9D00?style=for-the-badge&logo=huggingface"/></a>
35
+ <a href="https://www.modelscope.cn/models/MPX0222/SQL-R1-7B"><img src="https://img.shields.io/badge/πŸ€–_ModelScope-946CE6?style=for-the-badge"/></a>
36
+ <a href="https://github.com/IDEA-FinAI/SQL-R1/stargazers"><img src="https://img.shields.io/github/stars/IDEA-FinAI/SQL-R1?style=for-the-badge&color=white"/></a>
37
  </div>
38
+ <br>
39
 
40
+ <div align="center" style="display: flex; gap: 5px; justify-content: center;">
41
+ <p>
42
+ <b>πŸ”₯ Our work is accepted by NeurIPS 2025. Welcome to star and cite our work! ✨</b>
43
+ </p>
44
+ </div>
45
 
46
  ## πŸ“– Overview
47
 
48
  Natural Language to SQL (NL2SQL) enables intuitive interactions with databases by transforming natural language queries into structured SQL statements. Despite recent advancements in enhancing human-computer interaction within database applications, significant challenges persist, particularly regarding the inference performance in complex scenarios involving multi-table joins and nested queries. Current methodologies primarily utilize supervised fine-tuning (SFT) to train the NL2SQL model, which may limit adaptability and interpretability in new environments (e.g., finance and healthcare). In order to enhance the reasoning performance of the NL2SQL model in the above complex situations, we introduce SQL-R1, a novel NL2SQL reasoning model trained by the reinforcement learning (RL) algorithms. We design a specialized RL-based reward function tailored for NL2SQL tasks and discussed the impact of cold start on the effectiveness of intensive training. In addition, we achieve competitive accuracy using only a tiny amount of synthetic NL2SQL data for augmented training and further explore data engineering for RL. In existing experiments, SQL-R1 achieves execution accuracy of 88.6\% and 67.1\% on the benchmark Spider and BIRD, respectively.
49
 
50
+ <div align="center">
51
+ <img src="images/overview.png" alt="SQL-R1 Overview" width="800"/>
52
+ <p align="center">
53
+ Figure 1: Demonstration of our work.
54
+ </p>
55
+
56
+ </div>
57
+
58
+
59
  ## πŸ“š Citations
60
 
61
  ```bibtex
 
65
  journal={arXiv preprint arXiv:2504.08600},
66
  year={2025}
67
  }
68
+ ```
69
+
70
+ ## πŸ“° News
71
+
72
+ - **[2025.09.18]** πŸŽ‰ SQL-R1 is accept by NeurIPS 2025! We will soon update the full version of the paper and poster. Welcome to star and cite our work!
73
+ - **[2025.05.27]** πŸŽ‰ We have released the full version of SQL-R1.
74
+ - **[2025.05.21]** πŸŽ‰ We have released our model weights on HuggingFace! Check out the [Model Weights](#-model-weights) section below.
75
+ - **[2025.04.11]** πŸ“‘ Our paper is now available on [arXiv](https://arxiv.org/abs/2504.08600).
76
+
77
+
78
+ ## πŸš€ Coming Soon Checklist
79
+
80
+ - [x] πŸ“ Update the camera-ready version of the paper, homepage and poster. coming sooooon!
81
+ - [x] πŸ“Š Release model weights on HuggingFace and ModelScope
82
+ - [x] πŸ”§ Open source training code and RL dataset
83
+ - [x] πŸ“ Detailed documentation
84
+ - [x] πŸ› οΈ Environment setup guide
85
+
86
+
87
+ ## πŸ€– Model Weights
88
+
89
+ We are excited to release our SQL-R1 model weights! You can find them on HuggingFace:
90
+
91
+ | Model | Size | HuggingFace Link | ModelScope Link |
92
+ |-------------|-------------|------|------|
93
+ | SQL-R1 (3B) | 3B | [πŸ€— Download](https://huggingface.co/MPX0222forHF/SQL-R1-3B) | - |
94
+ | SQL-R1 (7B) | 7B | [πŸ€— Download](https://huggingface.co/MPX0222forHF/SQL-R1-7B) | [οΏ½οΏ½οΏ½ Download](https://www.modelscope.cn/models/MPX0222/SQL-R1-7B) |
95
+ | SQL-R1 (14B) | 14B | [πŸ€— Download](https://huggingface.co/MPX0222forHF/SQL-R1-14B) | - |
96
+
97
+
98
+ ## πŸ“‘ Documentation Structure
99
+
100
+ This repository is organized as follows:
101
+
102
+ ```
103
+ SQL-R1/
104
+ β”œβ”€β”€ data/ # Datasets
105
+ β”‚ β”œβ”€β”€ Spider/
106
+ β”‚ └── BIRD/
107
+ β”œβ”€β”€ models/ # Base models or checkpoints
108
+ β”‚ β”œβ”€β”€ Qwen2.5-Coder-3B-Instruct/
109
+ β”‚ └── Qwen2.5-Coder-7B-Instruct/
110
+ ......
111
+ β”œβ”€β”€ sh/ # Scripts for data processing, training, inference and evaluation
112
+ β”œβ”€β”€ src/ # Source code
113
+ └── verl/ # Verl framework
114
+ ```
115
+
116
+
117
+ ## πŸ› οΈ Environment Setup
118
+
119
+ > [!NOTE]
120
+ > Before getting started, make sure your computing environment supports the following settings:
121
+ > - Environment: Python 3.9+
122
+ > - CUDA Version: 12.0+ (for verl and vllm integration)
123
+ > - GPU Prerequisites: 8 x 80GB+ GPU (for training) / 2 x 40GB GPU (for inference)
124
+
125
+ ### Installation
126
+
127
+ 1. Clone the repository:
128
+ ```bash
129
+ git clone https://github.com/IDEA-FinAI/SQL-R1.git
130
+ cd SQL-R1
131
+ ```
132
+
133
+ 2. Create and activate a virtual environment (recommended):
134
+ ```bash
135
+ conda create -n sqlr1 python=3.9
136
+ ```
137
+
138
+ 3. Install dependencies:
139
+ ```bash
140
+ pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu121
141
+ pip install vllm==0.6.3 ray
142
+ pip install flash-attn --no-build-isolation
143
+ pip install -e . # For verl integration
144
+ pip install wandb IPython matplotlib sqlparse func_timeout nltk ijson
145
+ ```
146
+
147
+ 4. Download the model weights from HuggingFace and put them in the `models/` directory.
148
+
149
+ 5. For training, copy the training dataset in the `example_data` directory to the `data` directory. For inference, copy the database information in the `db_info` directory (including files forSpider-dev, Spider-test and BIRD-dev) to the related dataset (`data/Spider`, `data/BIRD`) directory.
150
+
151
+ ## πŸš€ Quick Start
152
+
153
+ 1. Run training:
154
+ ```bash
155
+ sh sh/train.sh
156
+ ```
157
+
158
+ 2. Run inference:
159
+ ```bash
160
+ sh sh/inference.sh
161
+ ```
162
+
163
+ 3. Run evaluation:
164
+ ```bash
165
+ # evaluate spider
166
+ sh sh/eval_spider.sh
167
+ # evaluate bird
168
+ sh sh/eval_bird.sh
169
+ ```
170
+
171
+ ## 🌟 Applications
172
+
173
+ SQL-R1 can be effectively utilized in the following key areas:
174
+
175
+ 1. **Foundation Model for Workflow Systems**
176
+ - Serves as a base model for complex database operations
177
+ - Enables seamless integration with existing workflow automation systems
178
+ - Supports customization and fine-tuning for specific business processes
179
+ - Provides robust API endpoints for system integration
180
+
181
+ 2. **Enhanced NL2SQL Interpretability**
182
+ - Generates detailed explanations for SQL query construction
183
+ - Provides step-by-step reasoning for query transformation
184
+ - Helps users understand the relationship between natural language and SQL syntax
185
+ - Offers visualization of query execution plans
186
+
187
+ 3. **Self-Evolving NL2SQL Agent Checkpoint**
188
+ - Acts as a foundation checkpoint for continuous learning
189
+ - Enables iterative improvement through feedback loops
190
+ - Supports transfer learning for domain-specific adaptations
191
+ - Facilitates model versioning and performance tracking
192
+
193
+ This may be added in the future work. If you have any ideas, please feel free to contact us.
194
+
195
+ ## Thanks for
196
+
197
+ We thank [OmniSQL](https://github.com/RUCKBReasoning/OmniSQL) and follow their evaluation code and database information retrieval code. We have adapted and modified their evaluation scripts for our project.
198
+
199
+ ## Star History
200
+
201
+ [![Star History Chart](https://api.star-history.com/svg?repos=IDEA-FinAI/SQL-R1&type=Date)](https://www.star-history.com/#IDEA-FinAI/SQL-R1&Date)