Update README.md
Browse files
README.md
CHANGED
|
@@ -22,7 +22,6 @@ metrics:
|
|
| 22 |
<img src="https://raw.githubusercontent.com/imsanjoykb/deepSQL-R1-distill-8B/refs/heads/master/assets/logomain.png" alt="Repo banner">
|
| 23 |
</div>
|
| 24 |
|
| 25 |
-
|
| 26 |
<div align="center" style="line-height: 1;">
|
| 27 |
<a href="https://huggingface.co/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
|
| 28 |
<img alt="Hugging Face Model" src="https://img.shields.io/badge/HuggingFace-Model-FF6F00?style=for-the-badge&logo=huggingface&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
|
@@ -47,7 +46,6 @@ metrics:
|
|
| 47 |
</a>
|
| 48 |
</div>
|
| 49 |
|
| 50 |
-
|
| 51 |
## Abstract
|
| 52 |
State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
|
| 53 |
|
|
@@ -90,8 +88,6 @@ State-of-the-art advances in LLMs have pushed NLP to its limits, where even comp
|
|
| 90 |
| 5️⃣ | llama3.2 | 75 | 75 | 77 | 72 | 82 | 76 | 74 | 71 | 77 | 74 |
|
| 91 |
| 6️⃣ | Mistral-7B | 70 | 70 | 72 | 68 | 78 | 72 | 70 | 68 | 72 | 70 |
|
| 92 |
|
| 93 |
-
|
| 94 |
-
|
| 95 |
## Inference
|
| 96 |
|
| 97 |
Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
|
|
@@ -195,7 +191,6 @@ text_streamer = TextStreamer(tokenizer)
|
|
| 195 |
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
|
| 196 |
```
|
| 197 |
|
| 198 |
-
|
| 199 |
## Citing
|
| 200 |
```
|
| 201 |
@misc{,
|
|
@@ -237,8 +232,6 @@ _ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
|
|
| 237 |
</a>
|
| 238 |
</div>
|
| 239 |
|
| 240 |
-
|
| 241 |
-
|
| 242 |
## Usages Services
|
| 243 |
<div align="center" style="line-height: 1;">
|
| 244 |
<a href="#" target="_blank" style="margin: 2px;">
|
|
|
|
| 22 |
<img src="https://raw.githubusercontent.com/imsanjoykb/deepSQL-R1-distill-8B/refs/heads/master/assets/logomain.png" alt="Repo banner">
|
| 23 |
</div>
|
| 24 |
|
|
|
|
| 25 |
<div align="center" style="line-height: 1;">
|
| 26 |
<a href="https://huggingface.co/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
|
| 27 |
<img alt="Hugging Face Model" src="https://img.shields.io/badge/HuggingFace-Model-FF6F00?style=for-the-badge&logo=huggingface&logoColor=white" style="display: inline-block; vertical-align: middle;">
|
|
|
|
| 46 |
</a>
|
| 47 |
</div>
|
| 48 |
|
|
|
|
| 49 |
## Abstract
|
| 50 |
State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
|
| 51 |
|
|
|
|
| 88 |
| 5️⃣ | llama3.2 | 75 | 75 | 77 | 72 | 82 | 76 | 74 | 71 | 77 | 74 |
|
| 89 |
| 6️⃣ | Mistral-7B | 70 | 70 | 72 | 68 | 78 | 72 | 70 | 68 | 72 | 70 |
|
| 90 |
|
|
|
|
|
|
|
| 91 |
## Inference
|
| 92 |
|
| 93 |
Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
|
|
|
|
| 191 |
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
|
| 192 |
```
|
| 193 |
|
|
|
|
| 194 |
## Citing
|
| 195 |
```
|
| 196 |
@misc{,
|
|
|
|
| 232 |
</a>
|
| 233 |
</div>
|
| 234 |
|
|
|
|
|
|
|
| 235 |
## Usages Services
|
| 236 |
<div align="center" style="line-height: 1;">
|
| 237 |
<a href="#" target="_blank" style="margin: 2px;">
|