imsanjoykb commited on
Commit
5cf8f1d
·
verified ·
1 Parent(s): 3dfd4ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -7
README.md CHANGED
@@ -22,7 +22,6 @@ metrics:
22
  <img src="https://raw.githubusercontent.com/imsanjoykb/deepSQL-R1-distill-8B/refs/heads/master/assets/logomain.png" alt="Repo banner">
23
  </div>
24
 
25
-
26
  <div align="center" style="line-height: 1;">
27
  <a href="https://huggingface.co/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
28
  <img alt="Hugging Face Model" src="https://img.shields.io/badge/HuggingFace-Model-FF6F00?style=for-the-badge&logo=huggingface&logoColor=white" style="display: inline-block; vertical-align: middle;">
@@ -47,7 +46,6 @@ metrics:
47
  </a>
48
  </div>
49
 
50
-
51
  ## Abstract
52
  State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
53
 
@@ -90,8 +88,6 @@ State-of-the-art advances in LLMs have pushed NLP to its limits, where even comp
90
  | 5️⃣ | llama3.2 | 75 | 75 | 77 | 72 | 82 | 76 | 74 | 71 | 77 | 74 |
91
  | 6️⃣ | Mistral-7B | 70 | 70 | 72 | 68 | 78 | 72 | 70 | 68 | 72 | 70 |
92
 
93
-
94
-
95
  ## Inference
96
 
97
  Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
@@ -195,7 +191,6 @@ text_streamer = TextStreamer(tokenizer)
195
  _ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
196
  ```
197
 
198
-
199
  ## Citing
200
  ```
201
  @misc{,
@@ -237,8 +232,6 @@ _ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
237
  </a>
238
  </div>
239
 
240
-
241
-
242
  ## Usages Services
243
  <div align="center" style="line-height: 1;">
244
  <a href="#" target="_blank" style="margin: 2px;">
 
22
  <img src="https://raw.githubusercontent.com/imsanjoykb/deepSQL-R1-distill-8B/refs/heads/master/assets/logomain.png" alt="Repo banner">
23
  </div>
24
 
 
25
  <div align="center" style="line-height: 1;">
26
  <a href="https://huggingface.co/imsanjoykb/deepSQL-R1-distill-8B" target="_blank" style="margin: 2px;">
27
  <img alt="Hugging Face Model" src="https://img.shields.io/badge/HuggingFace-Model-FF6F00?style=for-the-badge&logo=huggingface&logoColor=white" style="display: inline-block; vertical-align: middle;">
 
46
  </a>
47
  </div>
48
 
 
49
  ## Abstract
50
  State-of-the-art advances in LLMs have pushed NLP to its limits, where even complex tasks, such as code generation, can be automated. This paper describes the deepSQL-R1-distill-8B, a fine-tuned and quantized model variant of the DeepSeek-R1 model architecture and specifically optimized for text-to-SQL conversion. Fine-tuning was performed using Unsloth, one of the most efficient frameworks for fine-tuning LLMs, in combination with Parameter-Efficient Fine-Tuning and the SFTTrainer framework. This allows domain-specific adaptation with minimal resource consumption. The approach fine-tunes curated datasets by LoRA, ensuring a more parameter-efficient and lower-memory-consuming model. Besides this, we investigate reinforcement learning techniques to further enhance the model's ability in generating accurate and contextually appropriate SQL queries. Combination of 8-bit quantization, LoRA, Unsloth, and reinforcement learning places deepSQL-R1-distill-8B as one of the cutting-edge solutions for automatic SQL code generation in real-world applications. Addressing major challenges in computational efficiency, domain-specific adaptation, and reinforcement-based refinement, this model is leading the way toward a more intuitive and resource-effective way of interacting with relational databases.
51
 
 
88
  | 5️⃣ | llama3.2 | 75 | 75 | 77 | 72 | 82 | 76 | 74 | 71 | 77 | 74 |
89
  | 6️⃣ | Mistral-7B | 70 | 70 | 72 | 68 | 78 | 72 | 70 | 68 | 72 | 70 |
90
 
 
 
91
  ## Inference
92
 
93
  Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
 
191
  _ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)
192
  ```
193
 
 
194
  ## Citing
195
  ```
196
  @misc{,
 
232
  </a>
233
  </div>
234
 
 
 
235
  ## Usages Services
236
  <div align="center" style="line-height: 1;">
237
  <a href="#" target="_blank" style="margin: 2px;">