Update README.md
Browse files
README.md
CHANGED
|
@@ -98,31 +98,6 @@ Gretel Synthetic Text-to-SQL dataset provides natural-language SQL requests, dat
|
|
| 98 |
| `sql_prompt` | Natural-language request |
|
| 99 |
| `sql` | Ground-truth SQL completion |
|
| 100 |
|
| 101 |
-
Training pipeline converts each example into an instruction-style prompt and trains the model specifically on the assistant SQL completion.
|
| 102 |
-
|
| 103 |
-
## Training Configuration
|
| 104 |
-
|
| 105 |
-
| Setting | Value |
|
| 106 |
-
| ----------------------- | ---------------------------------: |
|
| 107 |
-
| Base model | `Qwen/Qwen2.5-Coder-0.5B-Instruct` |
|
| 108 |
-
| LoRA rank | `16` |
|
| 109 |
-
| LoRA alpha | `32` |
|
| 110 |
-
| LoRA dropout | `0.05` |
|
| 111 |
-
| Epochs | **2** |
|
| 112 |
-
| Training examples | **20,000** |
|
| 113 |
-
| Validation examples | **1,000** |
|
| 114 |
-
| Train batch size | **4** |
|
| 115 |
-
| Gradient accumulation | **4** |
|
| 116 |
-
| Effective batch size | **16 sequences** |
|
| 117 |
-
| Learning rate | **2e-4** |
|
| 118 |
-
| Warmup ratio | **0.03** |
|
| 119 |
-
| Weight decay | **0.01** |
|
| 120 |
-
| LR scheduler | **Cosine** |
|
| 121 |
-
| Maximum sequence length | **2048** |
|
| 122 |
-
| Gradient checkpointing | **Enabled** |
|
| 123 |
-
| Training objective | **Completion-only SFT** |
|
| 124 |
-
|
| 125 |
-
Training automatically uses **BF16** where supported, falling back to FP16 on CUDA or FP32 when CUDA is unavailable.
|
| 126 |
|
| 127 |
## Training Run
|
| 128 |
|
|
|
|
| 98 |
| `sql_prompt` | Natural-language request |
|
| 99 |
| `sql` | Ground-truth SQL completion |
|
| 100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
|
| 102 |
## Training Run
|
| 103 |
|