jastorj commited on
Commit
59b5a34
·
verified ·
1 Parent(s): 7d1a1ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -43
README.md CHANGED
@@ -1,51 +1,51 @@
1
  ---
2
- license: apache-2.0
3
- language:
4
- - en
5
- tags:
6
- - text-to-sql
7
- - code
8
- - sql
9
- - fine-tuned
10
- - unsloth
11
- - lora
12
- base_model: Snowflake/Arctic-Text2SQL-R1-7B
13
- ---
14
 
15
- # Snowflake/Arctic-Text2SQL-R1-7B Fine-tuned for NL2SQL++ v8
16
 
17
- This model is a fine-tuned version of [Snowflake/Arctic-Text2SQL-R1-7B](https://huggingface.co/Snowflake/Arctic-Text2SQL-R1-7B) on the NL2SQL++ v8 dataset with code-with-thought reasoning.
18
 
19
- ## Model Details
20
 
21
- - **Base Model**: Snowflake/Arctic-Text2SQL-R1-7B
22
- - **Task**: Text-to-SQL generation
23
- - **Dataset**: NL2SQL++ v8 with code-with-thought reasoning
24
- - **Fine-tuning Method**: LoRA (Low-Rank Adaptation) with Unsloth
25
- - **Quantization**: 16-bit merged weights
26
- - **Maximum Sequence Length**: 32768 tokens
27
- - **Training Dataset Size**: 46344 examples
28
- - **Validation Dataset Size**: 1986 examples
29
 
30
- ## Training Configuration
31
 
32
- ### LoRA Parameters
33
- - **LoRA Rank (r)**: 64
34
- - **LoRA Alpha**: 128
35
- - **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
36
 
37
- ### Training Hyperparameters
38
- - **Learning Rate**: 0.0002
39
- - **Training Epochs**: 2
40
- - **Max Steps**: N/A (using epochs)
41
- - **Train Batch Size**: 64
42
- - **Eval Batch Size**: 50
43
- - **Gradient Accumulation Steps**: 2
44
- - **Effective Batch Size**: 128
45
- - **Warmup Steps**: 0
46
- - **Warmup Ratio**: 0.1
47
- - **Optimizer**: AdamW (torch)
48
- - **Learning Rate Scheduler**: Cosine
49
- - **Weight Decay**: 0.01
50
- - **Max Gradient Norm**: 1.0
51
- - **Seed**: 3407
 
1
  ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - text-to-sql
7
+ - code
8
+ - sql
9
+ - fine-tuned
10
+ - unsloth
11
+ - lora
12
+ base_model: Snowflake/Arctic-Text2SQL-R1-7B
13
+ ---
14
 
15
+ # Snowflake/Arctic-Text2SQL-R1-7B Fine-tuned for NL2SQL++ v8
16
 
17
+ This model is a fine-tuned version of [Snowflake/Arctic-Text2SQL-R1-7B](https://huggingface.co/Snowflake/Arctic-Text2SQL-R1-7B) on the NL2SQL++ v8 dataset with code-with-thought reasoning.
18
 
19
+ ## Model Details
20
 
21
+ - **Base Model**: Snowflake/Arctic-Text2SQL-R1-7B
22
+ - **Task**: Text-to-SQL generation
23
+ - **Dataset**: NL2SQL++ v8 with code-with-thought reasoning
24
+ - **Fine-tuning Method**: LoRA (Low-Rank Adaptation) with Unsloth
25
+ - **Quantization**: 16-bit merged weights
26
+ - **Maximum Sequence Length**: 32768 tokens
27
+ - **Training Dataset Size**: 46344 examples
28
+ - **Validation Dataset Size**: 1986 examples
29
 
30
+ ## Training Configuration
31
 
32
+ ### LoRA Parameters
33
+ - **LoRA Rank (r)**: 64
34
+ - **LoRA Alpha**: 128
35
+ - **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
36
 
37
+ ### Training Hyperparameters
38
+ - **Learning Rate**: 0.0002
39
+ - **Training Epochs**: 2
40
+ - **Max Steps**: N/A (using epochs)
41
+ - **Train Batch Size**: 64
42
+ - **Eval Batch Size**: 50
43
+ - **Gradient Accumulation Steps**: 2
44
+ - **Effective Batch Size**: 128
45
+ - **Warmup Steps**: 0
46
+ - **Warmup Ratio**: 0.1
47
+ - **Optimizer**: AdamW (torch)
48
+ - **Learning Rate Scheduler**: Cosine
49
+ - **Weight Decay**: 0.01
50
+ - **Max Gradient Norm**: 1.0
51
+ - **Seed**: 3407