Hugging Face's logo

zeeshan73
/
Text2SQL_mistral_7b_cosine_lr

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files
xet
Metrics Community
zeeshan73 commited on
Commit
837dbc8
·
verified ·
1 Parent(s): b0367d0

Model save

Browse files
Files changed (2) hide show
  1. README.md +90 -0
  2. adapter_model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: mistralai/Mistral-7B-Instruct-v0.3
3
+ datasets:
4
+ - generator
5
+ library_name: peft
6
+ license: apache-2.0
7
+ tags:
8
+ - trl
9
+ - sft
10
+ - generated_from_trainer
11
+ model-index:
12
+ - name: mistral_7b_cosine_lr
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # mistral_7b_cosine_lr
20
+
21
+ This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) on the generator dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 5.3993
24
+
25
+ ## Model description
26
+
27
+ More information needed
28
+
29
+ ## Intended uses & limitations
30
+
31
+ More information needed
32
+
33
+ ## Training and evaluation data
34
+
35
+ More information needed
36
+
37
+ ## Training procedure
38
+
39
+ ### Training hyperparameters
40
+
41
+ The following hyperparameters were used during training:
42
+ - learning_rate: 0.003
43
+ - train_batch_size: 3
44
+ - eval_batch_size: 8
45
+ - seed: 42
46
+ - gradient_accumulation_steps: 8
47
+ - total_train_batch_size: 24
48
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
+ - lr_scheduler_type: cosine
50
+ - lr_scheduler_warmup_ratio: 0.03
51
+ - lr_scheduler_warmup_steps: 15
52
+ - num_epochs: 4
53
+
54
+ ### Training results
55
+
56
+ | Training Loss | Epoch | Step | Validation Loss |
57
+ |:-------------:|:------:|:----:|:---------------:|
58
+ | 11.1885 | 0.0549 | 10 | 61.4970 |
59
+ | 37.6512 | 0.1098 | 20 | 12.9405 |
60
+ | 14.576 | 0.1647 | 30 | 27.9852 |
61
+ | 9.5892 | 0.2196 | 40 | 6.4722 |
62
+ | 7.7639 | 0.2745 | 50 | 6.8158 |
63
+ | 6.3878 | 0.3294 | 60 | 6.3811 |
64
+ | 6.6118 | 0.3844 | 70 | 5.9281 |
65
+ | 6.006 | 0.4393 | 80 | 5.6753 |
66
+ | 6.1011 | 0.4942 | 90 | 5.8083 |
67
+ | 5.7396 | 0.5491 | 100 | 5.6193 |
68
+ | 5.5128 | 0.6040 | 110 | 5.4848 |
69
+ | 5.4599 | 0.6589 | 120 | 5.4267 |
70
+ | 5.5193 | 0.7138 | 130 | 5.4757 |
71
+ | 5.4488 | 0.7687 | 140 | 5.4422 |
72
+ | 5.4257 | 0.8236 | 150 | 5.3845 |
73
+ | 5.3938 | 0.8785 | 160 | 5.3727 |
74
+ | 5.3937 | 0.9334 | 170 | 5.3646 |
75
+ | 5.3916 | 0.9883 | 180 | 5.4825 |
76
+ | 5.4217 | 1.0432 | 190 | 5.3534 |
77
+ | 5.3915 | 1.0981 | 200 | 5.3497 |
78
+ | 5.3656 | 1.1531 | 210 | 5.3416 |
79
+ | 5.3718 | 1.2080 | 220 | 5.3691 |
80
+ | 5.3763 | 1.2629 | 230 | 5.4102 |
81
+ | 5.4039 | 1.3178 | 240 | 5.3993 |
82
+
83
+
84
+ ### Framework versions
85
+
86
+ - PEFT 0.13.2
87
+ - Transformers 4.45.2
88
+ - Pytorch 2.4.1+cu121
89
+ - Datasets 3.0.1
90
+ - Tokenizers 0.20.0
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fd2905d5c0aa19650c79e5c7e12ca05f8f980d25f8468d6bfe834ca6a17b1f85
3
  size 3221320120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0358438fd12152111cd241341dac884cdecbf299049a43106d60373423beaed
3
  size 3221320120