Hugging Face's logo

rbelanec
/
train_wsc_1745950307

PEFT
Safetensors
llama-factory
lntuning
Generated from Trainer
Model card Files
xet
 Community
rbelanec commited on
Commit
e16318f
·
verified ·
1 Parent(s): d5bb1dd

Model save

Browse files
Files changed (2) hide show
  1. README.md +262 -0
  2. adapter_model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,262 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: apache-2.0
4
+ base_model: mistralai/Mistral-7B-Instruct-v0.3
5
+ tags:
6
+ - llama-factory
7
+ - generated_from_trainer
8
+ model-index:
9
+ - name: train_wsc_1745950307
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # train_wsc_1745950307
17
+
18
+ This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) on an unknown dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 1.3833
21
+ - Num Input Tokens Seen: 13676608
22
+
23
+ ## Model description
24
+
25
+ More information needed
26
+
27
+ ## Intended uses & limitations
28
+
29
+ More information needed
30
+
31
+ ## Training and evaluation data
32
+
33
+ More information needed
34
+
35
+ ## Training procedure
36
+
37
+ ### Training hyperparameters
38
+
39
+ The following hyperparameters were used during training:
40
+ - learning_rate: 5e-05
41
+ - train_batch_size: 2
42
+ - eval_batch_size: 2
43
+ - seed: 123
44
+ - gradient_accumulation_steps: 2
45
+ - total_train_batch_size: 4
46
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
47
+ - lr_scheduler_type: cosine
48
+ - training_steps: 40000
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
53
+ |:-------------:|:--------:|:-----:|:---------------:|:-----------------:|
54
+ | 2.38 | 1.6024 | 200 | 1.5036 | 68480 |
55
+ | 1.8434 | 3.2008 | 400 | 1.4091 | 137040 |
56
+ | 1.6131 | 4.8032 | 600 | 1.3973 | 205344 |
57
+ | 1.9089 | 6.4016 | 800 | 1.3924 | 273648 |
58
+ | 2.4603 | 8.0 | 1000 | 1.3827 | 342192 |
59
+ | 2.3568 | 9.6024 | 1200 | 1.3743 | 410624 |
60
+ | 2.6209 | 11.2008 | 1400 | 1.4002 | 479392 |
61
+ | 1.9548 | 12.8032 | 1600 | 1.3998 | 547360 |
62
+ | 2.3111 | 14.4016 | 1800 | 1.3934 | 616128 |
63
+ | 1.5327 | 16.0 | 2000 | 1.3905 | 683616 |
64
+ | 1.5612 | 17.6024 | 2200 | 1.3936 | 751520 |
65
+ | 2.6575 | 19.2008 | 2400 | 1.3690 | 820000 |
66
+ | 1.8487 | 20.8032 | 2600 | 1.3929 | 888576 |
67
+ | 1.5663 | 22.4016 | 2800 | 1.3796 | 956480 |
68
+ | 1.7485 | 24.0 | 3000 | 1.3739 | 1024784 |
69
+ | 1.7832 | 25.6024 | 3200 | 1.3798 | 1093536 |
70
+ | 1.84 | 27.2008 | 3400 | 1.3878 | 1161248 |
71
+ | 2.2367 | 28.8032 | 3600 | 1.3837 | 1229760 |
72
+ | 2.2398 | 30.4016 | 3800 | 1.3837 | 1298112 |
73
+ | 2.5764 | 32.0 | 4000 | 1.3718 | 1366864 |
74
+ | 2.3443 | 33.6024 | 4200 | 1.3770 | 1435664 |
75
+ | 1.5648 | 35.2008 | 4400 | 1.3796 | 1503408 |
76
+ | 2.418 | 36.8032 | 4600 | 1.3792 | 1572288 |
77
+ | 2.2188 | 38.4016 | 4800 | 1.3876 | 1640848 |
78
+ | 1.9725 | 40.0 | 5000 | 1.3678 | 1708416 |
79
+ | 2.2863 | 41.6024 | 5200 | 1.3855 | 1776416 |
80
+ | 1.4944 | 43.2008 | 5400 | 1.3841 | 1845088 |
81
+ | 1.4354 | 44.8032 | 5600 | 1.3899 | 1913360 |
82
+ | 1.2547 | 46.4016 | 5800 | 1.3720 | 1981136 |
83
+ | 2.239 | 48.0 | 6000 | 1.3813 | 2050304 |
84
+ | 2.0904 | 49.6024 | 6200 | 1.3658 | 2118640 |
85
+ | 2.3447 | 51.2008 | 6400 | 1.3733 | 2186992 |
86
+ | 2.5379 | 52.8032 | 6600 | 1.3852 | 2255392 |
87
+ | 2.2955 | 54.4016 | 6800 | 1.3662 | 2324240 |
88
+ | 1.7964 | 56.0 | 7000 | 1.3752 | 2391840 |
89
+ | 1.9475 | 57.6024 | 7200 | 1.3890 | 2460464 |
90
+ | 2.0349 | 59.2008 | 7400 | 1.3970 | 2528416 |
91
+ | 2.0624 | 60.8032 | 7600 | 1.3765 | 2597008 |
92
+ | 1.9295 | 62.4016 | 7800 | 1.3909 | 2664720 |
93
+ | 2.6086 | 64.0 | 8000 | 1.3866 | 2733360 |
94
+ | 1.4825 | 65.6024 | 8200 | 1.3820 | 2801792 |
95
+ | 1.4862 | 67.2008 | 8400 | 1.3797 | 2870768 |
96
+ | 1.311 | 68.8032 | 8600 | 1.3816 | 2939344 |
97
+ | 1.5444 | 70.4016 | 8800 | 1.3809 | 3007936 |
98
+ | 1.6452 | 72.0 | 9000 | 1.3795 | 3076384 |
99
+ | 1.2808 | 73.6024 | 9200 | 1.3821 | 3144624 |
100
+ | 1.4038 | 75.2008 | 9400 | 1.3899 | 3212896 |
101
+ | 2.0719 | 76.8032 | 9600 | 1.3870 | 3281408 |
102
+ | 2.1484 | 78.4016 | 9800 | 1.3826 | 3349872 |
103
+ | 1.3604 | 80.0 | 10000 | 1.3719 | 3418368 |
104
+ | 1.9583 | 81.6024 | 10200 | 1.3645 | 3486640 |
105
+ | 1.9835 | 83.2008 | 10400 | 1.3874 | 3555456 |
106
+ | 2.271 | 84.8032 | 10600 | 1.3794 | 3623440 |
107
+ | 1.738 | 86.4016 | 10800 | 1.3925 | 3691760 |
108
+ | 1.5836 | 88.0 | 11000 | 1.3949 | 3760416 |
109
+ | 2.4306 | 89.6024 | 11200 | 1.3694 | 3829184 |
110
+ | 1.8857 | 91.2008 | 11400 | 1.3643 | 3897520 |
111
+ | 1.915 | 92.8032 | 11600 | 1.3843 | 3965568 |
112
+ | 1.8895 | 94.4016 | 11800 | 1.3623 | 4033904 |
113
+ | 1.7895 | 96.0 | 12000 | 1.3735 | 4102480 |
114
+ | 2.4102 | 97.6024 | 12200 | 1.3713 | 4170912 |
115
+ | 1.6993 | 99.2008 | 12400 | 1.3902 | 4238208 |
116
+ | 1.9568 | 100.8032 | 12600 | 1.3873 | 4307408 |
117
+ | 2.684 | 102.4016 | 12800 | 1.3784 | 4375136 |
118
+ | 1.7455 | 104.0 | 13000 | 1.3705 | 4443232 |
119
+ | 1.8316 | 105.6024 | 13200 | 1.3711 | 4511824 |
120
+ | 1.8675 | 107.2008 | 13400 | 1.3649 | 4580464 |
121
+ | 1.998 | 108.8032 | 13600 | 1.3803 | 4648752 |
122
+ | 1.8191 | 110.4016 | 13800 | 1.3605 | 4717136 |
123
+ | 1.6216 | 112.0 | 14000 | 1.3584 | 4785328 |
124
+ | 1.359 | 113.6024 | 14200 | 1.3578 | 4853616 |
125
+ | 1.5381 | 115.2008 | 14400 | 1.3633 | 4922160 |
126
+ | 1.4842 | 116.8032 | 14600 | 1.3836 | 4990880 |
127
+ | 1.9195 | 118.4016 | 14800 | 1.3861 | 5059200 |
128
+ | 2.2324 | 120.0 | 15000 | 1.4003 | 5127856 |
129
+ | 2.6831 | 121.6024 | 15200 | 1.3824 | 5196320 |
130
+ | 2.9063 | 123.2008 | 15400 | 1.3948 | 5264752 |
131
+ | 2.2375 | 124.8032 | 15600 | 1.3633 | 5333360 |
132
+ | 1.4379 | 126.4016 | 15800 | 1.3655 | 5401648 |
133
+ | 2.022 | 128.0 | 16000 | 1.3829 | 5470144 |
134
+ | 1.5 | 129.6024 | 16200 | 1.3639 | 5539584 |
135
+ | 2.0066 | 131.2008 | 16400 | 1.3695 | 5606896 |
136
+ | 2.0182 | 132.8032 | 16600 | 1.3684 | 5675392 |
137
+ | 1.8751 | 134.4016 | 16800 | 1.3700 | 5743824 |
138
+ | 1.6614 | 136.0 | 17000 | 1.3650 | 5812000 |
139
+ | 1.621 | 137.6024 | 17200 | 1.4032 | 5880400 |
140
+ | 2.3474 | 139.2008 | 17400 | 1.3793 | 5949456 |
141
+ | 1.5025 | 140.8032 | 17600 | 1.3786 | 6017584 |
142
+ | 1.8176 | 142.4016 | 17800 | 1.3833 | 6086352 |
143
+ | 2.5774 | 144.0 | 18000 | 1.3774 | 6153776 |
144
+ | 1.6388 | 145.6024 | 18200 | 1.3680 | 6222672 |
145
+ | 2.3709 | 147.2008 | 18400 | 1.3629 | 6291168 |
146
+ | 1.7972 | 148.8032 | 18600 | 1.3776 | 6359136 |
147
+ | 2.2769 | 150.4016 | 18800 | 1.3718 | 6426976 |
148
+ | 2.4199 | 152.0 | 19000 | 1.3809 | 6495568 |
149
+ | 2.334 | 153.6024 | 19200 | 1.3765 | 6564224 |
150
+ | 1.3356 | 155.2008 | 19400 | 1.3622 | 6632768 |
151
+ | 1.5932 | 156.8032 | 19600 | 1.4022 | 6701376 |
152
+ | 2.132 | 158.4016 | 19800 | 1.3923 | 6769520 |
153
+ | 1.243 | 160.0 | 20000 | 1.3735 | 6837904 |
154
+ | 1.9679 | 161.6024 | 20200 | 1.3769 | 6905904 |
155
+ | 1.2943 | 163.2008 | 20400 | 1.3794 | 6974368 |
156
+ | 1.5976 | 164.8032 | 20600 | 1.3860 | 7043152 |
157
+ | 2.4079 | 166.4016 | 20800 | 1.3839 | 7112192 |
158
+ | 1.902 | 168.0 | 21000 | 1.3712 | 7179920 |
159
+ | 2.4094 | 169.6024 | 21200 | 1.3693 | 7248608 |
160
+ | 1.8267 | 171.2008 | 21400 | 1.3882 | 7316928 |
161
+ | 1.3429 | 172.8032 | 21600 | 1.3781 | 7385216 |
162
+ | 1.9929 | 174.4016 | 21800 | 1.3723 | 7453728 |
163
+ | 1.5492 | 176.0 | 22000 | 1.3745 | 7521888 |
164
+ | 2.029 | 177.6024 | 22200 | 1.3866 | 7590256 |
165
+ | 1.0526 | 179.2008 | 22400 | 1.3728 | 7658736 |
166
+ | 2.1402 | 180.8032 | 22600 | 1.3733 | 7727488 |
167
+ | 2.1717 | 182.4016 | 22800 | 1.3580 | 7796416 |
168
+ | 1.0474 | 184.0 | 23000 | 1.3782 | 7864592 |
169
+ | 2.6908 | 185.6024 | 23200 | 1.3840 | 7933232 |
170
+ | 1.6581 | 187.2008 | 23400 | 1.3909 | 8001808 |
171
+ | 1.737 | 188.8032 | 23600 | 1.3631 | 8070240 |
172
+ | 2.1513 | 190.4016 | 23800 | 1.3719 | 8138688 |
173
+ | 2.9168 | 192.0 | 24000 | 1.3730 | 8206576 |
174
+ | 1.3348 | 193.6024 | 24200 | 1.3669 | 8274800 |
175
+ | 1.8642 | 195.2008 | 24400 | 1.3766 | 8342976 |
176
+ | 1.8082 | 196.8032 | 24600 | 1.3738 | 8411584 |
177
+ | 1.9464 | 198.4016 | 24800 | 1.3706 | 8479856 |
178
+ | 1.3418 | 200.0 | 25000 | 1.3411 | 8548304 |
179
+ | 1.0372 | 201.6024 | 25200 | 1.3819 | 8617520 |
180
+ | 1.4196 | 203.2008 | 25400 | 1.3806 | 8685328 |
181
+ | 2.6419 | 204.8032 | 25600 | 1.3815 | 8753696 |
182
+ | 1.6081 | 206.4016 | 25800 | 1.3642 | 8821840 |
183
+ | 1.0938 | 208.0 | 26000 | 1.3757 | 8889904 |
184
+ | 1.7548 | 209.6024 | 26200 | 1.3723 | 8958528 |
185
+ | 0.4627 | 211.2008 | 26400 | 1.3632 | 9026416 |
186
+ | 1.8565 | 212.8032 | 26600 | 1.3725 | 9094992 |
187
+ | 1.8041 | 214.4016 | 26800 | 1.3807 | 9162896 |
188
+ | 2.2034 | 216.0 | 27000 | 1.3971 | 9231632 |
189
+ | 1.2453 | 217.6024 | 27200 | 1.3777 | 9299920 |
190
+ | 1.3627 | 219.2008 | 27400 | 1.3901 | 9368176 |
191
+ | 1.761 | 220.8032 | 27600 | 1.3853 | 9437280 |
192
+ | 2.4811 | 222.4016 | 27800 | 1.3785 | 9505712 |
193
+ | 1.2036 | 224.0 | 28000 | 1.3843 | 9573776 |
194
+ | 1.6312 | 225.6024 | 28200 | 1.3712 | 9641744 |
195
+ | 2.7126 | 227.2008 | 28400 | 1.3745 | 9710672 |
196
+ | 1.8068 | 228.8032 | 28600 | 1.3504 | 9778976 |
197
+ | 2.0016 | 230.4016 | 28800 | 1.3559 | 9846768 |
198
+ | 2.4666 | 232.0 | 29000 | 1.3740 | 9915328 |
199
+ | 2.3197 | 233.6024 | 29200 | 1.3657 | 9984304 |
200
+ | 2.2468 | 235.2008 | 29400 | 1.3896 | 10052656 |
201
+ | 2.5254 | 236.8032 | 29600 | 1.3690 | 10121152 |
202
+ | 1.7327 | 238.4016 | 29800 | 1.3695 | 10188944 |
203
+ | 2.1135 | 240.0 | 30000 | 1.3644 | 10257280 |
204
+ | 2.4984 | 241.6024 | 30200 | 1.3745 | 10326160 |
205
+ | 2.6298 | 243.2008 | 30400 | 1.3582 | 10393920 |
206
+ | 1.9454 | 244.8032 | 30600 | 1.3769 | 10462528 |
207
+ | 1.6705 | 246.4016 | 30800 | 1.3858 | 10530528 |
208
+ | 1.0821 | 248.0 | 31000 | 1.3936 | 10599104 |
209
+ | 0.9083 | 249.6024 | 31200 | 1.3718 | 10667920 |
210
+ | 2.1352 | 251.2008 | 31400 | 1.3749 | 10736624 |
211
+ | 2.3122 | 252.8032 | 31600 | 1.3664 | 10804624 |
212
+ | 1.9733 | 254.4016 | 31800 | 1.3804 | 10873200 |
213
+ | 1.9106 | 256.0 | 32000 | 1.3715 | 10941264 |
214
+ | 1.0159 | 257.6024 | 32200 | 1.3682 | 11010000 |
215
+ | 1.7219 | 259.2008 | 32400 | 1.3720 | 11077280 |
216
+ | 2.02 | 260.8032 | 32600 | 1.3732 | 11145744 |
217
+ | 1.8695 | 262.4016 | 32800 | 1.3925 | 11214112 |
218
+ | 0.979 | 264.0 | 33000 | 1.3623 | 11282096 |
219
+ | 1.4244 | 265.6024 | 33200 | 1.3711 | 11350608 |
220
+ | 1.158 | 267.2008 | 33400 | 1.3865 | 11418608 |
221
+ | 2.2895 | 268.8032 | 33600 | 1.3829 | 11487936 |
222
+ | 2.3647 | 270.4016 | 33800 | 1.3672 | 11556272 |
223
+ | 1.8704 | 272.0 | 34000 | 1.3715 | 11624208 |
224
+ | 2.0451 | 273.6024 | 34200 | 1.3936 | 11693424 |
225
+ | 1.933 | 275.2008 | 34400 | 1.3798 | 11761200 |
226
+ | 1.1507 | 276.8032 | 34600 | 1.3821 | 11830208 |
227
+ | 1.6466 | 278.4016 | 34800 | 1.3833 | 11898240 |
228
+ | 1.6418 | 280.0 | 35000 | 1.3833 | 11966432 |
229
+ | 2.0597 | 281.6024 | 35200 | 1.3833 | 12035232 |
230
+ | 1.4621 | 283.2008 | 35400 | 1.3833 | 12103232 |
231
+ | 2.1253 | 284.8032 | 35600 | 1.3833 | 12171376 |
232
+ | 1.6358 | 286.4016 | 35800 | 1.3833 | 12240128 |
233
+ | 2.2219 | 288.0 | 36000 | 1.3833 | 12308016 |
234
+ | 1.2531 | 289.6024 | 36200 | 1.3833 | 12375936 |
235
+ | 1.9197 | 291.2008 | 36400 | 1.3833 | 12444880 |
236
+ | 2.2558 | 292.8032 | 36600 | 1.3833 | 12513664 |
237
+ | 1.9804 | 294.4016 | 36800 | 1.3833 | 12581616 |
238
+ | 1.8568 | 296.0 | 37000 | 1.3833 | 12650688 |
239
+ | 1.4651 | 297.6024 | 37200 | 1.3833 | 12718976 |
240
+ | 1.8077 | 299.2008 | 37400 | 1.3833 | 12787680 |
241
+ | 1.3443 | 300.8032 | 37600 | 1.3833 | 12856448 |
242
+ | 2.9669 | 302.4016 | 37800 | 1.3833 | 12924128 |
243
+ | 1.9458 | 304.0 | 38000 | 1.3833 | 12992944 |
244
+ | 1.2774 | 305.6024 | 38200 | 1.3833 | 13060928 |
245
+ | 2.3874 | 307.2008 | 38400 | 1.3833 | 13129472 |
246
+ | 2.9663 | 308.8032 | 38600 | 1.3833 | 13198064 |
247
+ | 1.9925 | 310.4016 | 38800 | 1.3833 | 13266304 |
248
+ | 2.5085 | 312.0 | 39000 | 1.3833 | 13334832 |
249
+ | 1.2625 | 313.6024 | 39200 | 1.3833 | 13402912 |
250
+ | 1.2102 | 315.2008 | 39400 | 1.3833 | 13470656 |
251
+ | 1.5315 | 316.8032 | 39600 | 1.3833 | 13539984 |
252
+ | 1.4806 | 318.4016 | 39800 | 1.3833 | 13608768 |
253
+ | 1.8959 | 320.0 | 40000 | 1.3833 | 13676608 |
254
+
255
+
256
+ ### Framework versions
257
+
258
+ - PEFT 0.15.2.dev0
259
+ - Transformers 4.51.3
260
+ - Pytorch 2.6.0+cu124
261
+ - Datasets 3.5.0
262
+ - Tokenizers 0.21.1
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:59c569ba74a536bd81f6cba2fcf974aed1aaf54152f2483c83df482cd0c516be
3
  size 541712
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb1a9b20a7496dafb96fa56be1626e592148bef94ed3ea1fa3b5b5ce6bd3090f
3
  size 541712