Hugging Face's logo

IParraMartin
/
impossible-llms-german-random-fourgram

Text Generation
Transformers
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Model card Files
xet
 Community
IParraMartin commited on
Commit
d70711c
·
verified ·
1 Parent(s): 5612003

End of training

Browse files
Files changed (2) hide show
  1. README.md +230 -0
  2. generation_config.json +6 -0
README.md ADDED
@@ -0,0 +1,230 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: impossible-llms-german-random-fourgram
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # impossible-llms-german-random-fourgram
14
+
15
+ This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 5.9430
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+ ### Training hyperparameters
34
+
35
+ The following hyperparameters were used during training:
36
+ - learning_rate: 0.0001
37
+ - train_batch_size: 12
38
+ - eval_batch_size: 8
39
+ - seed: 0
40
+ - distributed_type: multi-GPU
41
+ - num_devices: 4
42
+ - gradient_accumulation_steps: 8
43
+ - total_train_batch_size: 384
44
+ - total_eval_batch_size: 32
45
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
+ - lr_scheduler_type: cosine
47
+ - lr_scheduler_warmup_ratio: 0.1
48
+ - training_steps: 3000
49
+ - mixed_precision_training: Native AMP
50
+ - label_smoothing_factor: 0.1
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss |
55
+ |:-------------:|:--------:|:----:|:---------------:|
56
+ | 37.7338 | 1.0 | 18 | 9.3648 |
57
+ | 35.7624 | 2.0 | 36 | 8.8937 |
58
+ | 33.8558 | 3.0 | 54 | 8.4733 |
59
+ | 32.0947 | 4.0 | 72 | 8.0390 |
60
+ | 30.1266 | 5.0 | 90 | 7.5550 |
61
+ | 28.1882 | 6.0 | 108 | 7.0654 |
62
+ | 26.5774 | 7.0 | 126 | 6.6686 |
63
+ | 25.9146 | 8.0 | 144 | 6.4362 |
64
+ | 25.232 | 9.0 | 162 | 6.3227 |
65
+ | 24.8537 | 10.0 | 180 | 6.2398 |
66
+ | 24.4154 | 11.0 | 198 | 6.1600 |
67
+ | 24.228 | 12.0 | 216 | 6.1035 |
68
+ | 24.0803 | 13.0 | 234 | 6.0586 |
69
+ | 24.0885 | 14.0 | 252 | 6.0192 |
70
+ | 23.6898 | 15.0 | 270 | 5.9871 |
71
+ | 23.4396 | 16.0 | 288 | 5.9482 |
72
+ | 23.4791 | 17.0 | 306 | 5.9201 |
73
+ | 23.2907 | 18.0 | 324 | 5.8843 |
74
+ | 23.3204 | 19.0 | 342 | 5.8601 |
75
+ | 22.9215 | 20.0 | 360 | 5.8303 |
76
+ | 22.8381 | 21.0 | 378 | 5.7999 |
77
+ | 22.9067 | 22.0 | 396 | 5.7706 |
78
+ | 22.4945 | 23.0 | 414 | 5.7395 |
79
+ | 22.5825 | 24.0 | 432 | 5.7105 |
80
+ | 22.2327 | 25.0 | 450 | 5.6771 |
81
+ | 22.1948 | 26.0 | 468 | 5.6481 |
82
+ | 22.1161 | 27.0 | 486 | 5.6131 |
83
+ | 21.7786 | 28.0 | 504 | 5.5857 |
84
+ | 21.608 | 29.0 | 522 | 5.5584 |
85
+ | 21.6092 | 30.0 | 540 | 5.5294 |
86
+ | 21.5158 | 31.0 | 558 | 5.5099 |
87
+ | 21.2812 | 32.0 | 576 | 5.4840 |
88
+ | 21.1324 | 33.0 | 594 | 5.4645 |
89
+ | 21.0367 | 34.0 | 612 | 5.4435 |
90
+ | 20.8917 | 35.0 | 630 | 5.4239 |
91
+ | 20.8304 | 36.0 | 648 | 5.4095 |
92
+ | 20.5685 | 37.0 | 666 | 5.3899 |
93
+ | 20.7227 | 38.0 | 684 | 5.3780 |
94
+ | 20.5019 | 39.0 | 702 | 5.3647 |
95
+ | 20.237 | 40.0 | 720 | 5.3504 |
96
+ | 20.2962 | 41.0 | 738 | 5.3395 |
97
+ | 20.1685 | 42.0 | 756 | 5.3290 |
98
+ | 20.0126 | 43.0 | 774 | 5.3224 |
99
+ | 19.9712 | 44.0 | 792 | 5.3135 |
100
+ | 20.0726 | 45.0 | 810 | 5.3062 |
101
+ | 19.7299 | 46.0 | 828 | 5.2992 |
102
+ | 19.623 | 47.0 | 846 | 5.2930 |
103
+ | 19.6938 | 48.0 | 864 | 5.2912 |
104
+ | 19.5416 | 49.0 | 882 | 5.2889 |
105
+ | 19.6437 | 50.0 | 900 | 5.2886 |
106
+ | 19.3772 | 51.0 | 918 | 5.2847 |
107
+ | 19.1646 | 52.0 | 936 | 5.2846 |
108
+ | 19.3211 | 53.0 | 954 | 5.2805 |
109
+ | 18.9739 | 54.0 | 972 | 5.2832 |
110
+ | 18.8837 | 55.0 | 990 | 5.2825 |
111
+ | 18.9044 | 56.0 | 1008 | 5.2842 |
112
+ | 18.9384 | 57.0 | 1026 | 5.2849 |
113
+ | 18.8621 | 58.0 | 1044 | 5.2873 |
114
+ | 18.7566 | 59.0 | 1062 | 5.2925 |
115
+ | 18.6145 | 60.0 | 1080 | 5.2943 |
116
+ | 18.4163 | 61.0 | 1098 | 5.3036 |
117
+ | 18.3534 | 62.0 | 1116 | 5.3079 |
118
+ | 18.502 | 63.0 | 1134 | 5.3107 |
119
+ | 18.2143 | 64.0 | 1152 | 5.3149 |
120
+ | 18.338 | 65.0 | 1170 | 5.3235 |
121
+ | 18.1699 | 66.0 | 1188 | 5.3293 |
122
+ | 18.1661 | 67.0 | 1206 | 5.3375 |
123
+ | 18.0224 | 68.0 | 1224 | 5.3471 |
124
+ | 17.6682 | 69.0 | 1242 | 5.3529 |
125
+ | 17.8052 | 70.0 | 1260 | 5.3615 |
126
+ | 17.5968 | 71.0 | 1278 | 5.3675 |
127
+ | 17.4778 | 72.0 | 1296 | 5.3773 |
128
+ | 17.6043 | 73.0 | 1314 | 5.3858 |
129
+ | 17.4908 | 74.0 | 1332 | 5.3964 |
130
+ | 17.3254 | 75.0 | 1350 | 5.4018 |
131
+ | 17.4003 | 76.0 | 1368 | 5.4124 |
132
+ | 17.3649 | 77.0 | 1386 | 5.4241 |
133
+ | 17.2245 | 78.0 | 1404 | 5.4301 |
134
+ | 17.3074 | 79.0 | 1422 | 5.4462 |
135
+ | 17.1521 | 80.0 | 1440 | 5.4551 |
136
+ | 16.811 | 81.0 | 1458 | 5.4611 |
137
+ | 16.9251 | 82.0 | 1476 | 5.4712 |
138
+ | 16.8495 | 83.0 | 1494 | 5.4820 |
139
+ | 16.9539 | 84.0 | 1512 | 5.4906 |
140
+ | 16.692 | 85.0 | 1530 | 5.4990 |
141
+ | 16.7383 | 86.0 | 1548 | 5.5123 |
142
+ | 16.6054 | 87.0 | 1566 | 5.5219 |
143
+ | 16.5755 | 88.0 | 1584 | 5.5302 |
144
+ | 16.4844 | 89.0 | 1602 | 5.5382 |
145
+ | 16.3689 | 90.0 | 1620 | 5.5526 |
146
+ | 16.3734 | 91.0 | 1638 | 5.5635 |
147
+ | 16.2672 | 92.0 | 1656 | 5.5733 |
148
+ | 16.2833 | 93.0 | 1674 | 5.5851 |
149
+ | 16.2626 | 94.0 | 1692 | 5.5924 |
150
+ | 16.1087 | 95.0 | 1710 | 5.6025 |
151
+ | 16.3242 | 96.0 | 1728 | 5.6100 |
152
+ | 16.0916 | 97.0 | 1746 | 5.6241 |
153
+ | 16.0279 | 98.0 | 1764 | 5.6313 |
154
+ | 15.9436 | 99.0 | 1782 | 5.6448 |
155
+ | 15.9385 | 100.0 | 1800 | 5.6488 |
156
+ | 15.9182 | 101.0 | 1818 | 5.6596 |
157
+ | 15.6631 | 102.0 | 1836 | 5.6721 |
158
+ | 15.7364 | 103.0 | 1854 | 5.6818 |
159
+ | 15.8297 | 104.0 | 1872 | 5.6852 |
160
+ | 15.7784 | 105.0 | 1890 | 5.6973 |
161
+ | 15.676 | 106.0 | 1908 | 5.7030 |
162
+ | 15.6292 | 107.0 | 1926 | 5.7139 |
163
+ | 15.5341 | 108.0 | 1944 | 5.7238 |
164
+ | 15.5014 | 109.0 | 1962 | 5.7322 |
165
+ | 15.4952 | 110.0 | 1980 | 5.7425 |
166
+ | 15.2674 | 111.0 | 1998 | 5.7469 |
167
+ | 15.3804 | 112.0 | 2016 | 5.7527 |
168
+ | 15.1628 | 113.0 | 2034 | 5.7648 |
169
+ | 15.2465 | 114.0 | 2052 | 5.7694 |
170
+ | 15.265 | 115.0 | 2070 | 5.7807 |
171
+ | 15.2491 | 116.0 | 2088 | 5.7871 |
172
+ | 15.187 | 117.0 | 2106 | 5.7928 |
173
+ | 15.2305 | 118.0 | 2124 | 5.8011 |
174
+ | 15.0624 | 119.0 | 2142 | 5.8061 |
175
+ | 15.0283 | 120.0 | 2160 | 5.8126 |
176
+ | 15.0128 | 121.0 | 2178 | 5.8241 |
177
+ | 14.9024 | 122.0 | 2196 | 5.8275 |
178
+ | 15.0507 | 123.0 | 2214 | 5.8336 |
179
+ | 14.9945 | 124.0 | 2232 | 5.8384 |
180
+ | 14.9695 | 125.0 | 2250 | 5.8458 |
181
+ | 14.9255 | 126.0 | 2268 | 5.8542 |
182
+ | 14.8809 | 127.0 | 2286 | 5.8562 |
183
+ | 14.8236 | 128.0 | 2304 | 5.8621 |
184
+ | 14.715 | 129.0 | 2322 | 5.8650 |
185
+ | 14.7913 | 130.0 | 2340 | 5.8739 |
186
+ | 14.8002 | 131.0 | 2358 | 5.8759 |
187
+ | 14.7913 | 132.0 | 2376 | 5.8817 |
188
+ | 14.7532 | 133.0 | 2394 | 5.8849 |
189
+ | 14.7149 | 134.0 | 2412 | 5.8883 |
190
+ | 14.7549 | 135.0 | 2430 | 5.8916 |
191
+ | 14.6338 | 136.0 | 2448 | 5.8976 |
192
+ | 14.6065 | 137.0 | 2466 | 5.9038 |
193
+ | 14.6682 | 138.0 | 2484 | 5.9063 |
194
+ | 14.6515 | 139.0 | 2502 | 5.9084 |
195
+ | 14.713 | 140.0 | 2520 | 5.9124 |
196
+ | 14.5961 | 141.0 | 2538 | 5.9161 |
197
+ | 14.6323 | 142.0 | 2556 | 5.9189 |
198
+ | 14.3841 | 143.0 | 2574 | 5.9198 |
199
+ | 14.4734 | 144.0 | 2592 | 5.9218 |
200
+ | 14.4725 | 145.0 | 2610 | 5.9243 |
201
+ | 14.5212 | 146.0 | 2628 | 5.9275 |
202
+ | 14.5615 | 147.0 | 2646 | 5.9290 |
203
+ | 14.5326 | 148.0 | 2664 | 5.9303 |
204
+ | 14.4589 | 149.0 | 2682 | 5.9320 |
205
+ | 14.4737 | 150.0 | 2700 | 5.9347 |
206
+ | 14.5569 | 151.0 | 2718 | 5.9346 |
207
+ | 14.6112 | 152.0 | 2736 | 5.9364 |
208
+ | 14.4516 | 153.0 | 2754 | 5.9379 |
209
+ | 14.4938 | 154.0 | 2772 | 5.9372 |
210
+ | 14.4729 | 155.0 | 2790 | 5.9397 |
211
+ | 14.3714 | 156.0 | 2808 | 5.9405 |
212
+ | 14.3877 | 157.0 | 2826 | 5.9403 |
213
+ | 14.4449 | 158.0 | 2844 | 5.9419 |
214
+ | 14.349 | 159.0 | 2862 | 5.9414 |
215
+ | 14.3506 | 160.0 | 2880 | 5.9424 |
216
+ | 14.4542 | 161.0 | 2898 | 5.9425 |
217
+ | 14.3577 | 162.0 | 2916 | 5.9427 |
218
+ | 14.4566 | 163.0 | 2934 | 5.9427 |
219
+ | 14.4671 | 164.0 | 2952 | 5.9429 |
220
+ | 14.4485 | 165.0 | 2970 | 5.9430 |
221
+ | 14.4603 | 166.0 | 2988 | 5.9430 |
222
+ | 28.6518 | 166.6857 | 3000 | 5.9430 |
223
+
224
+
225
+ ### Framework versions
226
+
227
+ - Transformers 4.49.0
228
+ - Pytorch 2.4.0+cu121
229
+ - Datasets 3.4.0
230
+ - Tokenizers 0.21.0
generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 0,
4
+ "eos_token_id": 0,
5
+ "transformers_version": "4.49.0"
6
+ }