Hugging Face's logo

augustocsc
/
Se124M100KInfPrompt_WT

PEFT
Safetensors
Generated from Trainer
Model card Files
xet
Community
augustocsc commited on
Commit
2d3a712
·
verified ·
1 Parent(s): 63ea4c8

Model save

Browse files
Files changed (3) hide show
  1. README.md +380 -59
  2. all_results.json +6 -6
  3. train_results.json +6 -6
README.md CHANGED
@@ -5,18 +5,18 @@ base_model: gpt2
5
  tags:
6
  - generated_from_trainer
7
  model-index:
8
- - name: Se124M10KInfPrompt
9
  results: []
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
  should probably proofread and complete it, then remove this comment. -->
14
 
15
- # Se124M10KInfPrompt
16
 
17
  This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 0.7128
20
 
21
  ## Model description
22
 
@@ -35,66 +35,387 @@ More information needed
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
- - learning_rate: 5e-05
39
- - train_batch_size: 32
40
- - eval_batch_size: 32
41
  - seed: 42
42
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
43
- - lr_scheduler_type: linear
44
- - num_epochs: 50
45
- - mixed_precision_training: Native AMP
 
 
46
 
47
  ### Training results
48
 
49
- | Training Loss | Epoch | Step | Validation Loss |
50
- |:-------------:|:-----:|:-----:|:---------------:|
51
- | 0.4014 | 1.0 | 267 | 1.0141 |
52
- | 0.2422 | 2.0 | 534 | 0.8523 |
53
- | 0.2202 | 3.0 | 801 | 0.8168 |
54
- | 0.2129 | 4.0 | 1068 | 0.7993 |
55
- | 0.2059 | 5.0 | 1335 | 0.7837 |
56
- | 0.2041 | 6.0 | 1602 | 0.7695 |
57
- | 0.2031 | 7.0 | 1869 | 0.7635 |
58
- | 0.1982 | 8.0 | 2136 | 0.7586 |
59
- | 0.1975 | 9.0 | 2403 | 0.7532 |
60
- | 0.1974 | 10.0 | 2670 | 0.7483 |
61
- | 0.1978 | 11.0 | 2937 | 0.7467 |
62
- | 0.1939 | 12.0 | 3204 | 0.7445 |
63
- | 0.1953 | 13.0 | 3471 | 0.7439 |
64
- | 0.1929 | 14.0 | 3738 | 0.7362 |
65
- | 0.1937 | 15.0 | 4005 | 0.7328 |
66
- | 0.1934 | 16.0 | 4272 | 0.7329 |
67
- | 0.1927 | 17.0 | 4539 | 0.7323 |
68
- | 0.1927 | 18.0 | 4806 | 0.7257 |
69
- | 0.1909 | 19.0 | 5073 | 0.7276 |
70
- | 0.1919 | 20.0 | 5340 | 0.7251 |
71
- | 0.1919 | 21.0 | 5607 | 0.7239 |
72
- | 0.1912 | 22.0 | 5874 | 0.7260 |
73
- | 0.1897 | 23.0 | 6141 | 0.7241 |
74
- | 0.1916 | 24.0 | 6408 | 0.7235 |
75
- | 0.1905 | 25.0 | 6675 | 0.7225 |
76
- | 0.1919 | 26.0 | 6942 | 0.7188 |
77
- | 0.1883 | 27.0 | 7209 | 0.7207 |
78
- | 0.1898 | 28.0 | 7476 | 0.7198 |
79
- | 0.1874 | 29.0 | 7743 | 0.7195 |
80
- | 0.188 | 30.0 | 8010 | 0.7194 |
81
- | 0.1873 | 31.0 | 8277 | 0.7182 |
82
- | 0.1878 | 32.0 | 8544 | 0.7212 |
83
- | 0.1866 | 33.0 | 8811 | 0.7171 |
84
- | 0.1883 | 34.0 | 9078 | 0.7151 |
85
- | 0.1881 | 35.0 | 9345 | 0.7176 |
86
- | 0.1868 | 36.0 | 9612 | 0.7149 |
87
- | 0.1871 | 37.0 | 9879 | 0.7157 |
88
- | 0.1876 | 38.0 | 10146 | 0.7162 |
89
- | 0.188 | 39.0 | 10413 | 0.7142 |
90
- | 0.1861 | 40.0 | 10680 | 0.7149 |
91
- | 0.1862 | 41.0 | 10947 | 0.7144 |
92
- | 0.1862 | 42.0 | 11214 | 0.7128 |
93
- | 0.186 | 43.0 | 11481 | 0.7136 |
94
- | 0.1868 | 44.0 | 11748 | 0.7137 |
95
- | 0.1837 | 45.0 | 12015 | 0.7138 |
96
- | 0.1868 | 46.0 | 12282 | 0.7141 |
97
- | 0.187 | 47.0 | 12549 | 0.7133 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
98
 
99
 
100
  ### Framework versions
 
5
  tags:
6
  - generated_from_trainer
7
  model-index:
8
+ - name: Se124M100KInfPrompt_WT
9
  results: []
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
  should probably proofread and complete it, then remove this comment. -->
14
 
15
+ # Se124M100KInfPrompt_WT
16
 
17
  This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 0.7371
20
 
21
  ## Model description
22
 
 
35
  ### Training hyperparameters
36
 
37
  The following hyperparameters were used during training:
38
+ - learning_rate: 2e-05
39
+ - train_batch_size: 16
40
+ - eval_batch_size: 16
41
  - seed: 42
42
+ - gradient_accumulation_steps: 2
43
+ - total_train_batch_size: 32
44
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
+ - lr_scheduler_type: cosine
46
+ - lr_scheduler_warmup_ratio: 0.03
47
+ - num_epochs: 3
48
 
49
  ### Training results
50
 
51
+ | Training Loss | Epoch | Step | Validation Loss |
52
+ |:-------------:|:------:|:----:|:---------------:|
53
+ | 3.2505 | 0.0082 | 20 | 2.8871 |
54
+ | 3.1482 | 0.0164 | 40 | 2.8898 |
55
+ | 3.1815 | 0.0246 | 60 | 2.8780 |
56
+ | 3.1657 | 0.0327 | 80 | 2.8574 |
57
+ | 3.0926 | 0.0409 | 100 | 2.8249 |
58
+ | 3.1184 | 0.0491 | 120 | 2.7654 |
59
+ | 3.0128 | 0.0573 | 140 | 2.7067 |
60
+ | 2.9348 | 0.0655 | 160 | 2.6351 |
61
+ | 2.7728 | 0.0737 | 180 | 2.5450 |
62
+ | 2.6372 | 0.0819 | 200 | 2.4318 |
63
+ | 2.4966 | 0.0901 | 220 | 2.2950 |
64
+ | 2.3591 | 0.0982 | 240 | 2.1465 |
65
+ | 2.2302 | 0.1064 | 260 | 2.0135 |
66
+ | 2.0753 | 0.1146 | 280 | 1.8699 |
67
+ | 1.9052 | 0.1228 | 300 | 1.7487 |
68
+ | 1.8 | 0.1310 | 320 | 1.6347 |
69
+ | 1.7122 | 0.1392 | 340 | 1.5290 |
70
+ | 1.6217 | 0.1474 | 360 | 1.4386 |
71
+ | 1.5754 | 0.1555 | 380 | 1.3520 |
72
+ | 1.4438 | 0.1637 | 400 | 1.2721 |
73
+ | 1.4155 | 0.1719 | 420 | 1.2061 |
74
+ | 1.3491 | 0.1801 | 440 | 1.1527 |
75
+ | 1.2966 | 0.1883 | 460 | 1.1089 |
76
+ | 1.2319 | 0.1965 | 480 | 1.0730 |
77
+ | 1.2031 | 0.2047 | 500 | 1.0470 |
78
+ | 1.1872 | 0.2129 | 520 | 1.0232 |
79
+ | 1.1362 | 0.2210 | 540 | 1.0026 |
80
+ | 1.13 | 0.2292 | 560 | 0.9844 |
81
+ | 1.0864 | 0.2374 | 580 | 0.9677 |
82
+ | 1.0712 | 0.2456 | 600 | 0.9563 |
83
+ | 1.0732 | 0.2538 | 620 | 0.9418 |
84
+ | 1.0519 | 0.2620 | 640 | 0.9327 |
85
+ | 1.0337 | 0.2702 | 660 | 0.9218 |
86
+ | 1.0408 | 0.2783 | 680 | 0.9093 |
87
+ | 1.004 | 0.2865 | 700 | 0.9030 |
88
+ | 0.9896 | 0.2947 | 720 | 0.8942 |
89
+ | 0.9668 | 0.3029 | 740 | 0.8870 |
90
+ | 0.9539 | 0.3111 | 760 | 0.8814 |
91
+ | 0.953 | 0.3193 | 780 | 0.8736 |
92
+ | 0.9388 | 0.3275 | 800 | 0.8696 |
93
+ | 0.9497 | 0.3357 | 820 | 0.8647 |
94
+ | 0.9309 | 0.3438 | 840 | 0.8619 |
95
+ | 0.9326 | 0.3520 | 860 | 0.8568 |
96
+ | 0.9272 | 0.3602 | 880 | 0.8519 |
97
+ | 0.9355 | 0.3684 | 900 | 0.8498 |
98
+ | 0.9147 | 0.3766 | 920 | 0.8460 |
99
+ | 0.9189 | 0.3848 | 940 | 0.8431 |
100
+ | 0.9061 | 0.3930 | 960 | 0.8394 |
101
+ | 0.9121 | 0.4011 | 980 | 0.8376 |
102
+ | 0.9007 | 0.4093 | 1000 | 0.8373 |
103
+ | 0.8897 | 0.4175 | 1020 | 0.8344 |
104
+ | 0.9037 | 0.4257 | 1040 | 0.8326 |
105
+ | 0.8987 | 0.4339 | 1060 | 0.8282 |
106
+ | 0.8968 | 0.4421 | 1080 | 0.8260 |
107
+ | 0.8906 | 0.4503 | 1100 | 0.8258 |
108
+ | 0.8915 | 0.4585 | 1120 | 0.8231 |
109
+ | 0.8911 | 0.4666 | 1140 | 0.8194 |
110
+ | 0.892 | 0.4748 | 1160 | 0.8171 |
111
+ | 0.8725 | 0.4830 | 1180 | 0.8169 |
112
+ | 0.8732 | 0.4912 | 1200 | 0.8168 |
113
+ | 0.8752 | 0.4994 | 1220 | 0.8154 |
114
+ | 0.8679 | 0.5076 | 1240 | 0.8147 |
115
+ | 0.8519 | 0.5158 | 1260 | 0.8139 |
116
+ | 0.8621 | 0.5239 | 1280 | 0.8092 |
117
+ | 0.8516 | 0.5321 | 1300 | 0.8093 |
118
+ | 0.8588 | 0.5403 | 1320 | 0.8070 |
119
+ | 0.8777 | 0.5485 | 1340 | 0.8080 |
120
+ | 0.8517 | 0.5567 | 1360 | 0.8050 |
121
+ | 0.8572 | 0.5649 | 1380 | 0.8032 |
122
+ | 0.8408 | 0.5731 | 1400 | 0.8052 |
123
+ | 0.8509 | 0.5813 | 1420 | 0.8042 |
124
+ | 0.8478 | 0.5894 | 1440 | 0.8039 |
125
+ | 0.8422 | 0.5976 | 1460 | 0.7995 |
126
+ | 0.8348 | 0.6058 | 1480 | 0.7999 |
127
+ | 0.8328 | 0.6140 | 1500 | 0.7998 |
128
+ | 0.8358 | 0.6222 | 1520 | 0.7988 |
129
+ | 0.825 | 0.6304 | 1540 | 0.7978 |
130
+ | 0.8342 | 0.6386 | 1560 | 0.7975 |
131
+ | 0.839 | 0.6467 | 1580 | 0.7963 |
132
+ | 0.8294 | 0.6549 | 1600 | 0.7954 |
133
+ | 0.8523 | 0.6631 | 1620 | 0.7958 |
134
+ | 0.8294 | 0.6713 | 1640 | 0.7922 |
135
+ | 0.8279 | 0.6795 | 1660 | 0.7939 |
136
+ | 0.8094 | 0.6877 | 1680 | 0.7951 |
137
+ | 0.8388 | 0.6959 | 1700 | 0.7914 |
138
+ | 0.8256 | 0.7041 | 1720 | 0.7907 |
139
+ | 0.8303 | 0.7122 | 1740 | 0.7906 |
140
+ | 0.8196 | 0.7204 | 1760 | 0.7901 |
141
+ | 0.8139 | 0.7286 | 1780 | 0.7891 |
142
+ | 0.8269 | 0.7368 | 1800 | 0.7880 |
143
+ | 0.8265 | 0.7450 | 1820 | 0.7868 |
144
+ | 0.835 | 0.7532 | 1840 | 0.7838 |
145
+ | 0.8354 | 0.7614 | 1860 | 0.7852 |
146
+ | 0.8209 | 0.7695 | 1880 | 0.7842 |
147
+ | 0.8135 | 0.7777 | 1900 | 0.7823 |
148
+ | 0.8207 | 0.7859 | 1920 | 0.7823 |
149
+ | 0.8251 | 0.7941 | 1940 | 0.7820 |
150
+ | 0.8063 | 0.8023 | 1960 | 0.7822 |
151
+ | 0.829 | 0.8105 | 1980 | 0.7800 |
152
+ | 0.8163 | 0.8187 | 2000 | 0.7815 |
153
+ | 0.8266 | 0.8269 | 2020 | 0.7792 |
154
+ | 0.835 | 0.8350 | 2040 | 0.7786 |
155
+ | 0.8102 | 0.8432 | 2060 | 0.7779 |
156
+ | 0.8296 | 0.8514 | 2080 | 0.7771 |
157
+ | 0.7994 | 0.8596 | 2100 | 0.7776 |
158
+ | 0.8085 | 0.8678 | 2120 | 0.7744 |
159
+ | 0.8123 | 0.8760 | 2140 | 0.7738 |
160
+ | 0.811 | 0.8842 | 2160 | 0.7748 |
161
+ | 0.8232 | 0.8923 | 2180 | 0.7738 |
162
+ | 0.8053 | 0.9005 | 2200 | 0.7740 |
163
+ | 0.82 | 0.9087 | 2220 | 0.7719 |
164
+ | 0.8112 | 0.9169 | 2240 | 0.7726 |
165
+ | 0.832 | 0.9251 | 2260 | 0.7712 |
166
+ | 0.8147 | 0.9333 | 2280 | 0.7711 |
167
+ | 0.7964 | 0.9415 | 2300 | 0.7715 |
168
+ | 0.8108 | 0.9497 | 2320 | 0.7688 |
169
+ | 0.8086 | 0.9578 | 2340 | 0.7703 |
170
+ | 0.7982 | 0.9660 | 2360 | 0.7698 |
171
+ | 0.8012 | 0.9742 | 2380 | 0.7681 |
172
+ | 0.8217 | 0.9824 | 2400 | 0.7668 |
173
+ | 0.8001 | 0.9906 | 2420 | 0.7677 |
174
+ | 0.8066 | 0.9988 | 2440 | 0.7676 |
175
+ | 0.7948 | 1.0070 | 2460 | 0.7648 |
176
+ | 0.8126 | 1.0151 | 2480 | 0.7648 |
177
+ | 0.8062 | 1.0233 | 2500 | 0.7639 |
178
+ | 0.8094 | 1.0315 | 2520 | 0.7665 |
179
+ | 0.7977 | 1.0397 | 2540 | 0.7648 |
180
+ | 0.8154 | 1.0479 | 2560 | 0.7635 |
181
+ | 0.7989 | 1.0561 | 2580 | 0.7645 |
182
+ | 0.7976 | 1.0643 | 2600 | 0.7642 |
183
+ | 0.8038 | 1.0725 | 2620 | 0.7624 |
184
+ | 0.7932 | 1.0806 | 2640 | 0.7615 |
185
+ | 0.8001 | 1.0888 | 2660 | 0.7625 |
186
+ | 0.8049 | 1.0970 | 2680 | 0.7617 |
187
+ | 0.7959 | 1.1052 | 2700 | 0.7601 |
188
+ | 0.8094 | 1.1134 | 2720 | 0.7623 |
189
+ | 0.7935 | 1.1216 | 2740 | 0.7619 |
190
+ | 0.7844 | 1.1298 | 2760 | 0.7620 |
191
+ | 0.7842 | 1.1379 | 2780 | 0.7605 |
192
+ | 0.789 | 1.1461 | 2800 | 0.7626 |
193
+ | 0.7963 | 1.1543 | 2820 | 0.7606 |
194
+ | 0.7908 | 1.1625 | 2840 | 0.7578 |
195
+ | 0.7906 | 1.1707 | 2860 | 0.7588 |
196
+ | 0.7819 | 1.1789 | 2880 | 0.7611 |
197
+ | 0.8136 | 1.1871 | 2900 | 0.7594 |
198
+ | 0.8006 | 1.1953 | 2920 | 0.7598 |
199
+ | 0.8006 | 1.2034 | 2940 | 0.7585 |
200
+ | 0.7933 | 1.2116 | 2960 | 0.7571 |
201
+ | 0.7872 | 1.2198 | 2980 | 0.7595 |
202
+ | 0.7915 | 1.2280 | 3000 | 0.7560 |
203
+ | 0.7963 | 1.2362 | 3020 | 0.7557 |
204
+ | 0.7911 | 1.2444 | 3040 | 0.7577 |
205
+ | 0.788 | 1.2526 | 3060 | 0.7562 |
206
+ | 0.7883 | 1.2607 | 3080 | 0.7558 |
207
+ | 0.7901 | 1.2689 | 3100 | 0.7555 |
208
+ | 0.7839 | 1.2771 | 3120 | 0.7551 |
209
+ | 0.8046 | 1.2853 | 3140 | 0.7560 |
210
+ | 0.7944 | 1.2935 | 3160 | 0.7547 |
211
+ | 0.7909 | 1.3017 | 3180 | 0.7547 |
212
+ | 0.7867 | 1.3099 | 3200 | 0.7554 |
213
+ | 0.7877 | 1.3181 | 3220 | 0.7537 |
214
+ | 0.781 | 1.3262 | 3240 | 0.7531 |
215
+ | 0.7902 | 1.3344 | 3260 | 0.7531 |
216
+ | 0.788 | 1.3426 | 3280 | 0.7555 |
217
+ | 0.7906 | 1.3508 | 3300 | 0.7555 |
218
+ | 0.7856 | 1.3590 | 3320 | 0.7544 |
219
+ | 0.7877 | 1.3672 | 3340 | 0.7532 |
220
+ | 0.7925 | 1.3754 | 3360 | 0.7525 |
221
+ | 0.7841 | 1.3835 | 3380 | 0.7534 |
222
+ | 0.799 | 1.3917 | 3400 | 0.7520 |
223
+ | 0.7876 | 1.3999 | 3420 | 0.7500 |
224
+ | 0.7769 | 1.4081 | 3440 | 0.7510 |
225
+ | 0.8041 | 1.4163 | 3460 | 0.7500 |
226
+ | 0.7893 | 1.4245 | 3480 | 0.7526 |
227
+ | 0.7774 | 1.4327 | 3500 | 0.7503 |
228
+ | 0.782 | 1.4409 | 3520 | 0.7501 |
229
+ | 0.7824 | 1.4490 | 3540 | 0.7510 |
230
+ | 0.7813 | 1.4572 | 3560 | 0.7505 |
231
+ | 0.7919 | 1.4654 | 3580 | 0.7513 |
232
+ | 0.7801 | 1.4736 | 3600 | 0.7505 |
233
+ | 0.7751 | 1.4818 | 3620 | 0.7502 |
234
+ | 0.7723 | 1.4900 | 3640 | 0.7488 |
235
+ | 0.7841 | 1.4982 | 3660 | 0.7484 |
236
+ | 0.7938 | 1.5063 | 3680 | 0.7490 |
237
+ | 0.7888 | 1.5145 | 3700 | 0.7496 |
238
+ | 0.7831 | 1.5227 | 3720 | 0.7487 |
239
+ | 0.7881 | 1.5309 | 3740 | 0.7491 |
240
+ | 0.7933 | 1.5391 | 3760 | 0.7464 |
241
+ | 0.781 | 1.5473 | 3780 | 0.7491 |
242
+ | 0.7885 | 1.5555 | 3800 | 0.7474 |
243
+ | 0.7856 | 1.5637 | 3820 | 0.7475 |
244
+ | 0.7871 | 1.5718 | 3840 | 0.7471 |
245
+ | 0.7829 | 1.5800 | 3860 | 0.7464 |
246
+ | 0.8159 | 1.5882 | 3880 | 0.7464 |
247
+ | 0.7836 | 1.5964 | 3900 | 0.7466 |
248
+ | 0.7825 | 1.6046 | 3920 | 0.7472 |
249
+ | 0.7689 | 1.6128 | 3940 | 0.7466 |
250
+ | 0.776 | 1.6210 | 3960 | 0.7476 |
251
+ | 0.7718 | 1.6291 | 3980 | 0.7461 |
252
+ | 0.7905 | 1.6373 | 4000 | 0.7462 |
253
+ | 0.7776 | 1.6455 | 4020 | 0.7475 |
254
+ | 0.7743 | 1.6537 | 4040 | 0.7462 |
255
+ | 0.7778 | 1.6619 | 4060 | 0.7455 |
256
+ | 0.7928 | 1.6701 | 4080 | 0.7449 |
257
+ | 0.8031 | 1.6783 | 4100 | 0.7451 |
258
+ | 0.7845 | 1.6865 | 4120 | 0.7440 |
259
+ | 0.7763 | 1.6946 | 4140 | 0.7453 |
260
+ | 0.7841 | 1.7028 | 4160 | 0.7455 |
261
+ | 0.7814 | 1.7110 | 4180 | 0.7450 |
262
+ | 0.7843 | 1.7192 | 4200 | 0.7441 |
263
+ | 0.7733 | 1.7274 | 4220 | 0.7449 |
264
+ | 0.7779 | 1.7356 | 4240 | 0.7437 |
265
+ | 0.7855 | 1.7438 | 4260 | 0.7448 |
266
+ | 0.7775 | 1.7519 | 4280 | 0.7443 |
267
+ | 0.7802 | 1.7601 | 4300 | 0.7432 |
268
+ | 0.783 | 1.7683 | 4320 | 0.7431 |
269
+ | 0.7753 | 1.7765 | 4340 | 0.7441 |
270
+ | 0.7772 | 1.7847 | 4360 | 0.7433 |
271
+ | 0.7813 | 1.7929 | 4380 | 0.7432 |
272
+ | 0.7817 | 1.8011 | 4400 | 0.7423 |
273
+ | 0.7769 | 1.8093 | 4420 | 0.7426 |
274
+ | 0.7843 | 1.8174 | 4440 | 0.7428 |
275
+ | 0.7719 | 1.8256 | 4460 | 0.7428 |
276
+ | 0.7872 | 1.8338 | 4480 | 0.7427 |
277
+ | 0.7741 | 1.8420 | 4500 | 0.7421 |
278
+ | 0.7683 | 1.8502 | 4520 | 0.7422 |
279
+ | 0.7844 | 1.8584 | 4540 | 0.7433 |
280
+ | 0.7705 | 1.8666 | 4560 | 0.7425 |
281
+ | 0.7838 | 1.8747 | 4580 | 0.7427 |
282
+ | 0.7822 | 1.8829 | 4600 | 0.7422 |
283
+ | 0.7867 | 1.8911 | 4620 | 0.7415 |
284
+ | 0.7742 | 1.8993 | 4640 | 0.7428 |
285
+ | 0.7683 | 1.9075 | 4660 | 0.7420 |
286
+ | 0.7706 | 1.9157 | 4680 | 0.7413 |
287
+ | 0.7804 | 1.9239 | 4700 | 0.7420 |
288
+ | 0.7951 | 1.9321 | 4720 | 0.7417 |
289
+ | 0.7686 | 1.9402 | 4740 | 0.7411 |
290
+ | 0.7798 | 1.9484 | 4760 | 0.7400 |
291
+ | 0.7885 | 1.9566 | 4780 | 0.7402 |
292
+ | 0.7757 | 1.9648 | 4800 | 0.7408 |
293
+ | 0.7783 | 1.9730 | 4820 | 0.7408 |
294
+ | 0.7679 | 1.9812 | 4840 | 0.7404 |
295
+ | 0.7767 | 1.9894 | 4860 | 0.7409 |
296
+ | 0.7676 | 1.9975 | 4880 | 0.7415 |
297
+ | 0.7548 | 2.0057 | 4900 | 0.7410 |
298
+ | 0.7687 | 2.0139 | 4920 | 0.7414 |
299
+ | 0.7895 | 2.0221 | 4940 | 0.7403 |
300
+ | 0.7826 | 2.0303 | 4960 | 0.7403 |
301
+ | 0.7675 | 2.0385 | 4980 | 0.7419 |
302
+ | 0.7714 | 2.0467 | 5000 | 0.7401 |
303
+ | 0.7686 | 2.0549 | 5020 | 0.7417 |
304
+ | 0.7645 | 2.0630 | 5040 | 0.7408 |
305
+ | 0.7792 | 2.0712 | 5060 | 0.7403 |
306
+ | 0.77 | 2.0794 | 5080 | 0.7396 |
307
+ | 0.7752 | 2.0876 | 5100 | 0.7390 |
308
+ | 0.7797 | 2.0958 | 5120 | 0.7398 |
309
+ | 0.7785 | 2.1040 | 5140 | 0.7401 |
310
+ | 0.7727 | 2.1122 | 5160 | 0.7403 |
311
+ | 0.7748 | 2.1203 | 5180 | 0.7395 |
312
+ | 0.7657 | 2.1285 | 5200 | 0.7396 |
313
+ | 0.7709 | 2.1367 | 5220 | 0.7405 |
314
+ | 0.7947 | 2.1449 | 5240 | 0.7394 |
315
+ | 0.7758 | 2.1531 | 5260 | 0.7396 |
316
+ | 0.779 | 2.1613 | 5280 | 0.7397 |
317
+ | 0.7727 | 2.1695 | 5300 | 0.7395 |
318
+ | 0.7841 | 2.1777 | 5320 | 0.7394 |
319
+ | 0.7809 | 2.1858 | 5340 | 0.7391 |
320
+ | 0.7722 | 2.1940 | 5360 | 0.7398 |
321
+ | 0.7703 | 2.2022 | 5380 | 0.7391 |
322
+ | 0.7845 | 2.2104 | 5400 | 0.7390 |
323
+ | 0.7691 | 2.2186 | 5420 | 0.7392 |
324
+ | 0.7781 | 2.2268 | 5440 | 0.7397 |
325
+ | 0.7719 | 2.2350 | 5460 | 0.7382 |
326
+ | 0.7829 | 2.2431 | 5480 | 0.7383 |
327
+ | 0.7839 | 2.2513 | 5500 | 0.7391 |
328
+ | 0.7666 | 2.2595 | 5520 | 0.7384 |
329
+ | 0.782 | 2.2677 | 5540 | 0.7390 |
330
+ | 0.7773 | 2.2759 | 5560 | 0.7389 |
331
+ | 0.7844 | 2.2841 | 5580 | 0.7385 |
332
+ | 0.7522 | 2.2923 | 5600 | 0.7388 |
333
+ | 0.7645 | 2.3005 | 5620 | 0.7394 |
334
+ | 0.7921 | 2.3086 | 5640 | 0.7377 |
335
+ | 0.7716 | 2.3168 | 5660 | 0.7378 |
336
+ | 0.7699 | 2.3250 | 5680 | 0.7384 |
337
+ | 0.7812 | 2.3332 | 5700 | 0.7385 |
338
+ | 0.7853 | 2.3414 | 5720 | 0.7387 |
339
+ | 0.7898 | 2.3496 | 5740 | 0.7384 |
340
+ | 0.7727 | 2.3578 | 5760 | 0.7376 |
341
+ | 0.7752 | 2.3659 | 5780 | 0.7374 |
342
+ | 0.7723 | 2.3741 | 5800 | 0.7379 |
343
+ | 0.7611 | 2.3823 | 5820 | 0.7383 |
344
+ | 0.7733 | 2.3905 | 5840 | 0.7380 |
345
+ | 0.7733 | 2.3987 | 5860 | 0.7382 |
346
+ | 0.7723 | 2.4069 | 5880 | 0.7375 |
347
+ | 0.777 | 2.4151 | 5900 | 0.7379 |
348
+ | 0.7733 | 2.4233 | 5920 | 0.7379 |
349
+ | 0.7788 | 2.4314 | 5940 | 0.7379 |
350
+ | 0.769 | 2.4396 | 5960 | 0.7371 |
351
+ | 0.7832 | 2.4478 | 5980 | 0.7385 |
352
+ | 0.763 | 2.4560 | 6000 | 0.7380 |
353
+ | 0.7807 | 2.4642 | 6020 | 0.7380 |
354
+ | 0.7875 | 2.4724 | 6040 | 0.7374 |
355
+ | 0.7711 | 2.4806 | 6060 | 0.7376 |
356
+ | 0.7774 | 2.4887 | 6080 | 0.7384 |
357
+ | 0.7843 | 2.4969 | 6100 | 0.7377 |
358
+ | 0.7717 | 2.5051 | 6120 | 0.7375 |
359
+ | 0.7611 | 2.5133 | 6140 | 0.7372 |
360
+ | 0.7804 | 2.5215 | 6160 | 0.7373 |
361
+ | 0.7818 | 2.5297 | 6180 | 0.7377 |
362
+ | 0.7635 | 2.5379 | 6200 | 0.7373 |
363
+ | 0.7699 | 2.5460 | 6220 | 0.7381 |
364
+ | 0.7751 | 2.5542 | 6240 | 0.7378 |
365
+ | 0.7729 | 2.5624 | 6260 | 0.7384 |
366
+ | 0.7645 | 2.5706 | 6280 | 0.7375 |
367
+ | 0.7653 | 2.5788 | 6300 | 0.7381 |
368
+ | 0.7776 | 2.5870 | 6320 | 0.7383 |
369
+ | 0.7812 | 2.5952 | 6340 | 0.7376 |
370
+ | 0.7597 | 2.6034 | 6360 | 0.7374 |
371
+ | 0.7627 | 2.6115 | 6380 | 0.7370 |
372
+ | 0.7722 | 2.6197 | 6400 | 0.7378 |
373
+ | 0.7832 | 2.6279 | 6420 | 0.7373 |
374
+ | 0.7723 | 2.6361 | 6440 | 0.7370 |
375
+ | 0.7655 | 2.6443 | 6460 | 0.7372 |
376
+ | 0.7825 | 2.6525 | 6480 | 0.7373 |
377
+ | 0.7677 | 2.6607 | 6500 | 0.7377 |
378
+ | 0.7728 | 2.6688 | 6520 | 0.7376 |
379
+ | 0.779 | 2.6770 | 6540 | 0.7370 |
380
+ | 0.7693 | 2.6852 | 6560 | 0.7369 |
381
+ | 0.7601 | 2.6934 | 6580 | 0.7374 |
382
+ | 0.7768 | 2.7016 | 6600 | 0.7373 |
383
+ | 0.7792 | 2.7098 | 6620 | 0.7373 |
384
+ | 0.7678 | 2.7180 | 6640 | 0.7374 |
385
+ | 0.7822 | 2.7262 | 6660 | 0.7376 |
386
+ | 0.7774 | 2.7343 | 6680 | 0.7371 |
387
+ | 0.7689 | 2.7425 | 6700 | 0.7373 |
388
+ | 0.7681 | 2.7507 | 6720 | 0.7373 |
389
+ | 0.7665 | 2.7589 | 6740 | 0.7374 |
390
+ | 0.7718 | 2.7671 | 6760 | 0.7372 |
391
+ | 0.7708 | 2.7753 | 6780 | 0.7375 |
392
+ | 0.7703 | 2.7835 | 6800 | 0.7374 |
393
+ | 0.7611 | 2.7916 | 6820 | 0.7372 |
394
+ | 0.7702 | 2.7998 | 6840 | 0.7375 |
395
+ | 0.7736 | 2.8080 | 6860 | 0.7376 |
396
+ | 0.7767 | 2.8162 | 6880 | 0.7371 |
397
+ | 0.7913 | 2.8244 | 6900 | 0.7369 |
398
+ | 0.7761 | 2.8326 | 6920 | 0.7375 |
399
+ | 0.7805 | 2.8408 | 6940 | 0.7377 |
400
+ | 0.7715 | 2.8490 | 6960 | 0.7374 |
401
+ | 0.77 | 2.8571 | 6980 | 0.7377 |
402
+ | 0.7688 | 2.8653 | 7000 | 0.7377 |
403
+ | 0.7721 | 2.8735 | 7020 | 0.7374 |
404
+ | 0.7834 | 2.8817 | 7040 | 0.7371 |
405
+ | 0.7747 | 2.8899 | 7060 | 0.7377 |
406
+ | 0.7817 | 2.8981 | 7080 | 0.7375 |
407
+ | 0.773 | 2.9063 | 7100 | 0.7371 |
408
+ | 0.7694 | 2.9144 | 7120 | 0.7377 |
409
+ | 0.7961 | 2.9226 | 7140 | 0.7374 |
410
+ | 0.7653 | 2.9308 | 7160 | 0.7377 |
411
+ | 0.7582 | 2.9390 | 7180 | 0.7375 |
412
+ | 0.775 | 2.9472 | 7200 | 0.7375 |
413
+ | 0.7741 | 2.9554 | 7220 | 0.7373 |
414
+ | 0.7789 | 2.9636 | 7240 | 0.7382 |
415
+ | 0.7632 | 2.9718 | 7260 | 0.7373 |
416
+ | 0.777 | 2.9799 | 7280 | 0.7370 |
417
+ | 0.7652 | 2.9881 | 7300 | 0.7370 |
418
+ | 0.7671 | 2.9963 | 7320 | 0.7371 |
419
 
420
 
421
  ### Framework versions
all_results.json CHANGED
@@ -1,13 +1,13 @@
1
  {
2
- "epoch": 47.0,
3
  "eval_loss": 0.7128096222877502,
4
  "eval_runtime": 11.4031,
5
  "eval_samples_per_second": 160.307,
6
  "eval_steps_per_second": 5.086,
7
  "perplexity": 2.039714041484612,
8
- "total_flos": 2.5044104326938624e+16,
9
- "train_loss": 0.1997737506061201,
10
- "train_runtime": 1307.9406,
11
- "train_samples_per_second": 326.009,
12
- "train_steps_per_second": 10.207
13
  }
 
1
  {
2
+ "epoch": 3.0,
3
  "eval_loss": 0.7128096222877502,
4
  "eval_runtime": 11.4031,
5
  "eval_samples_per_second": 160.307,
6
  "eval_steps_per_second": 5.086,
7
  "perplexity": 2.039714041484612,
8
+ "total_flos": 1.4043296291217408e+16,
9
+ "train_loss": 0.9056532961688859,
10
+ "train_runtime": 6572.1588,
11
+ "train_samples_per_second": 35.683,
12
+ "train_steps_per_second": 1.115
13
  }
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 47.0,
3
- "total_flos": 2.5044104326938624e+16,
4
- "train_loss": 0.1997737506061201,
5
- "train_runtime": 1307.9406,
6
- "train_samples_per_second": 326.009,
7
- "train_steps_per_second": 10.207
8
  }
 
1
  {
2
+ "epoch": 3.0,
3
+ "total_flos": 1.4043296291217408e+16,
4
+ "train_loss": 0.9056532961688859,
5
+ "train_runtime": 6572.1588,
6
+ "train_samples_per_second": 35.683,
7
+ "train_steps_per_second": 1.115
8
  }