Hugging Face's logo

Artanis1551
/
bert_trainer

Text Classification
Transformers
TensorBoard
Safetensors
bert
Generated from Trainer
Model card Files
xet
 Metrics Community
Artanis1551 commited on
Commit
351a4ac
·
1 Parent(s): 40f220b

End of training

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +57 -0
  2. config.json +41 -0
  3. model.safetensors +3 -0
  4. run-0/checkpoint-2500/config.json +41 -0
  5. run-0/checkpoint-2500/model.safetensors +3 -0
  6. run-0/checkpoint-2500/optimizer.pt +3 -0
  7. run-0/checkpoint-2500/rng_state.pth +3 -0
  8. run-0/checkpoint-2500/scheduler.pt +3 -0
  9. run-0/checkpoint-2500/trainer_state.json +714 -0
  10. run-0/checkpoint-2500/training_args.bin +3 -0
  11. run-0/checkpoint-2750/config.json +41 -0
  12. run-0/checkpoint-2750/model.safetensors +3 -0
  13. run-0/checkpoint-2750/optimizer.pt +3 -0
  14. run-0/checkpoint-2750/rng_state.pth +3 -0
  15. run-0/checkpoint-2750/scheduler.pt +3 -0
  16. run-0/checkpoint-2750/trainer_state.json +783 -0
  17. run-0/checkpoint-2750/training_args.bin +3 -0
  18. run-0/checkpoint-3000/config.json +41 -0
  19. run-0/checkpoint-3000/model.safetensors +3 -0
  20. run-0/checkpoint-3000/optimizer.pt +3 -0
  21. run-0/checkpoint-3000/rng_state.pth +3 -0
  22. run-0/checkpoint-3000/scheduler.pt +3 -0
  23. run-0/checkpoint-3000/trainer_state.json +852 -0
  24. run-0/checkpoint-3000/training_args.bin +3 -0
  25. run-1/checkpoint-2000/config.json +41 -0
  26. run-1/checkpoint-2000/model.safetensors +3 -0
  27. run-1/checkpoint-2000/optimizer.pt +3 -0
  28. run-1/checkpoint-2000/rng_state.pth +3 -0
  29. run-1/checkpoint-2000/scheduler.pt +3 -0
  30. run-1/checkpoint-2000/trainer_state.json +576 -0
  31. run-1/checkpoint-2000/training_args.bin +3 -0
  32. run-1/checkpoint-2750/config.json +41 -0
  33. run-1/checkpoint-2750/model.safetensors +3 -0
  34. run-1/checkpoint-2750/optimizer.pt +3 -0
  35. run-1/checkpoint-2750/rng_state.pth +3 -0
  36. run-1/checkpoint-2750/scheduler.pt +3 -0
  37. run-1/checkpoint-2750/trainer_state.json +783 -0
  38. run-1/checkpoint-2750/training_args.bin +3 -0
  39. run-1/checkpoint-3000/config.json +41 -0
  40. run-1/checkpoint-3000/model.safetensors +3 -0
  41. run-1/checkpoint-3000/optimizer.pt +3 -0
  42. run-1/checkpoint-3000/rng_state.pth +3 -0
  43. run-1/checkpoint-3000/scheduler.pt +3 -0
  44. run-1/checkpoint-3000/trainer_state.json +852 -0
  45. run-1/checkpoint-3000/training_args.bin +3 -0
  46. run-2/checkpoint-2500/config.json +41 -0
  47. run-2/checkpoint-2500/model.safetensors +3 -0
  48. run-2/checkpoint-2500/optimizer.pt +3 -0
  49. run-2/checkpoint-2500/rng_state.pth +3 -0
  50. run-2/checkpoint-2500/scheduler.pt +3 -0
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: bert-base-cased
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: bert_trainer
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # bert_trainer
15
+
16
+ This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the None dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - eval_loss: 0.6901
19
+ - eval_accuracy: 0.7558
20
+ - eval_runtime: 30.0507
21
+ - eval_samples_per_second: 103.292
22
+ - eval_steps_per_second: 25.823
23
+ - epoch: 0.16
24
+ - step: 250
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 6.036609853019458e-06
44
+ - train_batch_size: 8
45
+ - eval_batch_size: 4
46
+ - seed: 42
47
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
+ - lr_scheduler_type: constant_with_warmup
49
+ - lr_scheduler_warmup_steps: 50
50
+ - training_steps: 3000
51
+
52
+ ### Framework versions
53
+
54
+ - Transformers 4.36.0.dev0
55
+ - Pytorch 2.1.1+cu121
56
+ - Datasets 2.15.0
57
+ - Tokenizers 0.15.0
config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd32e24b75429e0cb9367040b8e23512009bab779fbedc84c7cd676a547332c8
3
+ size 433279996
run-0/checkpoint-2500/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-0/checkpoint-2500/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d3e6ee528249c0e1c6388d2e72c4e47db8e42f96022ddc8f107119bbde5e135
3
+ size 433279996
run-0/checkpoint-2500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48b0eabf74e09755cd69eb9a84a842e5681649a25f2c588f1e78331b8358fd78
3
+ size 866681082
run-0/checkpoint-2500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e254f06883fec1e30ea9be59e97a84e2814064af09a93a2e0e9f4655ea082cd
3
+ size 14244
run-0/checkpoint-2500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06617cdbcc181bf9ec7d4df7aa9e71ef685d4759e304bde8f28a58df45e09177
3
+ size 1064
run-0/checkpoint-2500/trainer_state.json ADDED
@@ -0,0 +1,714 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8743556701030928,
3
+ "best_model_checkpoint": "bert_trainer/run-0/checkpoint-1750",
4
+ "epoch": 3.2216494845360826,
5
+ "eval_steps": 250,
6
+ "global_step": 2500,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 2.650169893369787e-06,
14
+ "loss": 1.4561,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 5.300339786739574e-06,
20
+ "loss": 1.2146,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 5.300339786739574e-06,
26
+ "loss": 1.0949,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 5.300339786739574e-06,
32
+ "loss": 0.9846,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 5.300339786739574e-06,
38
+ "loss": 0.9161,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 5.300339786739574e-06,
44
+ "loss": 0.8466,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 5.300339786739574e-06,
50
+ "loss": 0.7954,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 5.300339786739574e-06,
56
+ "loss": 0.7311,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 5.300339786739574e-06,
62
+ "loss": 0.6688,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 5.300339786739574e-06,
68
+ "loss": 0.6816,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.7857603092783505,
74
+ "eval_loss": 0.6313713192939758,
75
+ "eval_runtime": 78.939,
76
+ "eval_samples_per_second": 39.322,
77
+ "eval_steps_per_second": 9.83,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 5.300339786739574e-06,
83
+ "loss": 0.6339,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 5.300339786739574e-06,
89
+ "loss": 0.6102,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 5.300339786739574e-06,
95
+ "loss": 0.6572,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 5.300339786739574e-06,
101
+ "loss": 0.6575,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 5.300339786739574e-06,
107
+ "loss": 0.5946,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 5.300339786739574e-06,
113
+ "loss": 0.5779,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 5.300339786739574e-06,
119
+ "loss": 0.5773,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 5.300339786739574e-06,
125
+ "loss": 0.5461,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 5.300339786739574e-06,
131
+ "loss": 0.576,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 5.300339786739574e-06,
137
+ "loss": 0.4864,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8176546391752577,
143
+ "eval_loss": 0.5011568069458008,
144
+ "eval_runtime": 78.7987,
145
+ "eval_samples_per_second": 39.392,
146
+ "eval_steps_per_second": 9.848,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 5.300339786739574e-06,
152
+ "loss": 0.5618,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 5.300339786739574e-06,
158
+ "loss": 0.5325,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 5.300339786739574e-06,
164
+ "loss": 0.4656,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 5.300339786739574e-06,
170
+ "loss": 0.4966,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 5.300339786739574e-06,
176
+ "loss": 0.457,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 5.300339786739574e-06,
182
+ "loss": 0.4715,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 5.300339786739574e-06,
188
+ "loss": 0.4621,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 5.300339786739574e-06,
194
+ "loss": 0.4679,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 5.300339786739574e-06,
200
+ "loss": 0.4236,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 5.300339786739574e-06,
206
+ "loss": 0.399,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8443943298969072,
212
+ "eval_loss": 0.4396892488002777,
213
+ "eval_runtime": 78.9384,
214
+ "eval_samples_per_second": 39.322,
215
+ "eval_steps_per_second": 9.83,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 5.300339786739574e-06,
221
+ "loss": 0.4321,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 5.300339786739574e-06,
227
+ "loss": 0.3953,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 5.300339786739574e-06,
233
+ "loss": 0.379,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 5.300339786739574e-06,
239
+ "loss": 0.3415,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 5.300339786739574e-06,
245
+ "loss": 0.3705,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 5.300339786739574e-06,
251
+ "loss": 0.3612,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 5.300339786739574e-06,
257
+ "loss": 0.4108,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 5.300339786739574e-06,
263
+ "loss": 0.4167,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 5.300339786739574e-06,
269
+ "loss": 0.3916,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 5.300339786739574e-06,
275
+ "loss": 0.3947,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8579252577319587,
281
+ "eval_loss": 0.40539947152137756,
282
+ "eval_runtime": 78.7689,
283
+ "eval_samples_per_second": 39.406,
284
+ "eval_steps_per_second": 9.852,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 5.300339786739574e-06,
290
+ "loss": 0.3812,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 5.300339786739574e-06,
296
+ "loss": 0.3804,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 5.300339786739574e-06,
302
+ "loss": 0.3783,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 5.300339786739574e-06,
308
+ "loss": 0.337,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 5.300339786739574e-06,
314
+ "loss": 0.3191,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 5.300339786739574e-06,
320
+ "loss": 0.3277,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 5.300339786739574e-06,
326
+ "loss": 0.3601,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 5.300339786739574e-06,
332
+ "loss": 0.3932,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 5.300339786739574e-06,
338
+ "loss": 0.3964,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 5.300339786739574e-06,
344
+ "loss": 0.3705,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8524484536082474,
350
+ "eval_loss": 0.40534982085227966,
351
+ "eval_runtime": 78.953,
352
+ "eval_samples_per_second": 39.315,
353
+ "eval_steps_per_second": 9.829,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 5.300339786739574e-06,
359
+ "loss": 0.4567,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 5.300339786739574e-06,
365
+ "loss": 0.384,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 5.300339786739574e-06,
371
+ "loss": 0.3205,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 5.300339786739574e-06,
377
+ "loss": 0.3841,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 5.300339786739574e-06,
383
+ "loss": 0.322,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 5.300339786739574e-06,
389
+ "loss": 0.3742,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 5.300339786739574e-06,
395
+ "loss": 0.3407,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 5.300339786739574e-06,
401
+ "loss": 0.3342,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 5.300339786739574e-06,
407
+ "loss": 0.3068,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 5.300339786739574e-06,
413
+ "loss": 0.3288,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8688788659793815,
419
+ "eval_loss": 0.36364319920539856,
420
+ "eval_runtime": 78.9463,
421
+ "eval_samples_per_second": 39.318,
422
+ "eval_steps_per_second": 9.829,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 5.300339786739574e-06,
428
+ "loss": 0.2918,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 5.300339786739574e-06,
434
+ "loss": 0.3571,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 5.300339786739574e-06,
440
+ "loss": 0.3341,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 5.300339786739574e-06,
446
+ "loss": 0.3162,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 5.300339786739574e-06,
452
+ "loss": 0.2384,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 5.300339786739574e-06,
458
+ "loss": 0.2934,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 5.300339786739574e-06,
464
+ "loss": 0.2686,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 5.300339786739574e-06,
470
+ "loss": 0.2806,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 5.300339786739574e-06,
476
+ "loss": 0.2897,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 5.300339786739574e-06,
482
+ "loss": 0.2417,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8743556701030928,
488
+ "eval_loss": 0.39001935720443726,
489
+ "eval_runtime": 78.9576,
490
+ "eval_samples_per_second": 39.312,
491
+ "eval_steps_per_second": 9.828,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 5.300339786739574e-06,
497
+ "loss": 0.2849,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 5.300339786739574e-06,
503
+ "loss": 0.2653,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 5.300339786739574e-06,
509
+ "loss": 0.2266,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 5.300339786739574e-06,
515
+ "loss": 0.2401,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 5.300339786739574e-06,
521
+ "loss": 0.2208,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 5.300339786739574e-06,
527
+ "loss": 0.2615,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 5.300339786739574e-06,
533
+ "loss": 0.2913,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 5.300339786739574e-06,
539
+ "loss": 0.2474,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 5.300339786739574e-06,
545
+ "loss": 0.2388,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 5.300339786739574e-06,
551
+ "loss": 0.2374,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8730670103092784,
557
+ "eval_loss": 0.39385583996772766,
558
+ "eval_runtime": 78.9575,
559
+ "eval_samples_per_second": 39.312,
560
+ "eval_steps_per_second": 9.828,
561
+ "step": 2000
562
+ },
563
+ {
564
+ "epoch": 2.61,
565
+ "learning_rate": 5.300339786739574e-06,
566
+ "loss": 0.2697,
567
+ "step": 2025
568
+ },
569
+ {
570
+ "epoch": 2.64,
571
+ "learning_rate": 5.300339786739574e-06,
572
+ "loss": 0.2592,
573
+ "step": 2050
574
+ },
575
+ {
576
+ "epoch": 2.67,
577
+ "learning_rate": 5.300339786739574e-06,
578
+ "loss": 0.2636,
579
+ "step": 2075
580
+ },
581
+ {
582
+ "epoch": 2.71,
583
+ "learning_rate": 5.300339786739574e-06,
584
+ "loss": 0.2349,
585
+ "step": 2100
586
+ },
587
+ {
588
+ "epoch": 2.74,
589
+ "learning_rate": 5.300339786739574e-06,
590
+ "loss": 0.2697,
591
+ "step": 2125
592
+ },
593
+ {
594
+ "epoch": 2.77,
595
+ "learning_rate": 5.300339786739574e-06,
596
+ "loss": 0.2507,
597
+ "step": 2150
598
+ },
599
+ {
600
+ "epoch": 2.8,
601
+ "learning_rate": 5.300339786739574e-06,
602
+ "loss": 0.2608,
603
+ "step": 2175
604
+ },
605
+ {
606
+ "epoch": 2.84,
607
+ "learning_rate": 5.300339786739574e-06,
608
+ "loss": 0.2463,
609
+ "step": 2200
610
+ },
611
+ {
612
+ "epoch": 2.87,
613
+ "learning_rate": 5.300339786739574e-06,
614
+ "loss": 0.2408,
615
+ "step": 2225
616
+ },
617
+ {
618
+ "epoch": 2.9,
619
+ "learning_rate": 5.300339786739574e-06,
620
+ "loss": 0.3525,
621
+ "step": 2250
622
+ },
623
+ {
624
+ "epoch": 2.9,
625
+ "eval_accuracy": 0.8627577319587629,
626
+ "eval_loss": 0.4113520085811615,
627
+ "eval_runtime": 78.9601,
628
+ "eval_samples_per_second": 39.311,
629
+ "eval_steps_per_second": 9.828,
630
+ "step": 2250
631
+ },
632
+ {
633
+ "epoch": 2.93,
634
+ "learning_rate": 5.300339786739574e-06,
635
+ "loss": 0.3033,
636
+ "step": 2275
637
+ },
638
+ {
639
+ "epoch": 2.96,
640
+ "learning_rate": 5.300339786739574e-06,
641
+ "loss": 0.2186,
642
+ "step": 2300
643
+ },
644
+ {
645
+ "epoch": 3.0,
646
+ "learning_rate": 5.300339786739574e-06,
647
+ "loss": 0.2619,
648
+ "step": 2325
649
+ },
650
+ {
651
+ "epoch": 3.03,
652
+ "learning_rate": 5.300339786739574e-06,
653
+ "loss": 0.1473,
654
+ "step": 2350
655
+ },
656
+ {
657
+ "epoch": 3.06,
658
+ "learning_rate": 5.300339786739574e-06,
659
+ "loss": 0.1662,
660
+ "step": 2375
661
+ },
662
+ {
663
+ "epoch": 3.09,
664
+ "learning_rate": 5.300339786739574e-06,
665
+ "loss": 0.1863,
666
+ "step": 2400
667
+ },
668
+ {
669
+ "epoch": 3.12,
670
+ "learning_rate": 5.300339786739574e-06,
671
+ "loss": 0.1869,
672
+ "step": 2425
673
+ },
674
+ {
675
+ "epoch": 3.16,
676
+ "learning_rate": 5.300339786739574e-06,
677
+ "loss": 0.1729,
678
+ "step": 2450
679
+ },
680
+ {
681
+ "epoch": 3.19,
682
+ "learning_rate": 5.300339786739574e-06,
683
+ "loss": 0.1998,
684
+ "step": 2475
685
+ },
686
+ {
687
+ "epoch": 3.22,
688
+ "learning_rate": 5.300339786739574e-06,
689
+ "loss": 0.1778,
690
+ "step": 2500
691
+ },
692
+ {
693
+ "epoch": 3.22,
694
+ "eval_accuracy": 0.873389175257732,
695
+ "eval_loss": 0.44405418634414673,
696
+ "eval_runtime": 78.9751,
697
+ "eval_samples_per_second": 39.304,
698
+ "eval_steps_per_second": 9.826,
699
+ "step": 2500
700
+ }
701
+ ],
702
+ "logging_steps": 25,
703
+ "max_steps": 3000,
704
+ "num_input_tokens_seen": 0,
705
+ "num_train_epochs": 4,
706
+ "save_steps": 250,
707
+ "total_flos": 1.0521042044565504e+16,
708
+ "trial_name": null,
709
+ "trial_params": {
710
+ "learning_rate": 5.300339786739574e-06,
711
+ "per_device_eval_batch_size": 4,
712
+ "per_device_train_batch_size": 16
713
+ }
714
+ }
run-0/checkpoint-2500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff966c9d210a3dcc91765468d3bd580c3a94cec8f318bfe2ceb18c02bfa0afcf
3
+ size 4728
run-0/checkpoint-2750/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-0/checkpoint-2750/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1336ad1a77da09cbf25876d022a58770130c3b2ffe4b80c420562bcdc7d1ed9d
3
+ size 433279996
run-0/checkpoint-2750/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1162a3ddf040cd09a94a4dd7463544f40b0659e8996a1d72eb0f3668b66b34d1
3
+ size 866681082
run-0/checkpoint-2750/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2f146112a2b6651d5740f314c7a7760a6b10b307030d3ca8f929af57ce20006
3
+ size 14244
run-0/checkpoint-2750/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2990860678a4cf60168b8269ff8709a95193f5ee58d391b8ba0badb27d72ce0
3
+ size 1064
run-0/checkpoint-2750/trainer_state.json ADDED
@@ -0,0 +1,783 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8743556701030928,
3
+ "best_model_checkpoint": "bert_trainer/run-0/checkpoint-1750",
4
+ "epoch": 3.5438144329896906,
5
+ "eval_steps": 250,
6
+ "global_step": 2750,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 2.650169893369787e-06,
14
+ "loss": 1.4561,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 5.300339786739574e-06,
20
+ "loss": 1.2146,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 5.300339786739574e-06,
26
+ "loss": 1.0949,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 5.300339786739574e-06,
32
+ "loss": 0.9846,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 5.300339786739574e-06,
38
+ "loss": 0.9161,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 5.300339786739574e-06,
44
+ "loss": 0.8466,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 5.300339786739574e-06,
50
+ "loss": 0.7954,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 5.300339786739574e-06,
56
+ "loss": 0.7311,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 5.300339786739574e-06,
62
+ "loss": 0.6688,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 5.300339786739574e-06,
68
+ "loss": 0.6816,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.7857603092783505,
74
+ "eval_loss": 0.6313713192939758,
75
+ "eval_runtime": 78.939,
76
+ "eval_samples_per_second": 39.322,
77
+ "eval_steps_per_second": 9.83,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 5.300339786739574e-06,
83
+ "loss": 0.6339,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 5.300339786739574e-06,
89
+ "loss": 0.6102,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 5.300339786739574e-06,
95
+ "loss": 0.6572,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 5.300339786739574e-06,
101
+ "loss": 0.6575,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 5.300339786739574e-06,
107
+ "loss": 0.5946,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 5.300339786739574e-06,
113
+ "loss": 0.5779,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 5.300339786739574e-06,
119
+ "loss": 0.5773,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 5.300339786739574e-06,
125
+ "loss": 0.5461,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 5.300339786739574e-06,
131
+ "loss": 0.576,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 5.300339786739574e-06,
137
+ "loss": 0.4864,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8176546391752577,
143
+ "eval_loss": 0.5011568069458008,
144
+ "eval_runtime": 78.7987,
145
+ "eval_samples_per_second": 39.392,
146
+ "eval_steps_per_second": 9.848,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 5.300339786739574e-06,
152
+ "loss": 0.5618,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 5.300339786739574e-06,
158
+ "loss": 0.5325,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 5.300339786739574e-06,
164
+ "loss": 0.4656,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 5.300339786739574e-06,
170
+ "loss": 0.4966,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 5.300339786739574e-06,
176
+ "loss": 0.457,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 5.300339786739574e-06,
182
+ "loss": 0.4715,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 5.300339786739574e-06,
188
+ "loss": 0.4621,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 5.300339786739574e-06,
194
+ "loss": 0.4679,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 5.300339786739574e-06,
200
+ "loss": 0.4236,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 5.300339786739574e-06,
206
+ "loss": 0.399,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8443943298969072,
212
+ "eval_loss": 0.4396892488002777,
213
+ "eval_runtime": 78.9384,
214
+ "eval_samples_per_second": 39.322,
215
+ "eval_steps_per_second": 9.83,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 5.300339786739574e-06,
221
+ "loss": 0.4321,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 5.300339786739574e-06,
227
+ "loss": 0.3953,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 5.300339786739574e-06,
233
+ "loss": 0.379,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 5.300339786739574e-06,
239
+ "loss": 0.3415,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 5.300339786739574e-06,
245
+ "loss": 0.3705,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 5.300339786739574e-06,
251
+ "loss": 0.3612,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 5.300339786739574e-06,
257
+ "loss": 0.4108,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 5.300339786739574e-06,
263
+ "loss": 0.4167,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 5.300339786739574e-06,
269
+ "loss": 0.3916,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 5.300339786739574e-06,
275
+ "loss": 0.3947,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8579252577319587,
281
+ "eval_loss": 0.40539947152137756,
282
+ "eval_runtime": 78.7689,
283
+ "eval_samples_per_second": 39.406,
284
+ "eval_steps_per_second": 9.852,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 5.300339786739574e-06,
290
+ "loss": 0.3812,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 5.300339786739574e-06,
296
+ "loss": 0.3804,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 5.300339786739574e-06,
302
+ "loss": 0.3783,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 5.300339786739574e-06,
308
+ "loss": 0.337,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 5.300339786739574e-06,
314
+ "loss": 0.3191,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 5.300339786739574e-06,
320
+ "loss": 0.3277,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 5.300339786739574e-06,
326
+ "loss": 0.3601,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 5.300339786739574e-06,
332
+ "loss": 0.3932,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 5.300339786739574e-06,
338
+ "loss": 0.3964,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 5.300339786739574e-06,
344
+ "loss": 0.3705,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8524484536082474,
350
+ "eval_loss": 0.40534982085227966,
351
+ "eval_runtime": 78.953,
352
+ "eval_samples_per_second": 39.315,
353
+ "eval_steps_per_second": 9.829,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 5.300339786739574e-06,
359
+ "loss": 0.4567,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 5.300339786739574e-06,
365
+ "loss": 0.384,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 5.300339786739574e-06,
371
+ "loss": 0.3205,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 5.300339786739574e-06,
377
+ "loss": 0.3841,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 5.300339786739574e-06,
383
+ "loss": 0.322,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 5.300339786739574e-06,
389
+ "loss": 0.3742,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 5.300339786739574e-06,
395
+ "loss": 0.3407,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 5.300339786739574e-06,
401
+ "loss": 0.3342,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 5.300339786739574e-06,
407
+ "loss": 0.3068,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 5.300339786739574e-06,
413
+ "loss": 0.3288,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8688788659793815,
419
+ "eval_loss": 0.36364319920539856,
420
+ "eval_runtime": 78.9463,
421
+ "eval_samples_per_second": 39.318,
422
+ "eval_steps_per_second": 9.829,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 5.300339786739574e-06,
428
+ "loss": 0.2918,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 5.300339786739574e-06,
434
+ "loss": 0.3571,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 5.300339786739574e-06,
440
+ "loss": 0.3341,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 5.300339786739574e-06,
446
+ "loss": 0.3162,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 5.300339786739574e-06,
452
+ "loss": 0.2384,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 5.300339786739574e-06,
458
+ "loss": 0.2934,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 5.300339786739574e-06,
464
+ "loss": 0.2686,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 5.300339786739574e-06,
470
+ "loss": 0.2806,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 5.300339786739574e-06,
476
+ "loss": 0.2897,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 5.300339786739574e-06,
482
+ "loss": 0.2417,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8743556701030928,
488
+ "eval_loss": 0.39001935720443726,
489
+ "eval_runtime": 78.9576,
490
+ "eval_samples_per_second": 39.312,
491
+ "eval_steps_per_second": 9.828,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 5.300339786739574e-06,
497
+ "loss": 0.2849,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 5.300339786739574e-06,
503
+ "loss": 0.2653,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 5.300339786739574e-06,
509
+ "loss": 0.2266,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 5.300339786739574e-06,
515
+ "loss": 0.2401,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 5.300339786739574e-06,
521
+ "loss": 0.2208,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 5.300339786739574e-06,
527
+ "loss": 0.2615,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 5.300339786739574e-06,
533
+ "loss": 0.2913,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 5.300339786739574e-06,
539
+ "loss": 0.2474,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 5.300339786739574e-06,
545
+ "loss": 0.2388,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 5.300339786739574e-06,
551
+ "loss": 0.2374,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8730670103092784,
557
+ "eval_loss": 0.39385583996772766,
558
+ "eval_runtime": 78.9575,
559
+ "eval_samples_per_second": 39.312,
560
+ "eval_steps_per_second": 9.828,
561
+ "step": 2000
562
+ },
563
+ {
564
+ "epoch": 2.61,
565
+ "learning_rate": 5.300339786739574e-06,
566
+ "loss": 0.2697,
567
+ "step": 2025
568
+ },
569
+ {
570
+ "epoch": 2.64,
571
+ "learning_rate": 5.300339786739574e-06,
572
+ "loss": 0.2592,
573
+ "step": 2050
574
+ },
575
+ {
576
+ "epoch": 2.67,
577
+ "learning_rate": 5.300339786739574e-06,
578
+ "loss": 0.2636,
579
+ "step": 2075
580
+ },
581
+ {
582
+ "epoch": 2.71,
583
+ "learning_rate": 5.300339786739574e-06,
584
+ "loss": 0.2349,
585
+ "step": 2100
586
+ },
587
+ {
588
+ "epoch": 2.74,
589
+ "learning_rate": 5.300339786739574e-06,
590
+ "loss": 0.2697,
591
+ "step": 2125
592
+ },
593
+ {
594
+ "epoch": 2.77,
595
+ "learning_rate": 5.300339786739574e-06,
596
+ "loss": 0.2507,
597
+ "step": 2150
598
+ },
599
+ {
600
+ "epoch": 2.8,
601
+ "learning_rate": 5.300339786739574e-06,
602
+ "loss": 0.2608,
603
+ "step": 2175
604
+ },
605
+ {
606
+ "epoch": 2.84,
607
+ "learning_rate": 5.300339786739574e-06,
608
+ "loss": 0.2463,
609
+ "step": 2200
610
+ },
611
+ {
612
+ "epoch": 2.87,
613
+ "learning_rate": 5.300339786739574e-06,
614
+ "loss": 0.2408,
615
+ "step": 2225
616
+ },
617
+ {
618
+ "epoch": 2.9,
619
+ "learning_rate": 5.300339786739574e-06,
620
+ "loss": 0.3525,
621
+ "step": 2250
622
+ },
623
+ {
624
+ "epoch": 2.9,
625
+ "eval_accuracy": 0.8627577319587629,
626
+ "eval_loss": 0.4113520085811615,
627
+ "eval_runtime": 78.9601,
628
+ "eval_samples_per_second": 39.311,
629
+ "eval_steps_per_second": 9.828,
630
+ "step": 2250
631
+ },
632
+ {
633
+ "epoch": 2.93,
634
+ "learning_rate": 5.300339786739574e-06,
635
+ "loss": 0.3033,
636
+ "step": 2275
637
+ },
638
+ {
639
+ "epoch": 2.96,
640
+ "learning_rate": 5.300339786739574e-06,
641
+ "loss": 0.2186,
642
+ "step": 2300
643
+ },
644
+ {
645
+ "epoch": 3.0,
646
+ "learning_rate": 5.300339786739574e-06,
647
+ "loss": 0.2619,
648
+ "step": 2325
649
+ },
650
+ {
651
+ "epoch": 3.03,
652
+ "learning_rate": 5.300339786739574e-06,
653
+ "loss": 0.1473,
654
+ "step": 2350
655
+ },
656
+ {
657
+ "epoch": 3.06,
658
+ "learning_rate": 5.300339786739574e-06,
659
+ "loss": 0.1662,
660
+ "step": 2375
661
+ },
662
+ {
663
+ "epoch": 3.09,
664
+ "learning_rate": 5.300339786739574e-06,
665
+ "loss": 0.1863,
666
+ "step": 2400
667
+ },
668
+ {
669
+ "epoch": 3.12,
670
+ "learning_rate": 5.300339786739574e-06,
671
+ "loss": 0.1869,
672
+ "step": 2425
673
+ },
674
+ {
675
+ "epoch": 3.16,
676
+ "learning_rate": 5.300339786739574e-06,
677
+ "loss": 0.1729,
678
+ "step": 2450
679
+ },
680
+ {
681
+ "epoch": 3.19,
682
+ "learning_rate": 5.300339786739574e-06,
683
+ "loss": 0.1998,
684
+ "step": 2475
685
+ },
686
+ {
687
+ "epoch": 3.22,
688
+ "learning_rate": 5.300339786739574e-06,
689
+ "loss": 0.1778,
690
+ "step": 2500
691
+ },
692
+ {
693
+ "epoch": 3.22,
694
+ "eval_accuracy": 0.873389175257732,
695
+ "eval_loss": 0.44405418634414673,
696
+ "eval_runtime": 78.9751,
697
+ "eval_samples_per_second": 39.304,
698
+ "eval_steps_per_second": 9.826,
699
+ "step": 2500
700
+ },
701
+ {
702
+ "epoch": 3.25,
703
+ "learning_rate": 5.300339786739574e-06,
704
+ "loss": 0.171,
705
+ "step": 2525
706
+ },
707
+ {
708
+ "epoch": 3.29,
709
+ "learning_rate": 5.300339786739574e-06,
710
+ "loss": 0.1642,
711
+ "step": 2550
712
+ },
713
+ {
714
+ "epoch": 3.32,
715
+ "learning_rate": 5.300339786739574e-06,
716
+ "loss": 0.2303,
717
+ "step": 2575
718
+ },
719
+ {
720
+ "epoch": 3.35,
721
+ "learning_rate": 5.300339786739574e-06,
722
+ "loss": 0.2194,
723
+ "step": 2600
724
+ },
725
+ {
726
+ "epoch": 3.38,
727
+ "learning_rate": 5.300339786739574e-06,
728
+ "loss": 0.1756,
729
+ "step": 2625
730
+ },
731
+ {
732
+ "epoch": 3.41,
733
+ "learning_rate": 5.300339786739574e-06,
734
+ "loss": 0.1956,
735
+ "step": 2650
736
+ },
737
+ {
738
+ "epoch": 3.45,
739
+ "learning_rate": 5.300339786739574e-06,
740
+ "loss": 0.2669,
741
+ "step": 2675
742
+ },
743
+ {
744
+ "epoch": 3.48,
745
+ "learning_rate": 5.300339786739574e-06,
746
+ "loss": 0.1514,
747
+ "step": 2700
748
+ },
749
+ {
750
+ "epoch": 3.51,
751
+ "learning_rate": 5.300339786739574e-06,
752
+ "loss": 0.1263,
753
+ "step": 2725
754
+ },
755
+ {
756
+ "epoch": 3.54,
757
+ "learning_rate": 5.300339786739574e-06,
758
+ "loss": 0.259,
759
+ "step": 2750
760
+ },
761
+ {
762
+ "epoch": 3.54,
763
+ "eval_accuracy": 0.8566365979381443,
764
+ "eval_loss": 0.4988997280597687,
765
+ "eval_runtime": 78.9372,
766
+ "eval_samples_per_second": 39.322,
767
+ "eval_steps_per_second": 9.831,
768
+ "step": 2750
769
+ }
770
+ ],
771
+ "logging_steps": 25,
772
+ "max_steps": 3000,
773
+ "num_input_tokens_seen": 0,
774
+ "num_train_epochs": 4,
775
+ "save_steps": 250,
776
+ "total_flos": 1.1573514614421504e+16,
777
+ "trial_name": null,
778
+ "trial_params": {
779
+ "learning_rate": 5.300339786739574e-06,
780
+ "per_device_eval_batch_size": 4,
781
+ "per_device_train_batch_size": 16
782
+ }
783
+ }
run-0/checkpoint-2750/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff966c9d210a3dcc91765468d3bd580c3a94cec8f318bfe2ceb18c02bfa0afcf
3
+ size 4728
run-0/checkpoint-3000/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-0/checkpoint-3000/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97635f937f6839acf8375df2ae47c49059b50b550f9a87c413496a140714f269
3
+ size 433279996
run-0/checkpoint-3000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:255f8e93744d2bc7a27bc7622955b2b19abef2409dcd84b03f15ac2f69afffbf
3
+ size 866681082
run-0/checkpoint-3000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5475dcc749e5bb84d23e503506ffb30e2c214153e88098ffa2eba5466de55a64
3
+ size 14244
run-0/checkpoint-3000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11b18eab96c7f27ea28cf71b3d2f6754e002f163c751b2006fe6b5fd2f4b1877
3
+ size 1064
run-0/checkpoint-3000/trainer_state.json ADDED
@@ -0,0 +1,852 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8798324742268041,
3
+ "best_model_checkpoint": "bert_trainer/run-0/checkpoint-3000",
4
+ "epoch": 3.865979381443299,
5
+ "eval_steps": 250,
6
+ "global_step": 3000,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 2.650169893369787e-06,
14
+ "loss": 1.4561,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 5.300339786739574e-06,
20
+ "loss": 1.2146,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 5.300339786739574e-06,
26
+ "loss": 1.0949,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 5.300339786739574e-06,
32
+ "loss": 0.9846,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 5.300339786739574e-06,
38
+ "loss": 0.9161,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 5.300339786739574e-06,
44
+ "loss": 0.8466,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 5.300339786739574e-06,
50
+ "loss": 0.7954,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 5.300339786739574e-06,
56
+ "loss": 0.7311,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 5.300339786739574e-06,
62
+ "loss": 0.6688,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 5.300339786739574e-06,
68
+ "loss": 0.6816,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.7857603092783505,
74
+ "eval_loss": 0.6313713192939758,
75
+ "eval_runtime": 78.939,
76
+ "eval_samples_per_second": 39.322,
77
+ "eval_steps_per_second": 9.83,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 5.300339786739574e-06,
83
+ "loss": 0.6339,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 5.300339786739574e-06,
89
+ "loss": 0.6102,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 5.300339786739574e-06,
95
+ "loss": 0.6572,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 5.300339786739574e-06,
101
+ "loss": 0.6575,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 5.300339786739574e-06,
107
+ "loss": 0.5946,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 5.300339786739574e-06,
113
+ "loss": 0.5779,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 5.300339786739574e-06,
119
+ "loss": 0.5773,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 5.300339786739574e-06,
125
+ "loss": 0.5461,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 5.300339786739574e-06,
131
+ "loss": 0.576,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 5.300339786739574e-06,
137
+ "loss": 0.4864,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8176546391752577,
143
+ "eval_loss": 0.5011568069458008,
144
+ "eval_runtime": 78.7987,
145
+ "eval_samples_per_second": 39.392,
146
+ "eval_steps_per_second": 9.848,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 5.300339786739574e-06,
152
+ "loss": 0.5618,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 5.300339786739574e-06,
158
+ "loss": 0.5325,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 5.300339786739574e-06,
164
+ "loss": 0.4656,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 5.300339786739574e-06,
170
+ "loss": 0.4966,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 5.300339786739574e-06,
176
+ "loss": 0.457,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 5.300339786739574e-06,
182
+ "loss": 0.4715,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 5.300339786739574e-06,
188
+ "loss": 0.4621,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 5.300339786739574e-06,
194
+ "loss": 0.4679,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 5.300339786739574e-06,
200
+ "loss": 0.4236,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 5.300339786739574e-06,
206
+ "loss": 0.399,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8443943298969072,
212
+ "eval_loss": 0.4396892488002777,
213
+ "eval_runtime": 78.9384,
214
+ "eval_samples_per_second": 39.322,
215
+ "eval_steps_per_second": 9.83,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 5.300339786739574e-06,
221
+ "loss": 0.4321,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 5.300339786739574e-06,
227
+ "loss": 0.3953,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 5.300339786739574e-06,
233
+ "loss": 0.379,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 5.300339786739574e-06,
239
+ "loss": 0.3415,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 5.300339786739574e-06,
245
+ "loss": 0.3705,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 5.300339786739574e-06,
251
+ "loss": 0.3612,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 5.300339786739574e-06,
257
+ "loss": 0.4108,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 5.300339786739574e-06,
263
+ "loss": 0.4167,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 5.300339786739574e-06,
269
+ "loss": 0.3916,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 5.300339786739574e-06,
275
+ "loss": 0.3947,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8579252577319587,
281
+ "eval_loss": 0.40539947152137756,
282
+ "eval_runtime": 78.7689,
283
+ "eval_samples_per_second": 39.406,
284
+ "eval_steps_per_second": 9.852,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 5.300339786739574e-06,
290
+ "loss": 0.3812,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 5.300339786739574e-06,
296
+ "loss": 0.3804,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 5.300339786739574e-06,
302
+ "loss": 0.3783,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 5.300339786739574e-06,
308
+ "loss": 0.337,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 5.300339786739574e-06,
314
+ "loss": 0.3191,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 5.300339786739574e-06,
320
+ "loss": 0.3277,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 5.300339786739574e-06,
326
+ "loss": 0.3601,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 5.300339786739574e-06,
332
+ "loss": 0.3932,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 5.300339786739574e-06,
338
+ "loss": 0.3964,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 5.300339786739574e-06,
344
+ "loss": 0.3705,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8524484536082474,
350
+ "eval_loss": 0.40534982085227966,
351
+ "eval_runtime": 78.953,
352
+ "eval_samples_per_second": 39.315,
353
+ "eval_steps_per_second": 9.829,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 5.300339786739574e-06,
359
+ "loss": 0.4567,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 5.300339786739574e-06,
365
+ "loss": 0.384,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 5.300339786739574e-06,
371
+ "loss": 0.3205,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 5.300339786739574e-06,
377
+ "loss": 0.3841,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 5.300339786739574e-06,
383
+ "loss": 0.322,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 5.300339786739574e-06,
389
+ "loss": 0.3742,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 5.300339786739574e-06,
395
+ "loss": 0.3407,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 5.300339786739574e-06,
401
+ "loss": 0.3342,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 5.300339786739574e-06,
407
+ "loss": 0.3068,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 5.300339786739574e-06,
413
+ "loss": 0.3288,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8688788659793815,
419
+ "eval_loss": 0.36364319920539856,
420
+ "eval_runtime": 78.9463,
421
+ "eval_samples_per_second": 39.318,
422
+ "eval_steps_per_second": 9.829,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 5.300339786739574e-06,
428
+ "loss": 0.2918,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 5.300339786739574e-06,
434
+ "loss": 0.3571,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 5.300339786739574e-06,
440
+ "loss": 0.3341,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 5.300339786739574e-06,
446
+ "loss": 0.3162,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 5.300339786739574e-06,
452
+ "loss": 0.2384,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 5.300339786739574e-06,
458
+ "loss": 0.2934,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 5.300339786739574e-06,
464
+ "loss": 0.2686,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 5.300339786739574e-06,
470
+ "loss": 0.2806,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 5.300339786739574e-06,
476
+ "loss": 0.2897,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 5.300339786739574e-06,
482
+ "loss": 0.2417,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8743556701030928,
488
+ "eval_loss": 0.39001935720443726,
489
+ "eval_runtime": 78.9576,
490
+ "eval_samples_per_second": 39.312,
491
+ "eval_steps_per_second": 9.828,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 5.300339786739574e-06,
497
+ "loss": 0.2849,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 5.300339786739574e-06,
503
+ "loss": 0.2653,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 5.300339786739574e-06,
509
+ "loss": 0.2266,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 5.300339786739574e-06,
515
+ "loss": 0.2401,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 5.300339786739574e-06,
521
+ "loss": 0.2208,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 5.300339786739574e-06,
527
+ "loss": 0.2615,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 5.300339786739574e-06,
533
+ "loss": 0.2913,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 5.300339786739574e-06,
539
+ "loss": 0.2474,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 5.300339786739574e-06,
545
+ "loss": 0.2388,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 5.300339786739574e-06,
551
+ "loss": 0.2374,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8730670103092784,
557
+ "eval_loss": 0.39385583996772766,
558
+ "eval_runtime": 78.9575,
559
+ "eval_samples_per_second": 39.312,
560
+ "eval_steps_per_second": 9.828,
561
+ "step": 2000
562
+ },
563
+ {
564
+ "epoch": 2.61,
565
+ "learning_rate": 5.300339786739574e-06,
566
+ "loss": 0.2697,
567
+ "step": 2025
568
+ },
569
+ {
570
+ "epoch": 2.64,
571
+ "learning_rate": 5.300339786739574e-06,
572
+ "loss": 0.2592,
573
+ "step": 2050
574
+ },
575
+ {
576
+ "epoch": 2.67,
577
+ "learning_rate": 5.300339786739574e-06,
578
+ "loss": 0.2636,
579
+ "step": 2075
580
+ },
581
+ {
582
+ "epoch": 2.71,
583
+ "learning_rate": 5.300339786739574e-06,
584
+ "loss": 0.2349,
585
+ "step": 2100
586
+ },
587
+ {
588
+ "epoch": 2.74,
589
+ "learning_rate": 5.300339786739574e-06,
590
+ "loss": 0.2697,
591
+ "step": 2125
592
+ },
593
+ {
594
+ "epoch": 2.77,
595
+ "learning_rate": 5.300339786739574e-06,
596
+ "loss": 0.2507,
597
+ "step": 2150
598
+ },
599
+ {
600
+ "epoch": 2.8,
601
+ "learning_rate": 5.300339786739574e-06,
602
+ "loss": 0.2608,
603
+ "step": 2175
604
+ },
605
+ {
606
+ "epoch": 2.84,
607
+ "learning_rate": 5.300339786739574e-06,
608
+ "loss": 0.2463,
609
+ "step": 2200
610
+ },
611
+ {
612
+ "epoch": 2.87,
613
+ "learning_rate": 5.300339786739574e-06,
614
+ "loss": 0.2408,
615
+ "step": 2225
616
+ },
617
+ {
618
+ "epoch": 2.9,
619
+ "learning_rate": 5.300339786739574e-06,
620
+ "loss": 0.3525,
621
+ "step": 2250
622
+ },
623
+ {
624
+ "epoch": 2.9,
625
+ "eval_accuracy": 0.8627577319587629,
626
+ "eval_loss": 0.4113520085811615,
627
+ "eval_runtime": 78.9601,
628
+ "eval_samples_per_second": 39.311,
629
+ "eval_steps_per_second": 9.828,
630
+ "step": 2250
631
+ },
632
+ {
633
+ "epoch": 2.93,
634
+ "learning_rate": 5.300339786739574e-06,
635
+ "loss": 0.3033,
636
+ "step": 2275
637
+ },
638
+ {
639
+ "epoch": 2.96,
640
+ "learning_rate": 5.300339786739574e-06,
641
+ "loss": 0.2186,
642
+ "step": 2300
643
+ },
644
+ {
645
+ "epoch": 3.0,
646
+ "learning_rate": 5.300339786739574e-06,
647
+ "loss": 0.2619,
648
+ "step": 2325
649
+ },
650
+ {
651
+ "epoch": 3.03,
652
+ "learning_rate": 5.300339786739574e-06,
653
+ "loss": 0.1473,
654
+ "step": 2350
655
+ },
656
+ {
657
+ "epoch": 3.06,
658
+ "learning_rate": 5.300339786739574e-06,
659
+ "loss": 0.1662,
660
+ "step": 2375
661
+ },
662
+ {
663
+ "epoch": 3.09,
664
+ "learning_rate": 5.300339786739574e-06,
665
+ "loss": 0.1863,
666
+ "step": 2400
667
+ },
668
+ {
669
+ "epoch": 3.12,
670
+ "learning_rate": 5.300339786739574e-06,
671
+ "loss": 0.1869,
672
+ "step": 2425
673
+ },
674
+ {
675
+ "epoch": 3.16,
676
+ "learning_rate": 5.300339786739574e-06,
677
+ "loss": 0.1729,
678
+ "step": 2450
679
+ },
680
+ {
681
+ "epoch": 3.19,
682
+ "learning_rate": 5.300339786739574e-06,
683
+ "loss": 0.1998,
684
+ "step": 2475
685
+ },
686
+ {
687
+ "epoch": 3.22,
688
+ "learning_rate": 5.300339786739574e-06,
689
+ "loss": 0.1778,
690
+ "step": 2500
691
+ },
692
+ {
693
+ "epoch": 3.22,
694
+ "eval_accuracy": 0.873389175257732,
695
+ "eval_loss": 0.44405418634414673,
696
+ "eval_runtime": 78.9751,
697
+ "eval_samples_per_second": 39.304,
698
+ "eval_steps_per_second": 9.826,
699
+ "step": 2500
700
+ },
701
+ {
702
+ "epoch": 3.25,
703
+ "learning_rate": 5.300339786739574e-06,
704
+ "loss": 0.171,
705
+ "step": 2525
706
+ },
707
+ {
708
+ "epoch": 3.29,
709
+ "learning_rate": 5.300339786739574e-06,
710
+ "loss": 0.1642,
711
+ "step": 2550
712
+ },
713
+ {
714
+ "epoch": 3.32,
715
+ "learning_rate": 5.300339786739574e-06,
716
+ "loss": 0.2303,
717
+ "step": 2575
718
+ },
719
+ {
720
+ "epoch": 3.35,
721
+ "learning_rate": 5.300339786739574e-06,
722
+ "loss": 0.2194,
723
+ "step": 2600
724
+ },
725
+ {
726
+ "epoch": 3.38,
727
+ "learning_rate": 5.300339786739574e-06,
728
+ "loss": 0.1756,
729
+ "step": 2625
730
+ },
731
+ {
732
+ "epoch": 3.41,
733
+ "learning_rate": 5.300339786739574e-06,
734
+ "loss": 0.1956,
735
+ "step": 2650
736
+ },
737
+ {
738
+ "epoch": 3.45,
739
+ "learning_rate": 5.300339786739574e-06,
740
+ "loss": 0.2669,
741
+ "step": 2675
742
+ },
743
+ {
744
+ "epoch": 3.48,
745
+ "learning_rate": 5.300339786739574e-06,
746
+ "loss": 0.1514,
747
+ "step": 2700
748
+ },
749
+ {
750
+ "epoch": 3.51,
751
+ "learning_rate": 5.300339786739574e-06,
752
+ "loss": 0.1263,
753
+ "step": 2725
754
+ },
755
+ {
756
+ "epoch": 3.54,
757
+ "learning_rate": 5.300339786739574e-06,
758
+ "loss": 0.259,
759
+ "step": 2750
760
+ },
761
+ {
762
+ "epoch": 3.54,
763
+ "eval_accuracy": 0.8566365979381443,
764
+ "eval_loss": 0.4988997280597687,
765
+ "eval_runtime": 78.9372,
766
+ "eval_samples_per_second": 39.322,
767
+ "eval_steps_per_second": 9.831,
768
+ "step": 2750
769
+ },
770
+ {
771
+ "epoch": 3.58,
772
+ "learning_rate": 5.300339786739574e-06,
773
+ "loss": 0.1378,
774
+ "step": 2775
775
+ },
776
+ {
777
+ "epoch": 3.61,
778
+ "learning_rate": 5.300339786739574e-06,
779
+ "loss": 0.1588,
780
+ "step": 2800
781
+ },
782
+ {
783
+ "epoch": 3.64,
784
+ "learning_rate": 5.300339786739574e-06,
785
+ "loss": 0.179,
786
+ "step": 2825
787
+ },
788
+ {
789
+ "epoch": 3.67,
790
+ "learning_rate": 5.300339786739574e-06,
791
+ "loss": 0.3413,
792
+ "step": 2850
793
+ },
794
+ {
795
+ "epoch": 3.7,
796
+ "learning_rate": 5.300339786739574e-06,
797
+ "loss": 0.169,
798
+ "step": 2875
799
+ },
800
+ {
801
+ "epoch": 3.74,
802
+ "learning_rate": 5.300339786739574e-06,
803
+ "loss": 0.208,
804
+ "step": 2900
805
+ },
806
+ {
807
+ "epoch": 3.77,
808
+ "learning_rate": 5.300339786739574e-06,
809
+ "loss": 0.1602,
810
+ "step": 2925
811
+ },
812
+ {
813
+ "epoch": 3.8,
814
+ "learning_rate": 5.300339786739574e-06,
815
+ "loss": 0.1406,
816
+ "step": 2950
817
+ },
818
+ {
819
+ "epoch": 3.83,
820
+ "learning_rate": 5.300339786739574e-06,
821
+ "loss": 0.1519,
822
+ "step": 2975
823
+ },
824
+ {
825
+ "epoch": 3.87,
826
+ "learning_rate": 5.300339786739574e-06,
827
+ "loss": 0.2028,
828
+ "step": 3000
829
+ },
830
+ {
831
+ "epoch": 3.87,
832
+ "eval_accuracy": 0.8798324742268041,
833
+ "eval_loss": 0.46966859698295593,
834
+ "eval_runtime": 78.9816,
835
+ "eval_samples_per_second": 39.3,
836
+ "eval_steps_per_second": 9.825,
837
+ "step": 3000
838
+ }
839
+ ],
840
+ "logging_steps": 25,
841
+ "max_steps": 3000,
842
+ "num_input_tokens_seen": 0,
843
+ "num_train_epochs": 4,
844
+ "save_steps": 250,
845
+ "total_flos": 1.2625987184277504e+16,
846
+ "trial_name": null,
847
+ "trial_params": {
848
+ "learning_rate": 5.300339786739574e-06,
849
+ "per_device_eval_batch_size": 4,
850
+ "per_device_train_batch_size": 16
851
+ }
852
+ }
run-0/checkpoint-3000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff966c9d210a3dcc91765468d3bd580c3a94cec8f318bfe2ceb18c02bfa0afcf
3
+ size 4728
run-1/checkpoint-2000/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-1/checkpoint-2000/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d2979aaf550705ed15db92c754402c39bde2cf239b007351cd8d694f464ea9d
3
+ size 433279996
run-1/checkpoint-2000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:797fb321bed095050f952383f9272d50d76f008a2b10fdb902f0877056a930b7
3
+ size 866681082
run-1/checkpoint-2000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e36c9a58997a8fd5116115315a1541147a866ccc21b5fd8f202de7af5118db
3
+ size 14244
run-1/checkpoint-2000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd52d0eb8a495b5ec54fb0f307d28eec2eec58a5a5ce1ea1132a7d75494f126c
3
+ size 1064
run-1/checkpoint-2000/trainer_state.json ADDED
@@ -0,0 +1,576 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8833762886597938,
3
+ "best_model_checkpoint": "bert_trainer/run-1/checkpoint-2000",
4
+ "epoch": 2.5773195876288657,
5
+ "eval_steps": 250,
6
+ "global_step": 2000,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 8.555424210708822e-06,
14
+ "loss": 1.3732,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 1.7110848421417645e-05,
20
+ "loss": 1.0422,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 1.7110848421417645e-05,
26
+ "loss": 0.913,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 1.7110848421417645e-05,
32
+ "loss": 0.7626,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 1.7110848421417645e-05,
38
+ "loss": 0.7043,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 1.7110848421417645e-05,
44
+ "loss": 0.6356,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 1.7110848421417645e-05,
50
+ "loss": 0.6031,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 1.7110848421417645e-05,
56
+ "loss": 0.5454,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 1.7110848421417645e-05,
62
+ "loss": 0.4999,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 1.7110848421417645e-05,
68
+ "loss": 0.5111,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.8279639175257731,
74
+ "eval_loss": 0.4562760293483734,
75
+ "eval_runtime": 65.4867,
76
+ "eval_samples_per_second": 47.399,
77
+ "eval_steps_per_second": 11.85,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 1.7110848421417645e-05,
83
+ "loss": 0.511,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 1.7110848421417645e-05,
89
+ "loss": 0.4849,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 1.7110848421417645e-05,
95
+ "loss": 0.5027,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 1.7110848421417645e-05,
101
+ "loss": 0.565,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 1.7110848421417645e-05,
107
+ "loss": 0.4704,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 1.7110848421417645e-05,
113
+ "loss": 0.4931,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 1.7110848421417645e-05,
119
+ "loss": 0.4647,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 1.7110848421417645e-05,
125
+ "loss": 0.4491,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 1.7110848421417645e-05,
131
+ "loss": 0.4752,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 1.7110848421417645e-05,
137
+ "loss": 0.3899,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8630798969072165,
143
+ "eval_loss": 0.3841710388660431,
144
+ "eval_runtime": 65.5356,
145
+ "eval_samples_per_second": 47.364,
146
+ "eval_steps_per_second": 11.841,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 1.7110848421417645e-05,
152
+ "loss": 0.477,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 1.7110848421417645e-05,
158
+ "loss": 0.3964,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 1.7110848421417645e-05,
164
+ "loss": 0.3763,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 1.7110848421417645e-05,
170
+ "loss": 0.3928,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 1.7110848421417645e-05,
176
+ "loss": 0.4235,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 1.7110848421417645e-05,
182
+ "loss": 0.4024,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 1.7110848421417645e-05,
188
+ "loss": 0.4054,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 1.7110848421417645e-05,
194
+ "loss": 0.3731,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 1.7110848421417645e-05,
200
+ "loss": 0.3749,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 1.7110848421417645e-05,
206
+ "loss": 0.3315,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8666237113402062,
212
+ "eval_loss": 0.3664605915546417,
213
+ "eval_runtime": 65.4939,
214
+ "eval_samples_per_second": 47.394,
215
+ "eval_steps_per_second": 11.848,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 1.7110848421417645e-05,
221
+ "loss": 0.3572,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 1.7110848421417645e-05,
227
+ "loss": 0.2822,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 1.7110848421417645e-05,
233
+ "loss": 0.2383,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 1.7110848421417645e-05,
239
+ "loss": 0.2685,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 1.7110848421417645e-05,
245
+ "loss": 0.3076,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 1.7110848421417645e-05,
251
+ "loss": 0.2832,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 1.7110848421417645e-05,
257
+ "loss": 0.2976,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 1.7110848421417645e-05,
263
+ "loss": 0.2826,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 1.7110848421417645e-05,
269
+ "loss": 0.2585,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 1.7110848421417645e-05,
275
+ "loss": 0.2816,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8753221649484536,
281
+ "eval_loss": 0.3534678518772125,
282
+ "eval_runtime": 65.5301,
283
+ "eval_samples_per_second": 47.368,
284
+ "eval_steps_per_second": 11.842,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 1.7110848421417645e-05,
290
+ "loss": 0.2583,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 1.7110848421417645e-05,
296
+ "loss": 0.2405,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 1.7110848421417645e-05,
302
+ "loss": 0.2626,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 1.7110848421417645e-05,
308
+ "loss": 0.1829,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 1.7110848421417645e-05,
314
+ "loss": 0.217,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 1.7110848421417645e-05,
320
+ "loss": 0.2426,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 1.7110848421417645e-05,
326
+ "loss": 0.2937,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 1.7110848421417645e-05,
332
+ "loss": 0.283,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 1.7110848421417645e-05,
338
+ "loss": 0.2939,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 1.7110848421417645e-05,
344
+ "loss": 0.2477,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8663015463917526,
350
+ "eval_loss": 0.37554699182510376,
351
+ "eval_runtime": 65.5483,
352
+ "eval_samples_per_second": 47.354,
353
+ "eval_steps_per_second": 11.839,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 1.7110848421417645e-05,
359
+ "loss": 0.3323,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 1.7110848421417645e-05,
365
+ "loss": 0.2489,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 1.7110848421417645e-05,
371
+ "loss": 0.2451,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 1.7110848421417645e-05,
377
+ "loss": 0.2911,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 1.7110848421417645e-05,
383
+ "loss": 0.2348,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 1.7110848421417645e-05,
389
+ "loss": 0.2565,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 1.7110848421417645e-05,
395
+ "loss": 0.1898,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 1.7110848421417645e-05,
401
+ "loss": 0.2489,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 1.7110848421417645e-05,
407
+ "loss": 0.2442,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 1.7110848421417645e-05,
413
+ "loss": 0.2745,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8717783505154639,
419
+ "eval_loss": 0.3890085220336914,
420
+ "eval_runtime": 65.5564,
421
+ "eval_samples_per_second": 47.349,
422
+ "eval_steps_per_second": 11.837,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 1.7110848421417645e-05,
428
+ "loss": 0.208,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 1.7110848421417645e-05,
434
+ "loss": 0.2804,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 1.7110848421417645e-05,
440
+ "loss": 0.186,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 1.7110848421417645e-05,
446
+ "loss": 0.206,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 1.7110848421417645e-05,
452
+ "loss": 0.1658,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 1.7110848421417645e-05,
458
+ "loss": 0.1576,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 1.7110848421417645e-05,
464
+ "loss": 0.1845,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 1.7110848421417645e-05,
470
+ "loss": 0.1601,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 1.7110848421417645e-05,
476
+ "loss": 0.1397,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 1.7110848421417645e-05,
482
+ "loss": 0.109,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8814432989690721,
488
+ "eval_loss": 0.5400919318199158,
489
+ "eval_runtime": 65.5029,
490
+ "eval_samples_per_second": 47.387,
491
+ "eval_steps_per_second": 11.847,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 1.7110848421417645e-05,
497
+ "loss": 0.1069,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 1.7110848421417645e-05,
503
+ "loss": 0.1495,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 1.7110848421417645e-05,
509
+ "loss": 0.1209,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 1.7110848421417645e-05,
515
+ "loss": 0.134,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 1.7110848421417645e-05,
521
+ "loss": 0.1357,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 1.7110848421417645e-05,
527
+ "loss": 0.1114,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 1.7110848421417645e-05,
533
+ "loss": 0.1436,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 1.7110848421417645e-05,
539
+ "loss": 0.1775,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 1.7110848421417645e-05,
545
+ "loss": 0.1232,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 1.7110848421417645e-05,
551
+ "loss": 0.115,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8833762886597938,
557
+ "eval_loss": 0.5348294973373413,
558
+ "eval_runtime": 67.3607,
559
+ "eval_samples_per_second": 46.08,
560
+ "eval_steps_per_second": 11.52,
561
+ "step": 2000
562
+ }
563
+ ],
564
+ "logging_steps": 25,
565
+ "max_steps": 3000,
566
+ "num_input_tokens_seen": 0,
567
+ "num_train_epochs": 4,
568
+ "save_steps": 250,
569
+ "total_flos": 8414518195998720.0,
570
+ "trial_name": null,
571
+ "trial_params": {
572
+ "learning_rate": 1.7110848421417645e-05,
573
+ "per_device_eval_batch_size": 4,
574
+ "per_device_train_batch_size": 16
575
+ }
576
+ }
run-1/checkpoint-2000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ff7ff5e497d2fd17b0b0a335d1e1e52fdf9ab64c61fc776fe6063c480e0a9f6
3
+ size 4728
run-1/checkpoint-2750/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-1/checkpoint-2750/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fdf1daffe1979f97111b495ac9166af3936db88484feea619daefa7ee4afe6e
3
+ size 433279996
run-1/checkpoint-2750/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f352afcb5855371e75fc5f7c2a50086a26116ec8440ae8fb5ea58867ecb0b84f
3
+ size 866681082
run-1/checkpoint-2750/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2f146112a2b6651d5740f314c7a7760a6b10b307030d3ca8f929af57ce20006
3
+ size 14244
run-1/checkpoint-2750/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35c90518991fb03cf3fc2f6874522e66785a9b1bb9f1028bbfcb60f66f6de8fd
3
+ size 1064
run-1/checkpoint-2750/trainer_state.json ADDED
@@ -0,0 +1,783 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8833762886597938,
3
+ "best_model_checkpoint": "bert_trainer/run-1/checkpoint-2000",
4
+ "epoch": 3.5438144329896906,
5
+ "eval_steps": 250,
6
+ "global_step": 2750,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 8.555424210708822e-06,
14
+ "loss": 1.3732,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 1.7110848421417645e-05,
20
+ "loss": 1.0422,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 1.7110848421417645e-05,
26
+ "loss": 0.913,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 1.7110848421417645e-05,
32
+ "loss": 0.7626,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 1.7110848421417645e-05,
38
+ "loss": 0.7043,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 1.7110848421417645e-05,
44
+ "loss": 0.6356,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 1.7110848421417645e-05,
50
+ "loss": 0.6031,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 1.7110848421417645e-05,
56
+ "loss": 0.5454,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 1.7110848421417645e-05,
62
+ "loss": 0.4999,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 1.7110848421417645e-05,
68
+ "loss": 0.5111,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.8279639175257731,
74
+ "eval_loss": 0.4562760293483734,
75
+ "eval_runtime": 65.4867,
76
+ "eval_samples_per_second": 47.399,
77
+ "eval_steps_per_second": 11.85,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 1.7110848421417645e-05,
83
+ "loss": 0.511,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 1.7110848421417645e-05,
89
+ "loss": 0.4849,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 1.7110848421417645e-05,
95
+ "loss": 0.5027,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 1.7110848421417645e-05,
101
+ "loss": 0.565,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 1.7110848421417645e-05,
107
+ "loss": 0.4704,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 1.7110848421417645e-05,
113
+ "loss": 0.4931,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 1.7110848421417645e-05,
119
+ "loss": 0.4647,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 1.7110848421417645e-05,
125
+ "loss": 0.4491,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 1.7110848421417645e-05,
131
+ "loss": 0.4752,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 1.7110848421417645e-05,
137
+ "loss": 0.3899,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8630798969072165,
143
+ "eval_loss": 0.3841710388660431,
144
+ "eval_runtime": 65.5356,
145
+ "eval_samples_per_second": 47.364,
146
+ "eval_steps_per_second": 11.841,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 1.7110848421417645e-05,
152
+ "loss": 0.477,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 1.7110848421417645e-05,
158
+ "loss": 0.3964,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 1.7110848421417645e-05,
164
+ "loss": 0.3763,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 1.7110848421417645e-05,
170
+ "loss": 0.3928,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 1.7110848421417645e-05,
176
+ "loss": 0.4235,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 1.7110848421417645e-05,
182
+ "loss": 0.4024,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 1.7110848421417645e-05,
188
+ "loss": 0.4054,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 1.7110848421417645e-05,
194
+ "loss": 0.3731,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 1.7110848421417645e-05,
200
+ "loss": 0.3749,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 1.7110848421417645e-05,
206
+ "loss": 0.3315,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8666237113402062,
212
+ "eval_loss": 0.3664605915546417,
213
+ "eval_runtime": 65.4939,
214
+ "eval_samples_per_second": 47.394,
215
+ "eval_steps_per_second": 11.848,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 1.7110848421417645e-05,
221
+ "loss": 0.3572,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 1.7110848421417645e-05,
227
+ "loss": 0.2822,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 1.7110848421417645e-05,
233
+ "loss": 0.2383,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 1.7110848421417645e-05,
239
+ "loss": 0.2685,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 1.7110848421417645e-05,
245
+ "loss": 0.3076,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 1.7110848421417645e-05,
251
+ "loss": 0.2832,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 1.7110848421417645e-05,
257
+ "loss": 0.2976,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 1.7110848421417645e-05,
263
+ "loss": 0.2826,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 1.7110848421417645e-05,
269
+ "loss": 0.2585,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 1.7110848421417645e-05,
275
+ "loss": 0.2816,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8753221649484536,
281
+ "eval_loss": 0.3534678518772125,
282
+ "eval_runtime": 65.5301,
283
+ "eval_samples_per_second": 47.368,
284
+ "eval_steps_per_second": 11.842,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 1.7110848421417645e-05,
290
+ "loss": 0.2583,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 1.7110848421417645e-05,
296
+ "loss": 0.2405,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 1.7110848421417645e-05,
302
+ "loss": 0.2626,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 1.7110848421417645e-05,
308
+ "loss": 0.1829,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 1.7110848421417645e-05,
314
+ "loss": 0.217,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 1.7110848421417645e-05,
320
+ "loss": 0.2426,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 1.7110848421417645e-05,
326
+ "loss": 0.2937,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 1.7110848421417645e-05,
332
+ "loss": 0.283,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 1.7110848421417645e-05,
338
+ "loss": 0.2939,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 1.7110848421417645e-05,
344
+ "loss": 0.2477,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8663015463917526,
350
+ "eval_loss": 0.37554699182510376,
351
+ "eval_runtime": 65.5483,
352
+ "eval_samples_per_second": 47.354,
353
+ "eval_steps_per_second": 11.839,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 1.7110848421417645e-05,
359
+ "loss": 0.3323,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 1.7110848421417645e-05,
365
+ "loss": 0.2489,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 1.7110848421417645e-05,
371
+ "loss": 0.2451,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 1.7110848421417645e-05,
377
+ "loss": 0.2911,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 1.7110848421417645e-05,
383
+ "loss": 0.2348,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 1.7110848421417645e-05,
389
+ "loss": 0.2565,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 1.7110848421417645e-05,
395
+ "loss": 0.1898,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 1.7110848421417645e-05,
401
+ "loss": 0.2489,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 1.7110848421417645e-05,
407
+ "loss": 0.2442,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 1.7110848421417645e-05,
413
+ "loss": 0.2745,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8717783505154639,
419
+ "eval_loss": 0.3890085220336914,
420
+ "eval_runtime": 65.5564,
421
+ "eval_samples_per_second": 47.349,
422
+ "eval_steps_per_second": 11.837,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 1.7110848421417645e-05,
428
+ "loss": 0.208,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 1.7110848421417645e-05,
434
+ "loss": 0.2804,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 1.7110848421417645e-05,
440
+ "loss": 0.186,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 1.7110848421417645e-05,
446
+ "loss": 0.206,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 1.7110848421417645e-05,
452
+ "loss": 0.1658,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 1.7110848421417645e-05,
458
+ "loss": 0.1576,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 1.7110848421417645e-05,
464
+ "loss": 0.1845,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 1.7110848421417645e-05,
470
+ "loss": 0.1601,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 1.7110848421417645e-05,
476
+ "loss": 0.1397,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 1.7110848421417645e-05,
482
+ "loss": 0.109,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8814432989690721,
488
+ "eval_loss": 0.5400919318199158,
489
+ "eval_runtime": 65.5029,
490
+ "eval_samples_per_second": 47.387,
491
+ "eval_steps_per_second": 11.847,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 1.7110848421417645e-05,
497
+ "loss": 0.1069,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 1.7110848421417645e-05,
503
+ "loss": 0.1495,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 1.7110848421417645e-05,
509
+ "loss": 0.1209,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 1.7110848421417645e-05,
515
+ "loss": 0.134,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 1.7110848421417645e-05,
521
+ "loss": 0.1357,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 1.7110848421417645e-05,
527
+ "loss": 0.1114,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 1.7110848421417645e-05,
533
+ "loss": 0.1436,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 1.7110848421417645e-05,
539
+ "loss": 0.1775,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 1.7110848421417645e-05,
545
+ "loss": 0.1232,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 1.7110848421417645e-05,
551
+ "loss": 0.115,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8833762886597938,
557
+ "eval_loss": 0.5348294973373413,
558
+ "eval_runtime": 67.3607,
559
+ "eval_samples_per_second": 46.08,
560
+ "eval_steps_per_second": 11.52,
561
+ "step": 2000
562
+ },
563
+ {
564
+ "epoch": 2.61,
565
+ "learning_rate": 1.7110848421417645e-05,
566
+ "loss": 0.1429,
567
+ "step": 2025
568
+ },
569
+ {
570
+ "epoch": 2.64,
571
+ "learning_rate": 1.7110848421417645e-05,
572
+ "loss": 0.1612,
573
+ "step": 2050
574
+ },
575
+ {
576
+ "epoch": 2.67,
577
+ "learning_rate": 1.7110848421417645e-05,
578
+ "loss": 0.1565,
579
+ "step": 2075
580
+ },
581
+ {
582
+ "epoch": 2.71,
583
+ "learning_rate": 1.7110848421417645e-05,
584
+ "loss": 0.1188,
585
+ "step": 2100
586
+ },
587
+ {
588
+ "epoch": 2.74,
589
+ "learning_rate": 1.7110848421417645e-05,
590
+ "loss": 0.1576,
591
+ "step": 2125
592
+ },
593
+ {
594
+ "epoch": 2.77,
595
+ "learning_rate": 1.7110848421417645e-05,
596
+ "loss": 0.1832,
597
+ "step": 2150
598
+ },
599
+ {
600
+ "epoch": 2.8,
601
+ "learning_rate": 1.7110848421417645e-05,
602
+ "loss": 0.1643,
603
+ "step": 2175
604
+ },
605
+ {
606
+ "epoch": 2.84,
607
+ "learning_rate": 1.7110848421417645e-05,
608
+ "loss": 0.1668,
609
+ "step": 2200
610
+ },
611
+ {
612
+ "epoch": 2.87,
613
+ "learning_rate": 1.7110848421417645e-05,
614
+ "loss": 0.1984,
615
+ "step": 2225
616
+ },
617
+ {
618
+ "epoch": 2.9,
619
+ "learning_rate": 1.7110848421417645e-05,
620
+ "loss": 0.1839,
621
+ "step": 2250
622
+ },
623
+ {
624
+ "epoch": 2.9,
625
+ "eval_accuracy": 0.8424613402061856,
626
+ "eval_loss": 0.6889757513999939,
627
+ "eval_runtime": 65.603,
628
+ "eval_samples_per_second": 47.315,
629
+ "eval_steps_per_second": 11.829,
630
+ "step": 2250
631
+ },
632
+ {
633
+ "epoch": 2.93,
634
+ "learning_rate": 1.7110848421417645e-05,
635
+ "loss": 0.2497,
636
+ "step": 2275
637
+ },
638
+ {
639
+ "epoch": 2.96,
640
+ "learning_rate": 1.7110848421417645e-05,
641
+ "loss": 0.1096,
642
+ "step": 2300
643
+ },
644
+ {
645
+ "epoch": 3.0,
646
+ "learning_rate": 1.7110848421417645e-05,
647
+ "loss": 0.157,
648
+ "step": 2325
649
+ },
650
+ {
651
+ "epoch": 3.03,
652
+ "learning_rate": 1.7110848421417645e-05,
653
+ "loss": 0.0431,
654
+ "step": 2350
655
+ },
656
+ {
657
+ "epoch": 3.06,
658
+ "learning_rate": 1.7110848421417645e-05,
659
+ "loss": 0.0428,
660
+ "step": 2375
661
+ },
662
+ {
663
+ "epoch": 3.09,
664
+ "learning_rate": 1.7110848421417645e-05,
665
+ "loss": 0.1085,
666
+ "step": 2400
667
+ },
668
+ {
669
+ "epoch": 3.12,
670
+ "learning_rate": 1.7110848421417645e-05,
671
+ "loss": 0.0781,
672
+ "step": 2425
673
+ },
674
+ {
675
+ "epoch": 3.16,
676
+ "learning_rate": 1.7110848421417645e-05,
677
+ "loss": 0.1603,
678
+ "step": 2450
679
+ },
680
+ {
681
+ "epoch": 3.19,
682
+ "learning_rate": 1.7110848421417645e-05,
683
+ "loss": 0.0825,
684
+ "step": 2475
685
+ },
686
+ {
687
+ "epoch": 3.22,
688
+ "learning_rate": 1.7110848421417645e-05,
689
+ "loss": 0.0855,
690
+ "step": 2500
691
+ },
692
+ {
693
+ "epoch": 3.22,
694
+ "eval_accuracy": 0.8759664948453608,
695
+ "eval_loss": 0.6595790982246399,
696
+ "eval_runtime": 65.5897,
697
+ "eval_samples_per_second": 47.324,
698
+ "eval_steps_per_second": 11.831,
699
+ "step": 2500
700
+ },
701
+ {
702
+ "epoch": 3.25,
703
+ "learning_rate": 1.7110848421417645e-05,
704
+ "loss": 0.0614,
705
+ "step": 2525
706
+ },
707
+ {
708
+ "epoch": 3.29,
709
+ "learning_rate": 1.7110848421417645e-05,
710
+ "loss": 0.0686,
711
+ "step": 2550
712
+ },
713
+ {
714
+ "epoch": 3.32,
715
+ "learning_rate": 1.7110848421417645e-05,
716
+ "loss": 0.0716,
717
+ "step": 2575
718
+ },
719
+ {
720
+ "epoch": 3.35,
721
+ "learning_rate": 1.7110848421417645e-05,
722
+ "loss": 0.1359,
723
+ "step": 2600
724
+ },
725
+ {
726
+ "epoch": 3.38,
727
+ "learning_rate": 1.7110848421417645e-05,
728
+ "loss": 0.0863,
729
+ "step": 2625
730
+ },
731
+ {
732
+ "epoch": 3.41,
733
+ "learning_rate": 1.7110848421417645e-05,
734
+ "loss": 0.0915,
735
+ "step": 2650
736
+ },
737
+ {
738
+ "epoch": 3.45,
739
+ "learning_rate": 1.7110848421417645e-05,
740
+ "loss": 0.0595,
741
+ "step": 2675
742
+ },
743
+ {
744
+ "epoch": 3.48,
745
+ "learning_rate": 1.7110848421417645e-05,
746
+ "loss": 0.0714,
747
+ "step": 2700
748
+ },
749
+ {
750
+ "epoch": 3.51,
751
+ "learning_rate": 1.7110848421417645e-05,
752
+ "loss": 0.0914,
753
+ "step": 2725
754
+ },
755
+ {
756
+ "epoch": 3.54,
757
+ "learning_rate": 1.7110848421417645e-05,
758
+ "loss": 0.0774,
759
+ "step": 2750
760
+ },
761
+ {
762
+ "epoch": 3.54,
763
+ "eval_accuracy": 0.8740335051546392,
764
+ "eval_loss": 0.7505560517311096,
765
+ "eval_runtime": 65.5758,
766
+ "eval_samples_per_second": 47.335,
767
+ "eval_steps_per_second": 11.834,
768
+ "step": 2750
769
+ }
770
+ ],
771
+ "logging_steps": 25,
772
+ "max_steps": 3000,
773
+ "num_input_tokens_seen": 0,
774
+ "num_train_epochs": 4,
775
+ "save_steps": 250,
776
+ "total_flos": 1.156930472414208e+16,
777
+ "trial_name": null,
778
+ "trial_params": {
779
+ "learning_rate": 1.7110848421417645e-05,
780
+ "per_device_eval_batch_size": 4,
781
+ "per_device_train_batch_size": 16
782
+ }
783
+ }
run-1/checkpoint-2750/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ff7ff5e497d2fd17b0b0a335d1e1e52fdf9ab64c61fc776fe6063c480e0a9f6
3
+ size 4728
run-1/checkpoint-3000/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-1/checkpoint-3000/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de060276be3bcfe668ca25c8af00e40cad29ee8aaa2f3b02863a1935940a71e1
3
+ size 433279996
run-1/checkpoint-3000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10522d5e8817f1178a99dd7c1589a4fabda59fd8a020b8326b29d1d247dd0cdf
3
+ size 866681082
run-1/checkpoint-3000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5475dcc749e5bb84d23e503506ffb30e2c214153e88098ffa2eba5466de55a64
3
+ size 14244
run-1/checkpoint-3000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80bef91821d2e324748694916761e06c6958f568b9d748689db5bb337914680c
3
+ size 1064
run-1/checkpoint-3000/trainer_state.json ADDED
@@ -0,0 +1,852 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8833762886597938,
3
+ "best_model_checkpoint": "bert_trainer/run-1/checkpoint-2000",
4
+ "epoch": 3.865979381443299,
5
+ "eval_steps": 250,
6
+ "global_step": 3000,
7
+ "is_hyper_param_search": true,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.03,
13
+ "learning_rate": 8.555424210708822e-06,
14
+ "loss": 1.3732,
15
+ "step": 25
16
+ },
17
+ {
18
+ "epoch": 0.06,
19
+ "learning_rate": 1.7110848421417645e-05,
20
+ "loss": 1.0422,
21
+ "step": 50
22
+ },
23
+ {
24
+ "epoch": 0.1,
25
+ "learning_rate": 1.7110848421417645e-05,
26
+ "loss": 0.913,
27
+ "step": 75
28
+ },
29
+ {
30
+ "epoch": 0.13,
31
+ "learning_rate": 1.7110848421417645e-05,
32
+ "loss": 0.7626,
33
+ "step": 100
34
+ },
35
+ {
36
+ "epoch": 0.16,
37
+ "learning_rate": 1.7110848421417645e-05,
38
+ "loss": 0.7043,
39
+ "step": 125
40
+ },
41
+ {
42
+ "epoch": 0.19,
43
+ "learning_rate": 1.7110848421417645e-05,
44
+ "loss": 0.6356,
45
+ "step": 150
46
+ },
47
+ {
48
+ "epoch": 0.23,
49
+ "learning_rate": 1.7110848421417645e-05,
50
+ "loss": 0.6031,
51
+ "step": 175
52
+ },
53
+ {
54
+ "epoch": 0.26,
55
+ "learning_rate": 1.7110848421417645e-05,
56
+ "loss": 0.5454,
57
+ "step": 200
58
+ },
59
+ {
60
+ "epoch": 0.29,
61
+ "learning_rate": 1.7110848421417645e-05,
62
+ "loss": 0.4999,
63
+ "step": 225
64
+ },
65
+ {
66
+ "epoch": 0.32,
67
+ "learning_rate": 1.7110848421417645e-05,
68
+ "loss": 0.5111,
69
+ "step": 250
70
+ },
71
+ {
72
+ "epoch": 0.32,
73
+ "eval_accuracy": 0.8279639175257731,
74
+ "eval_loss": 0.4562760293483734,
75
+ "eval_runtime": 65.4867,
76
+ "eval_samples_per_second": 47.399,
77
+ "eval_steps_per_second": 11.85,
78
+ "step": 250
79
+ },
80
+ {
81
+ "epoch": 0.35,
82
+ "learning_rate": 1.7110848421417645e-05,
83
+ "loss": 0.511,
84
+ "step": 275
85
+ },
86
+ {
87
+ "epoch": 0.39,
88
+ "learning_rate": 1.7110848421417645e-05,
89
+ "loss": 0.4849,
90
+ "step": 300
91
+ },
92
+ {
93
+ "epoch": 0.42,
94
+ "learning_rate": 1.7110848421417645e-05,
95
+ "loss": 0.5027,
96
+ "step": 325
97
+ },
98
+ {
99
+ "epoch": 0.45,
100
+ "learning_rate": 1.7110848421417645e-05,
101
+ "loss": 0.565,
102
+ "step": 350
103
+ },
104
+ {
105
+ "epoch": 0.48,
106
+ "learning_rate": 1.7110848421417645e-05,
107
+ "loss": 0.4704,
108
+ "step": 375
109
+ },
110
+ {
111
+ "epoch": 0.52,
112
+ "learning_rate": 1.7110848421417645e-05,
113
+ "loss": 0.4931,
114
+ "step": 400
115
+ },
116
+ {
117
+ "epoch": 0.55,
118
+ "learning_rate": 1.7110848421417645e-05,
119
+ "loss": 0.4647,
120
+ "step": 425
121
+ },
122
+ {
123
+ "epoch": 0.58,
124
+ "learning_rate": 1.7110848421417645e-05,
125
+ "loss": 0.4491,
126
+ "step": 450
127
+ },
128
+ {
129
+ "epoch": 0.61,
130
+ "learning_rate": 1.7110848421417645e-05,
131
+ "loss": 0.4752,
132
+ "step": 475
133
+ },
134
+ {
135
+ "epoch": 0.64,
136
+ "learning_rate": 1.7110848421417645e-05,
137
+ "loss": 0.3899,
138
+ "step": 500
139
+ },
140
+ {
141
+ "epoch": 0.64,
142
+ "eval_accuracy": 0.8630798969072165,
143
+ "eval_loss": 0.3841710388660431,
144
+ "eval_runtime": 65.5356,
145
+ "eval_samples_per_second": 47.364,
146
+ "eval_steps_per_second": 11.841,
147
+ "step": 500
148
+ },
149
+ {
150
+ "epoch": 0.68,
151
+ "learning_rate": 1.7110848421417645e-05,
152
+ "loss": 0.477,
153
+ "step": 525
154
+ },
155
+ {
156
+ "epoch": 0.71,
157
+ "learning_rate": 1.7110848421417645e-05,
158
+ "loss": 0.3964,
159
+ "step": 550
160
+ },
161
+ {
162
+ "epoch": 0.74,
163
+ "learning_rate": 1.7110848421417645e-05,
164
+ "loss": 0.3763,
165
+ "step": 575
166
+ },
167
+ {
168
+ "epoch": 0.77,
169
+ "learning_rate": 1.7110848421417645e-05,
170
+ "loss": 0.3928,
171
+ "step": 600
172
+ },
173
+ {
174
+ "epoch": 0.81,
175
+ "learning_rate": 1.7110848421417645e-05,
176
+ "loss": 0.4235,
177
+ "step": 625
178
+ },
179
+ {
180
+ "epoch": 0.84,
181
+ "learning_rate": 1.7110848421417645e-05,
182
+ "loss": 0.4024,
183
+ "step": 650
184
+ },
185
+ {
186
+ "epoch": 0.87,
187
+ "learning_rate": 1.7110848421417645e-05,
188
+ "loss": 0.4054,
189
+ "step": 675
190
+ },
191
+ {
192
+ "epoch": 0.9,
193
+ "learning_rate": 1.7110848421417645e-05,
194
+ "loss": 0.3731,
195
+ "step": 700
196
+ },
197
+ {
198
+ "epoch": 0.93,
199
+ "learning_rate": 1.7110848421417645e-05,
200
+ "loss": 0.3749,
201
+ "step": 725
202
+ },
203
+ {
204
+ "epoch": 0.97,
205
+ "learning_rate": 1.7110848421417645e-05,
206
+ "loss": 0.3315,
207
+ "step": 750
208
+ },
209
+ {
210
+ "epoch": 0.97,
211
+ "eval_accuracy": 0.8666237113402062,
212
+ "eval_loss": 0.3664605915546417,
213
+ "eval_runtime": 65.4939,
214
+ "eval_samples_per_second": 47.394,
215
+ "eval_steps_per_second": 11.848,
216
+ "step": 750
217
+ },
218
+ {
219
+ "epoch": 1.0,
220
+ "learning_rate": 1.7110848421417645e-05,
221
+ "loss": 0.3572,
222
+ "step": 775
223
+ },
224
+ {
225
+ "epoch": 1.03,
226
+ "learning_rate": 1.7110848421417645e-05,
227
+ "loss": 0.2822,
228
+ "step": 800
229
+ },
230
+ {
231
+ "epoch": 1.06,
232
+ "learning_rate": 1.7110848421417645e-05,
233
+ "loss": 0.2383,
234
+ "step": 825
235
+ },
236
+ {
237
+ "epoch": 1.1,
238
+ "learning_rate": 1.7110848421417645e-05,
239
+ "loss": 0.2685,
240
+ "step": 850
241
+ },
242
+ {
243
+ "epoch": 1.13,
244
+ "learning_rate": 1.7110848421417645e-05,
245
+ "loss": 0.3076,
246
+ "step": 875
247
+ },
248
+ {
249
+ "epoch": 1.16,
250
+ "learning_rate": 1.7110848421417645e-05,
251
+ "loss": 0.2832,
252
+ "step": 900
253
+ },
254
+ {
255
+ "epoch": 1.19,
256
+ "learning_rate": 1.7110848421417645e-05,
257
+ "loss": 0.2976,
258
+ "step": 925
259
+ },
260
+ {
261
+ "epoch": 1.22,
262
+ "learning_rate": 1.7110848421417645e-05,
263
+ "loss": 0.2826,
264
+ "step": 950
265
+ },
266
+ {
267
+ "epoch": 1.26,
268
+ "learning_rate": 1.7110848421417645e-05,
269
+ "loss": 0.2585,
270
+ "step": 975
271
+ },
272
+ {
273
+ "epoch": 1.29,
274
+ "learning_rate": 1.7110848421417645e-05,
275
+ "loss": 0.2816,
276
+ "step": 1000
277
+ },
278
+ {
279
+ "epoch": 1.29,
280
+ "eval_accuracy": 0.8753221649484536,
281
+ "eval_loss": 0.3534678518772125,
282
+ "eval_runtime": 65.5301,
283
+ "eval_samples_per_second": 47.368,
284
+ "eval_steps_per_second": 11.842,
285
+ "step": 1000
286
+ },
287
+ {
288
+ "epoch": 1.32,
289
+ "learning_rate": 1.7110848421417645e-05,
290
+ "loss": 0.2583,
291
+ "step": 1025
292
+ },
293
+ {
294
+ "epoch": 1.35,
295
+ "learning_rate": 1.7110848421417645e-05,
296
+ "loss": 0.2405,
297
+ "step": 1050
298
+ },
299
+ {
300
+ "epoch": 1.39,
301
+ "learning_rate": 1.7110848421417645e-05,
302
+ "loss": 0.2626,
303
+ "step": 1075
304
+ },
305
+ {
306
+ "epoch": 1.42,
307
+ "learning_rate": 1.7110848421417645e-05,
308
+ "loss": 0.1829,
309
+ "step": 1100
310
+ },
311
+ {
312
+ "epoch": 1.45,
313
+ "learning_rate": 1.7110848421417645e-05,
314
+ "loss": 0.217,
315
+ "step": 1125
316
+ },
317
+ {
318
+ "epoch": 1.48,
319
+ "learning_rate": 1.7110848421417645e-05,
320
+ "loss": 0.2426,
321
+ "step": 1150
322
+ },
323
+ {
324
+ "epoch": 1.51,
325
+ "learning_rate": 1.7110848421417645e-05,
326
+ "loss": 0.2937,
327
+ "step": 1175
328
+ },
329
+ {
330
+ "epoch": 1.55,
331
+ "learning_rate": 1.7110848421417645e-05,
332
+ "loss": 0.283,
333
+ "step": 1200
334
+ },
335
+ {
336
+ "epoch": 1.58,
337
+ "learning_rate": 1.7110848421417645e-05,
338
+ "loss": 0.2939,
339
+ "step": 1225
340
+ },
341
+ {
342
+ "epoch": 1.61,
343
+ "learning_rate": 1.7110848421417645e-05,
344
+ "loss": 0.2477,
345
+ "step": 1250
346
+ },
347
+ {
348
+ "epoch": 1.61,
349
+ "eval_accuracy": 0.8663015463917526,
350
+ "eval_loss": 0.37554699182510376,
351
+ "eval_runtime": 65.5483,
352
+ "eval_samples_per_second": 47.354,
353
+ "eval_steps_per_second": 11.839,
354
+ "step": 1250
355
+ },
356
+ {
357
+ "epoch": 1.64,
358
+ "learning_rate": 1.7110848421417645e-05,
359
+ "loss": 0.3323,
360
+ "step": 1275
361
+ },
362
+ {
363
+ "epoch": 1.68,
364
+ "learning_rate": 1.7110848421417645e-05,
365
+ "loss": 0.2489,
366
+ "step": 1300
367
+ },
368
+ {
369
+ "epoch": 1.71,
370
+ "learning_rate": 1.7110848421417645e-05,
371
+ "loss": 0.2451,
372
+ "step": 1325
373
+ },
374
+ {
375
+ "epoch": 1.74,
376
+ "learning_rate": 1.7110848421417645e-05,
377
+ "loss": 0.2911,
378
+ "step": 1350
379
+ },
380
+ {
381
+ "epoch": 1.77,
382
+ "learning_rate": 1.7110848421417645e-05,
383
+ "loss": 0.2348,
384
+ "step": 1375
385
+ },
386
+ {
387
+ "epoch": 1.8,
388
+ "learning_rate": 1.7110848421417645e-05,
389
+ "loss": 0.2565,
390
+ "step": 1400
391
+ },
392
+ {
393
+ "epoch": 1.84,
394
+ "learning_rate": 1.7110848421417645e-05,
395
+ "loss": 0.1898,
396
+ "step": 1425
397
+ },
398
+ {
399
+ "epoch": 1.87,
400
+ "learning_rate": 1.7110848421417645e-05,
401
+ "loss": 0.2489,
402
+ "step": 1450
403
+ },
404
+ {
405
+ "epoch": 1.9,
406
+ "learning_rate": 1.7110848421417645e-05,
407
+ "loss": 0.2442,
408
+ "step": 1475
409
+ },
410
+ {
411
+ "epoch": 1.93,
412
+ "learning_rate": 1.7110848421417645e-05,
413
+ "loss": 0.2745,
414
+ "step": 1500
415
+ },
416
+ {
417
+ "epoch": 1.93,
418
+ "eval_accuracy": 0.8717783505154639,
419
+ "eval_loss": 0.3890085220336914,
420
+ "eval_runtime": 65.5564,
421
+ "eval_samples_per_second": 47.349,
422
+ "eval_steps_per_second": 11.837,
423
+ "step": 1500
424
+ },
425
+ {
426
+ "epoch": 1.97,
427
+ "learning_rate": 1.7110848421417645e-05,
428
+ "loss": 0.208,
429
+ "step": 1525
430
+ },
431
+ {
432
+ "epoch": 2.0,
433
+ "learning_rate": 1.7110848421417645e-05,
434
+ "loss": 0.2804,
435
+ "step": 1550
436
+ },
437
+ {
438
+ "epoch": 2.03,
439
+ "learning_rate": 1.7110848421417645e-05,
440
+ "loss": 0.186,
441
+ "step": 1575
442
+ },
443
+ {
444
+ "epoch": 2.06,
445
+ "learning_rate": 1.7110848421417645e-05,
446
+ "loss": 0.206,
447
+ "step": 1600
448
+ },
449
+ {
450
+ "epoch": 2.09,
451
+ "learning_rate": 1.7110848421417645e-05,
452
+ "loss": 0.1658,
453
+ "step": 1625
454
+ },
455
+ {
456
+ "epoch": 2.13,
457
+ "learning_rate": 1.7110848421417645e-05,
458
+ "loss": 0.1576,
459
+ "step": 1650
460
+ },
461
+ {
462
+ "epoch": 2.16,
463
+ "learning_rate": 1.7110848421417645e-05,
464
+ "loss": 0.1845,
465
+ "step": 1675
466
+ },
467
+ {
468
+ "epoch": 2.19,
469
+ "learning_rate": 1.7110848421417645e-05,
470
+ "loss": 0.1601,
471
+ "step": 1700
472
+ },
473
+ {
474
+ "epoch": 2.22,
475
+ "learning_rate": 1.7110848421417645e-05,
476
+ "loss": 0.1397,
477
+ "step": 1725
478
+ },
479
+ {
480
+ "epoch": 2.26,
481
+ "learning_rate": 1.7110848421417645e-05,
482
+ "loss": 0.109,
483
+ "step": 1750
484
+ },
485
+ {
486
+ "epoch": 2.26,
487
+ "eval_accuracy": 0.8814432989690721,
488
+ "eval_loss": 0.5400919318199158,
489
+ "eval_runtime": 65.5029,
490
+ "eval_samples_per_second": 47.387,
491
+ "eval_steps_per_second": 11.847,
492
+ "step": 1750
493
+ },
494
+ {
495
+ "epoch": 2.29,
496
+ "learning_rate": 1.7110848421417645e-05,
497
+ "loss": 0.1069,
498
+ "step": 1775
499
+ },
500
+ {
501
+ "epoch": 2.32,
502
+ "learning_rate": 1.7110848421417645e-05,
503
+ "loss": 0.1495,
504
+ "step": 1800
505
+ },
506
+ {
507
+ "epoch": 2.35,
508
+ "learning_rate": 1.7110848421417645e-05,
509
+ "loss": 0.1209,
510
+ "step": 1825
511
+ },
512
+ {
513
+ "epoch": 2.38,
514
+ "learning_rate": 1.7110848421417645e-05,
515
+ "loss": 0.134,
516
+ "step": 1850
517
+ },
518
+ {
519
+ "epoch": 2.42,
520
+ "learning_rate": 1.7110848421417645e-05,
521
+ "loss": 0.1357,
522
+ "step": 1875
523
+ },
524
+ {
525
+ "epoch": 2.45,
526
+ "learning_rate": 1.7110848421417645e-05,
527
+ "loss": 0.1114,
528
+ "step": 1900
529
+ },
530
+ {
531
+ "epoch": 2.48,
532
+ "learning_rate": 1.7110848421417645e-05,
533
+ "loss": 0.1436,
534
+ "step": 1925
535
+ },
536
+ {
537
+ "epoch": 2.51,
538
+ "learning_rate": 1.7110848421417645e-05,
539
+ "loss": 0.1775,
540
+ "step": 1950
541
+ },
542
+ {
543
+ "epoch": 2.55,
544
+ "learning_rate": 1.7110848421417645e-05,
545
+ "loss": 0.1232,
546
+ "step": 1975
547
+ },
548
+ {
549
+ "epoch": 2.58,
550
+ "learning_rate": 1.7110848421417645e-05,
551
+ "loss": 0.115,
552
+ "step": 2000
553
+ },
554
+ {
555
+ "epoch": 2.58,
556
+ "eval_accuracy": 0.8833762886597938,
557
+ "eval_loss": 0.5348294973373413,
558
+ "eval_runtime": 67.3607,
559
+ "eval_samples_per_second": 46.08,
560
+ "eval_steps_per_second": 11.52,
561
+ "step": 2000
562
+ },
563
+ {
564
+ "epoch": 2.61,
565
+ "learning_rate": 1.7110848421417645e-05,
566
+ "loss": 0.1429,
567
+ "step": 2025
568
+ },
569
+ {
570
+ "epoch": 2.64,
571
+ "learning_rate": 1.7110848421417645e-05,
572
+ "loss": 0.1612,
573
+ "step": 2050
574
+ },
575
+ {
576
+ "epoch": 2.67,
577
+ "learning_rate": 1.7110848421417645e-05,
578
+ "loss": 0.1565,
579
+ "step": 2075
580
+ },
581
+ {
582
+ "epoch": 2.71,
583
+ "learning_rate": 1.7110848421417645e-05,
584
+ "loss": 0.1188,
585
+ "step": 2100
586
+ },
587
+ {
588
+ "epoch": 2.74,
589
+ "learning_rate": 1.7110848421417645e-05,
590
+ "loss": 0.1576,
591
+ "step": 2125
592
+ },
593
+ {
594
+ "epoch": 2.77,
595
+ "learning_rate": 1.7110848421417645e-05,
596
+ "loss": 0.1832,
597
+ "step": 2150
598
+ },
599
+ {
600
+ "epoch": 2.8,
601
+ "learning_rate": 1.7110848421417645e-05,
602
+ "loss": 0.1643,
603
+ "step": 2175
604
+ },
605
+ {
606
+ "epoch": 2.84,
607
+ "learning_rate": 1.7110848421417645e-05,
608
+ "loss": 0.1668,
609
+ "step": 2200
610
+ },
611
+ {
612
+ "epoch": 2.87,
613
+ "learning_rate": 1.7110848421417645e-05,
614
+ "loss": 0.1984,
615
+ "step": 2225
616
+ },
617
+ {
618
+ "epoch": 2.9,
619
+ "learning_rate": 1.7110848421417645e-05,
620
+ "loss": 0.1839,
621
+ "step": 2250
622
+ },
623
+ {
624
+ "epoch": 2.9,
625
+ "eval_accuracy": 0.8424613402061856,
626
+ "eval_loss": 0.6889757513999939,
627
+ "eval_runtime": 65.603,
628
+ "eval_samples_per_second": 47.315,
629
+ "eval_steps_per_second": 11.829,
630
+ "step": 2250
631
+ },
632
+ {
633
+ "epoch": 2.93,
634
+ "learning_rate": 1.7110848421417645e-05,
635
+ "loss": 0.2497,
636
+ "step": 2275
637
+ },
638
+ {
639
+ "epoch": 2.96,
640
+ "learning_rate": 1.7110848421417645e-05,
641
+ "loss": 0.1096,
642
+ "step": 2300
643
+ },
644
+ {
645
+ "epoch": 3.0,
646
+ "learning_rate": 1.7110848421417645e-05,
647
+ "loss": 0.157,
648
+ "step": 2325
649
+ },
650
+ {
651
+ "epoch": 3.03,
652
+ "learning_rate": 1.7110848421417645e-05,
653
+ "loss": 0.0431,
654
+ "step": 2350
655
+ },
656
+ {
657
+ "epoch": 3.06,
658
+ "learning_rate": 1.7110848421417645e-05,
659
+ "loss": 0.0428,
660
+ "step": 2375
661
+ },
662
+ {
663
+ "epoch": 3.09,
664
+ "learning_rate": 1.7110848421417645e-05,
665
+ "loss": 0.1085,
666
+ "step": 2400
667
+ },
668
+ {
669
+ "epoch": 3.12,
670
+ "learning_rate": 1.7110848421417645e-05,
671
+ "loss": 0.0781,
672
+ "step": 2425
673
+ },
674
+ {
675
+ "epoch": 3.16,
676
+ "learning_rate": 1.7110848421417645e-05,
677
+ "loss": 0.1603,
678
+ "step": 2450
679
+ },
680
+ {
681
+ "epoch": 3.19,
682
+ "learning_rate": 1.7110848421417645e-05,
683
+ "loss": 0.0825,
684
+ "step": 2475
685
+ },
686
+ {
687
+ "epoch": 3.22,
688
+ "learning_rate": 1.7110848421417645e-05,
689
+ "loss": 0.0855,
690
+ "step": 2500
691
+ },
692
+ {
693
+ "epoch": 3.22,
694
+ "eval_accuracy": 0.8759664948453608,
695
+ "eval_loss": 0.6595790982246399,
696
+ "eval_runtime": 65.5897,
697
+ "eval_samples_per_second": 47.324,
698
+ "eval_steps_per_second": 11.831,
699
+ "step": 2500
700
+ },
701
+ {
702
+ "epoch": 3.25,
703
+ "learning_rate": 1.7110848421417645e-05,
704
+ "loss": 0.0614,
705
+ "step": 2525
706
+ },
707
+ {
708
+ "epoch": 3.29,
709
+ "learning_rate": 1.7110848421417645e-05,
710
+ "loss": 0.0686,
711
+ "step": 2550
712
+ },
713
+ {
714
+ "epoch": 3.32,
715
+ "learning_rate": 1.7110848421417645e-05,
716
+ "loss": 0.0716,
717
+ "step": 2575
718
+ },
719
+ {
720
+ "epoch": 3.35,
721
+ "learning_rate": 1.7110848421417645e-05,
722
+ "loss": 0.1359,
723
+ "step": 2600
724
+ },
725
+ {
726
+ "epoch": 3.38,
727
+ "learning_rate": 1.7110848421417645e-05,
728
+ "loss": 0.0863,
729
+ "step": 2625
730
+ },
731
+ {
732
+ "epoch": 3.41,
733
+ "learning_rate": 1.7110848421417645e-05,
734
+ "loss": 0.0915,
735
+ "step": 2650
736
+ },
737
+ {
738
+ "epoch": 3.45,
739
+ "learning_rate": 1.7110848421417645e-05,
740
+ "loss": 0.0595,
741
+ "step": 2675
742
+ },
743
+ {
744
+ "epoch": 3.48,
745
+ "learning_rate": 1.7110848421417645e-05,
746
+ "loss": 0.0714,
747
+ "step": 2700
748
+ },
749
+ {
750
+ "epoch": 3.51,
751
+ "learning_rate": 1.7110848421417645e-05,
752
+ "loss": 0.0914,
753
+ "step": 2725
754
+ },
755
+ {
756
+ "epoch": 3.54,
757
+ "learning_rate": 1.7110848421417645e-05,
758
+ "loss": 0.0774,
759
+ "step": 2750
760
+ },
761
+ {
762
+ "epoch": 3.54,
763
+ "eval_accuracy": 0.8740335051546392,
764
+ "eval_loss": 0.7505560517311096,
765
+ "eval_runtime": 65.5758,
766
+ "eval_samples_per_second": 47.335,
767
+ "eval_steps_per_second": 11.834,
768
+ "step": 2750
769
+ },
770
+ {
771
+ "epoch": 3.58,
772
+ "learning_rate": 1.7110848421417645e-05,
773
+ "loss": 0.0685,
774
+ "step": 2775
775
+ },
776
+ {
777
+ "epoch": 3.61,
778
+ "learning_rate": 1.7110848421417645e-05,
779
+ "loss": 0.0943,
780
+ "step": 2800
781
+ },
782
+ {
783
+ "epoch": 3.64,
784
+ "learning_rate": 1.7110848421417645e-05,
785
+ "loss": 0.1181,
786
+ "step": 2825
787
+ },
788
+ {
789
+ "epoch": 3.67,
790
+ "learning_rate": 1.7110848421417645e-05,
791
+ "loss": 0.1302,
792
+ "step": 2850
793
+ },
794
+ {
795
+ "epoch": 3.7,
796
+ "learning_rate": 1.7110848421417645e-05,
797
+ "loss": 0.1058,
798
+ "step": 2875
799
+ },
800
+ {
801
+ "epoch": 3.74,
802
+ "learning_rate": 1.7110848421417645e-05,
803
+ "loss": 0.1235,
804
+ "step": 2900
805
+ },
806
+ {
807
+ "epoch": 3.77,
808
+ "learning_rate": 1.7110848421417645e-05,
809
+ "loss": 0.1278,
810
+ "step": 2925
811
+ },
812
+ {
813
+ "epoch": 3.8,
814
+ "learning_rate": 1.7110848421417645e-05,
815
+ "loss": 0.1172,
816
+ "step": 2950
817
+ },
818
+ {
819
+ "epoch": 3.83,
820
+ "learning_rate": 1.7110848421417645e-05,
821
+ "loss": 0.1461,
822
+ "step": 2975
823
+ },
824
+ {
825
+ "epoch": 3.87,
826
+ "learning_rate": 1.7110848421417645e-05,
827
+ "loss": 0.1213,
828
+ "step": 3000
829
+ },
830
+ {
831
+ "epoch": 3.87,
832
+ "eval_accuracy": 0.8762886597938144,
833
+ "eval_loss": 0.6005998849868774,
834
+ "eval_runtime": 65.6002,
835
+ "eval_samples_per_second": 47.317,
836
+ "eval_steps_per_second": 11.829,
837
+ "step": 3000
838
+ }
839
+ ],
840
+ "logging_steps": 25,
841
+ "max_steps": 3000,
842
+ "num_input_tokens_seen": 0,
843
+ "num_train_epochs": 4,
844
+ "save_steps": 250,
845
+ "total_flos": 1.262177729399808e+16,
846
+ "trial_name": null,
847
+ "trial_params": {
848
+ "learning_rate": 1.7110848421417645e-05,
849
+ "per_device_eval_batch_size": 4,
850
+ "per_device_train_batch_size": 16
851
+ }
852
+ }
run-1/checkpoint-3000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ff7ff5e497d2fd17b0b0a335d1e1e52fdf9ab64c61fc776fe6063c480e0a9f6
3
+ size 4728
run-2/checkpoint-2500/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-cased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "LABEL_0",
14
+ "1": "LABEL_1",
15
+ "2": "LABEL_2",
16
+ "3": "LABEL_3",
17
+ "4": "LABEL_4"
18
+ },
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 3072,
21
+ "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3,
26
+ "LABEL_4": 4
27
+ },
28
+ "layer_norm_eps": 1e-12,
29
+ "max_position_embeddings": 512,
30
+ "model_type": "bert",
31
+ "num_attention_heads": 12,
32
+ "num_hidden_layers": 12,
33
+ "pad_token_id": 0,
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "single_label_classification",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.36.0.dev0",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 28996
41
+ }
run-2/checkpoint-2500/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0935e9b504369338f73d7b5f1be94c932055cbd56ef6d774f770dfbab5f141e9
3
+ size 433279996
run-2/checkpoint-2500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a8f066b1a71ae52b3aa715094859018a19afc6f47f577e38281af9bc85a0c94
3
+ size 866681082
run-2/checkpoint-2500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63399af7a409c183004eb47ca7dbc6f06cf71063012d7d165e74c89aaf482739
3
+ size 14244
run-2/checkpoint-2500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:390b240ea9153d81514422429b6a6dd0ee903146d58db9794a5b9cd5cbdd2201
3
+ size 1064