File size: 20,864 Bytes
c819893
 
 
 
18f4715
 
 
 
 
c819893
 
 
 
 
 
 
 
 
 
41be842
18f4715
6b710db
 
 
 
 
 
 
 
c819893
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41be842
 
 
c819893
68e20e0
 
c819893
41be842
 
c819893
68e20e0
c819893
41be842
c819893
 
 
 
41be842
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c819893
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
---
library_name: transformers
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: bert-sentiment-classifier
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-sentiment-classifier

This model was trained from scratch on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0820
- Accuracy: 0.9736
- F1: 0.9736
- Precision: 0.9736
- Recall: 0.9736
- F1 Negative: 0.9736
- F1 Neutral: 0.9736
- F1 Positive: 0.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2.0000000000000003e-06
- train_batch_size: 128
- eval_batch_size: 256
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 2
- total_train_batch_size: 512
- total_eval_batch_size: 512
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step   | Accuracy | F1     | F1 Negative | F1 Neutral | F1 Positive | Validation Loss | Precision | Recall |
|:-------------:|:------:|:------:|:--------:|:------:|:-----------:|:----------:|:-----------:|:---------------:|:---------:|:------:|
| 0.1503        | 0.0711 | 1000   | 0.9526   | 0.9526 | 0.9529      | 0.9524     | 0.0         | 0.1399          | 0.9527    | 0.9526 |
| 0.123         | 0.1422 | 2000   | 0.9613   | 0.9613 | 0.9612      | 0.9614     | 0.0         | 0.1169          | 0.9613    | 0.9613 |
| 0.1085        | 0.2133 | 3000   | 0.9623   | 0.9623 | 0.9619      | 0.9627     | 0.0         | 0.1103          | 0.9625    | 0.9623 |
| 0.1071        | 0.2845 | 4000   | 0.9658   | 0.9658 | 0.9656      | 0.9659     | 0.0         | 0.0997          | 0.9658    | 0.9658 |
| 0.1048        | 0.3556 | 5000   | 0.9669   | 0.9669 | 0.9671      | 0.9668     | 0.0         | 0.0973          | 0.9670    | 0.9669 |
| 0.0952        | 0.4267 | 6000   | 0.9674   | 0.9674 | 0.9676      | 0.9671     | 0.0         | 0.1002          | 0.9674    | 0.9674 |
| 0.098         | 0.4978 | 7000   | 0.9689   | 0.9689 | 0.9689      | 0.9689     | 0.0         | 0.0952          | 0.9689    | 0.9689 |
| 0.0967        | 0.5689 | 8000   | 0.9689   | 0.9689 | 0.9689      | 0.9690     | 0.0         | 0.0930          | 0.9689    | 0.9689 |
| 0.0936        | 0.6400 | 9000   | 0.9693   | 0.9693 | 0.9691      | 0.9695     | 0.0         | 0.0926          | 0.9694    | 0.9693 |
| 0.0904        | 0.7111 | 10000  | 0.9691   | 0.9691 | 0.9689      | 0.9694     | 0.0         | 0.0946          | 0.9693    | 0.9691 |
| 0.0943        | 0.7823 | 11000  | 0.9700   | 0.9700 | 0.9698      | 0.9701     | 0.0         | 0.0880          | 0.9700    | 0.9700 |
| 0.0921        | 0.8534 | 12000  | 0.9703   | 0.9703 | 0.9701      | 0.9704     | 0.0         | 0.0867          | 0.9703    | 0.9703 |
| 0.0867        | 0.9245 | 13000  | 0.9704   | 0.9704 | 0.9702      | 0.9706     | 0.0         | 0.0878          | 0.9704    | 0.9704 |
| 0.0863        | 0.9956 | 14000  | 0.9707   | 0.9707 | 0.9706      | 0.9708     | 0.0         | 0.0871          | 0.9707    | 0.9707 |
| 0.0798        | 1.0667 | 15000  | 0.9709   | 0.9709 | 0.9710      | 0.9709     | 0.0         | 0.0883          | 0.9710    | 0.9709 |
| 0.0772        | 1.1378 | 16000  | 0.9711   | 0.9711 | 0.9710      | 0.9712     | 0.0         | 0.0871          | 0.9712    | 0.9711 |
| 0.0759        | 1.2089 | 17000  | 0.9719   | 0.9719 | 0.9719      | 0.9719     | 0.0         | 0.0884          | 0.9719    | 0.9719 |
| 0.0767        | 1.2800 | 18000  | 0.9717   | 0.9717 | 0.9715      | 0.9718     | 0.0         | 0.0857          | 0.9717    | 0.9717 |
| 0.0791        | 1.3512 | 19000  | 0.9718   | 0.9718 | 0.9717      | 0.9719     | 0.0         | 0.0870          | 0.9718    | 0.9718 |
| 0.0766        | 1.4223 | 20000  | 0.9722   | 0.9722 | 0.9722      | 0.9721     | 0.0         | 0.0827          | 0.9722    | 0.9722 |
| 0.0808        | 1.4934 | 21000  | 0.9725   | 0.9725 | 0.9725      | 0.9725     | 0.0         | 0.0829          | 0.9725    | 0.9725 |
| 0.0784        | 1.5645 | 22000  | 0.9726   | 0.9726 | 0.9726      | 0.9725     | 0.0         | 0.0824          | 0.9726    | 0.9726 |
| 0.0814        | 1.6356 | 23000  | 0.9727   | 0.9727 | 0.9727      | 0.9727     | 0.0         | 0.0811          | 0.9727    | 0.9727 |
| 0.0789        | 1.7067 | 24000  | 0.9727   | 0.9727 | 0.9727      | 0.9727     | 0.0         | 0.0825          | 0.9727    | 0.9727 |
| 0.0762        | 1.7778 | 25000  | 0.9734   | 0.9734 | 0.9734      | 0.9734     | 0.0         | 0.0806          | 0.9734    | 0.9734 |
| 0.0766        | 1.8490 | 26000  | 0.9732   | 0.9732 | 0.9731      | 0.9732     | 0.0         | 0.0813          | 0.9732    | 0.9732 |
| 0.0764        | 1.9201 | 27000  | 0.9728   | 0.9728 | 0.9727      | 0.9730     | 0.0         | 0.0825          | 0.9729    | 0.9728 |
| 0.0737        | 1.9912 | 28000  | 0.9732   | 0.9732 | 0.9733      | 0.9730     | 0.0         | 0.0818          | 0.9732    | 0.9732 |
| 0.0644        | 2.0623 | 29000  | 0.9733   | 0.9733 | 0.9732      | 0.9733     | 0.0         | 0.0835          | 0.9733    | 0.9733 |
| 0.0678        | 2.1334 | 30000  | 0.9732   | 0.9732 | 0.9732      | 0.9732     | 0.0         | 0.0841          | 0.9732    | 0.9732 |
| 0.0653        | 2.2045 | 31000  | 0.9734   | 0.9734 | 0.9734      | 0.9734     | 0.0         | 0.0842          | 0.9734    | 0.9734 |
| 0.0639        | 2.2756 | 32000  | 0.9734   | 0.9734 | 0.9734      | 0.9735     | 0.0         | 0.0827          | 0.9734    | 0.9734 |
| 0.0617        | 2.3468 | 33000  | 0.9734   | 0.9734 | 0.9733      | 0.9734     | 0.0         | 0.0835          | 0.9734    | 0.9734 |
| 0.0645        | 2.4179 | 34000  | 0.9734   | 0.9734 | 0.9735      | 0.9734     | 0.0         | 0.0824          | 0.9734    | 0.9734 |
| 0.0623        | 2.4890 | 35000  | 0.9733   | 0.9733 | 0.9733      | 0.9734     | 0.0         | 0.0827          | 0.9734    | 0.9733 |
| 0.0602        | 2.5601 | 36000  | 0.9734   | 0.9734 | 0.9734      | 0.9734     | 0.0         | 0.0833          | 0.9734    | 0.9734 |
| 0.0625        | 2.6312 | 37000  | 0.9734   | 0.9734 | 0.9733      | 0.9734     | 0.0         | 0.0830          | 0.9734    | 0.9734 |
| 0.0649        | 2.7023 | 38000  | 0.9734   | 0.9734 | 0.9734      | 0.9735     | 0.0         | 0.0825          | 0.9734    | 0.9734 |
| 0.0574        | 2.7734 | 39000  | 0.9735   | 0.9735 | 0.9735      | 0.9735     | 0.0         | 0.0828          | 0.9735    | 0.9735 |
| 0.0632        | 2.8445 | 40000  | 0.9736   | 0.9736 | 0.9736      | 0.9736     | 0.0         | 0.0821          | 0.9736    | 0.9736 |
| 0.0643        | 2.9157 | 41000  | 0.9736   | 0.9736 | 0.9736      | 0.9736     | 0.0         | 0.0820          | 0.9736    | 0.9736 |
| 0.0605        | 2.9868 | 42000  | 0.9736   | 0.9736 | 0.9736      | 0.9736     | 0.0         | 0.0820          | 0.9736    | 0.9736 |
| 0.0648        | 3.0579 | 43000  | 0.0896   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9724          | 0.9722    | 0.0    |
| 0.0748        | 3.1290 | 44000  | 0.0896   | 0.9716 | 0.9716      | 0.9717     | 0.9716      | 0.9714          | 0.9718    | 0.0    |
| 0.067         | 3.2001 | 45000  | 0.0862   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9723          | 0.9721    | 0.0    |
| 0.0708        | 3.2712 | 46000  | 0.0861   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9726          | 0.9725    | 0.0    |
| 0.0678        | 3.3423 | 47000  | 0.0857   | 0.9724 | 0.9724      | 0.9724     | 0.9724      | 0.9723          | 0.9724    | 0.0    |
| 0.071         | 3.4135 | 48000  | 0.0877   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9721    | 0.0    |
| 0.0717        | 3.4846 | 49000  | 0.0846   | 0.9719 | 0.9719      | 0.9720     | 0.9719      | 0.9718          | 0.9721    | 0.0    |
| 0.0731        | 3.5557 | 50000  | 0.0836   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.073         | 3.6268 | 51000  | 0.0825   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9727          | 0.9725    | 0.0    |
| 0.0726        | 3.6979 | 52000  | 0.0823   | 0.9728 | 0.9728      | 0.9728     | 0.9728      | 0.9727          | 0.9728    | 0.0    |
| 0.0686        | 3.7690 | 53000  | 0.0826   | 0.9728 | 0.9728      | 0.9728     | 0.9728      | 0.9728          | 0.9729    | 0.0    |
| 0.068         | 3.8401 | 54000  | 0.0824   | 0.9728 | 0.9728      | 0.9728     | 0.9728      | 0.9728          | 0.9728    | 0.0    |
| 0.0713        | 3.9113 | 55000  | 0.0835   | 0.9728 | 0.9728      | 0.9729     | 0.9728      | 0.9729          | 0.9728    | 0.0    |
| 0.0706        | 3.9824 | 56000  | 0.0827   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9724          | 0.9726    | 0.0    |
| 0.0601        | 4.0535 | 57000  | 0.0844   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9726          | 0.9725    | 0.0    |
| 0.0589        | 4.1246 | 58000  | 0.0879   | 0.9724 | 0.9724      | 0.9724     | 0.9724      | 0.9724          | 0.9724    | 0.0    |
| 0.0618        | 4.1957 | 59000  | 0.0868   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9725          | 0.9725    | 0.0    |
| 0.0609        | 4.2668 | 60000  | 0.0876   | 0.9724 | 0.9724      | 0.9725     | 0.9724      | 0.9725          | 0.9723    | 0.0    |
| 0.0605        | 4.3379 | 61000  | 0.0934   | 0.9716 | 0.9716      | 0.9717     | 0.9716      | 0.9714          | 0.9718    | 0.0    |
| 0.0588        | 4.4090 | 62000  | 0.0929   | 0.9728 | 0.9728      | 0.9728     | 0.9728      | 0.9727          | 0.9728    | 0.0    |
| 0.0613        | 4.4802 | 63000  | 0.0875   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9722          | 0.9724    | 0.0    |
| 0.0591        | 4.5513 | 64000  | 0.0888   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9726          | 0.9726    | 0.0    |
| 0.0618        | 4.6224 | 65000  | 0.0856   | 0.9724 | 0.9724      | 0.9725     | 0.9724      | 0.9725          | 0.9724    | 0.0    |
| 0.0564        | 4.6935 | 66000  | 0.0884   | 0.9727 | 0.9727      | 0.9727     | 0.9727      | 0.9727          | 0.9727    | 0.0    |
| 0.0629        | 4.7646 | 67000  | 0.0854   | 0.9729 | 0.9729      | 0.9729     | 0.9729      | 0.9729          | 0.9729    | 0.0    |
| 0.0601        | 4.8357 | 68000  | 0.0881   | 0.9729 | 0.9729      | 0.9729     | 0.9729      | 0.9729          | 0.9729    | 0.0    |
| 0.0619        | 4.9068 | 69000  | 0.0857   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9726          | 0.9726    | 0.0    |
| 0.0591        | 4.9780 | 70000  | 0.0857   | 0.9726 | 0.9726      | 0.9726     | 0.9726      | 0.9725          | 0.9727    | 0.0    |
| 0.0499        | 5.0491 | 71000  | 0.0895   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9725          | 0.9726    | 0.0    |
| 0.0526        | 5.1202 | 72000  | 0.0912   | 0.9724 | 0.9724      | 0.9724     | 0.9724      | 0.9723          | 0.9724    | 0.0    |
| 0.0543        | 5.1913 | 73000  | 0.0943   | 0.9727 | 0.9727      | 0.9727     | 0.9727      | 0.9726          | 0.9727    | 0.0    |
| 0.0526        | 5.2624 | 74000  | 0.0920   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9723          | 0.9724    | 0.0    |
| 0.0576        | 5.3335 | 75000  | 0.0901   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9720          | 0.9722    | 0.0    |
| 0.0518        | 5.4046 | 76000  | 0.0951   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9717          | 0.9719    | 0.0    |
| 0.0533        | 5.4758 | 77000  | 0.0898   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.0485        | 5.5469 | 78000  | 0.0941   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9721          | 0.9722    | 0.0    |
| 0.052         | 5.6180 | 79000  | 0.0909   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9724          | 0.9725    | 0.0    |
| 0.0505        | 5.6891 | 80000  | 0.0957   | 0.9721 | 0.9721      | 0.9722     | 0.9721      | 0.9720          | 0.9722    | 0.0    |
| 0.0525        | 5.7602 | 81000  | 0.0934   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9723          | 0.9723    | 0.0    |
| 0.0491        | 5.8313 | 82000  | 0.0921   | 0.9724 | 0.9724      | 0.9724     | 0.9724      | 0.9723          | 0.9724    | 0.0    |
| 0.0538        | 5.9024 | 83000  | 0.0920   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9724          | 0.9725    | 0.0    |
| 0.0516        | 5.9735 | 84000  | 0.0917   | 0.9725 | 0.9725      | 0.9725     | 0.9725      | 0.9724          | 0.9725    | 0.0    |
| 0.041         | 6.0447 | 85000  | 0.1008   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9719          | 0.9722    | 0.0    |
| 0.0424        | 6.1158 | 86000  | 0.1065   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9718          | 0.9720    | 0.0    |
| 0.0442        | 6.1869 | 87000  | 0.0982   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9723    | 0.0    |
| 0.0435        | 6.2580 | 88000  | 0.1031   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9720          | 0.9722    | 0.0    |
| 0.0421        | 6.3291 | 89000  | 0.1003   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9719          | 0.9719    | 0.0    |
| 0.0408        | 6.4002 | 90000  | 0.1028   | 0.9720 | 0.9720      | 0.9720     | 0.9720      | 0.9719          | 0.9720    | 0.0    |
| 0.0441        | 6.4713 | 91000  | 0.0981   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9717          | 0.9719    | 0.0    |
| 0.0447        | 6.5425 | 92000  | 0.0952   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9723          | 0.9723    | 0.0    |
| 0.0461        | 6.6136 | 93000  | 0.0949   | 0.9720 | 0.9720      | 0.9720     | 0.9720      | 0.9720          | 0.9720    | 0.0    |
| 0.0439        | 6.6847 | 94000  | 0.0988   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9723    | 0.0    |
| 0.0443        | 6.7558 | 95000  | 0.0988   | 0.9720 | 0.9720      | 0.9720     | 0.9720      | 0.9719          | 0.9720    | 0.0    |
| 0.0395        | 6.8269 | 96000  | 0.1013   | 0.9723 | 0.9723      | 0.9723     | 0.9723      | 0.9724          | 0.9722    | 0.0    |
| 0.0414        | 6.8980 | 97000  | 0.1010   | 0.9724 | 0.9724      | 0.9724     | 0.9724      | 0.9724          | 0.9724    | 0.0    |
| 0.0469        | 6.9691 | 98000  | 0.0998   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9723    | 0.0    |
| 0.0329        | 7.0403 | 99000  | 0.1126   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9721          | 0.9721    | 0.0    |
| 0.038         | 7.1114 | 100000 | 0.1076   | 0.9714 | 0.9714      | 0.9714     | 0.9714      | 0.9713          | 0.9715    | 0.0    |
| 0.0374        | 7.1825 | 101000 | 0.1045   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.0377        | 7.2536 | 102000 | 0.1080   | 0.9718 | 0.6479      | 0.6479     | 0.6479      | 0.9717          | 0.0       | 0.9719 |
| 0.0393        | 7.3247 | 103000 | 0.1036   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.0393        | 7.3958 | 104000 | 0.1045   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.0382        | 7.4669 | 105000 | 0.1081   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9717          | 0.9719    | 0.0    |
| 0.0374        | 7.5380 | 106000 | 0.1010   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9719          | 0.9719    | 0.0    |
| 0.0355        | 7.6092 | 107000 | 0.1092   | 0.9717 | 0.9717      | 0.9718     | 0.9717      | 0.9716          | 0.9719    | 0.0    |
| 0.035         | 7.6803 | 108000 | 0.1101   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9720          | 0.9721    | 0.0    |
| 0.0344        | 7.7514 | 109000 | 0.1089   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9721          | 0.9722    | 0.0    |
| 0.0348        | 7.8225 | 110000 | 0.1095   | 0.9721 | 0.9721      | 0.9721     | 0.9721      | 0.9721          | 0.9721    | 0.0    |
| 0.0387        | 7.8936 | 111000 | 0.1062   | 0.9717 | 0.9717      | 0.9717     | 0.9717      | 0.9716          | 0.9718    | 0.0    |
| 0.0352        | 7.9647 | 112000 | 0.1076   | 0.9722 | 0.9722      | 0.9722     | 0.9722      | 0.9722          | 0.9722    | 0.0    |
| 0.0331        | 8.0358 | 113000 | 0.1149   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9718          | 0.9719    | 0.0    |
| 0.0362        | 8.1070 | 114000 | 0.1122   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9718          | 0.9718    | 0.0    |
| 0.0357        | 8.1781 | 115000 | 0.1107   | 0.9714 | 0.9714      | 0.9714     | 0.9714      | 0.9713          | 0.9715    | 0.0    |
| 0.0328        | 8.2492 | 116000 | 0.1136   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9719          | 0.9719    | 0.0    |
| 0.0327        | 8.3203 | 117000 | 0.1178   | 0.9717 | 0.9717      | 0.9717     | 0.9717      | 0.9717          | 0.9718    | 0.0    |
| 0.0315        | 8.3914 | 118000 | 0.1154   | 0.9715 | 0.6477      | 0.6477     | 0.6477      | 0.9714          | 0.0       | 0.9716 |
| 0.031         | 8.4625 | 119000 | 0.1137   | 0.9717 | 0.9717      | 0.9717     | 0.9717      | 0.9717          | 0.9718    | 0.0    |
| 0.0314        | 8.5336 | 120000 | 0.1128   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9718          | 0.9718    | 0.0    |
| 0.0352        | 8.6048 | 121000 | 0.1111   | 0.9717 | 0.9717      | 0.9717     | 0.9717      | 0.9716          | 0.9717    | 0.0    |
| 0.0356        | 8.6759 | 122000 | 0.1112   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9718          | 0.9719    | 0.0    |
| 0.0345        | 8.7470 | 123000 | 0.1135   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9718          | 0.9719    | 0.0    |
| 0.0351        | 8.8181 | 124000 | 0.1144   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9717          | 0.9718    | 0.0    |
| 0.0311        | 8.8892 | 125000 | 0.1149   | 0.9719 | 0.9719      | 0.9719     | 0.9719      | 0.9719          | 0.9720    | 0.0    |
| 0.0353        | 8.9603 | 126000 | 0.1118   | 0.9718 | 0.9718      | 0.9718     | 0.9718      | 0.9718          | 0.9718    | 0.0    |
| 0.0331        | 9.0314 | 127000 | 0.1162   | 0.9717 | 0.9717      | 0.9717     | 0.9717      | 0.9717          | 0.9718    | 0.0    |
| 0.0282        | 9.1025 | 128000 | 0.1181   | 0.9716 | 0.6478      | 0.6478     | 0.6478      | 0.9716          | 0.0       | 0.9717 |
| 0.0307        | 9.1737 | 129000 | 0.1176   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9716          | 0.0       | 0.9717 |
| 0.0297        | 9.2448 | 130000 | 0.1192   | 0.9716 | 0.6477      | 0.6478     | 0.6477      | 0.9716          | 0.0       | 0.9717 |
| 0.0334        | 9.3159 | 131000 | 0.1174   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9716          | 0.0       | 0.9717 |
| 0.0295        | 9.3870 | 132000 | 0.1178   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0321        | 9.4581 | 133000 | 0.1168   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0323        | 9.5292 | 134000 | 0.1169   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9716          | 0.0       | 0.9717 |
| 0.0295        | 9.6003 | 135000 | 0.1174   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9716          | 0.0       | 0.9718 |
| 0.0327        | 9.6715 | 136000 | 0.1179   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0323        | 9.7426 | 137000 | 0.1176   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0275        | 9.8137 | 138000 | 0.1177   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0339        | 9.8848 | 139000 | 0.1176   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |
| 0.0325        | 9.9559 | 140000 | 0.1175   | 0.9717 | 0.6478      | 0.6478     | 0.6478      | 0.9717          | 0.0       | 0.9718 |


### Framework versions

- Transformers 4.57.1
- Pytorch 2.9.0+cu128
- Datasets 4.3.0
- Tokenizers 0.22.1