bmuscato commited on
Commit
72d657c
·
verified ·
1 Parent(s): ff87c3a

Upload folder using huggingface_hub

Browse files
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ base_model: aubmindlab/bert-base-arabertv2
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: arabert_armis_multitask_hard
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # arabert_armis_multitask_hard
15
+
16
+ This model is a fine-tuned version of [aubmindlab/bert-base-arabertv2](https://huggingface.co/aubmindlab/bert-base-arabertv2) on the None dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 0.7896
19
+ - Disagreement Accuracy: 0.5035
20
+ - Disagreement F1: 0.4167
21
+ - Target Accuracy: 0.6525
22
+ - Target F1: 0.6202
23
+
24
+ ## Model description
25
+
26
+ More information needed
27
+
28
+ ## Intended uses & limitations
29
+
30
+ More information needed
31
+
32
+ ## Training and evaluation data
33
+
34
+ More information needed
35
+
36
+ ## Training procedure
37
+
38
+ ### Training hyperparameters
39
+
40
+ The following hyperparameters were used during training:
41
+ - learning_rate: 2e-05
42
+ - train_batch_size: 32
43
+ - eval_batch_size: 32
44
+ - seed: 42
45
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
+ - lr_scheduler_type: linear
47
+ - num_epochs: 20
48
+
49
+ ### Training results
50
+
51
+ | Training Loss | Epoch | Step | Validation Loss | Disagreement Accuracy | Disagreement F1 | Target Accuracy | Target F1 |
52
+ |:-------------:|:-----:|:----:|:---------------:|:---------------------:|:---------------:|:---------------:|:---------:|
53
+ | 0.8457 | 1.0 | 21 | 0.8400 | 0.5390 | 0.4961 | 0.4610 | 0.5778 |
54
+ | 0.8372 | 2.0 | 42 | 0.8325 | 0.5461 | 0.4386 | 0.6241 | 0.5225 |
55
+ | 0.8341 | 3.0 | 63 | 0.8290 | 0.5532 | 0.4324 | 0.5390 | 0.5517 |
56
+ | 0.8153 | 4.0 | 84 | 0.8240 | 0.5532 | 0.496 | 0.5603 | 0.5634 |
57
+ | 0.809 | 5.0 | 105 | 0.8209 | 0.5319 | 0.3125 | 0.5816 | 0.5874 |
58
+ | 0.79 | 6.0 | 126 | 0.8150 | 0.5390 | 0.4882 | 0.6099 | 0.5669 |
59
+ | 0.7857 | 7.0 | 147 | 0.8119 | 0.5177 | 0.3462 | 0.6312 | 0.5806 |
60
+ | 0.7775 | 8.0 | 168 | 0.8090 | 0.5035 | 0.3137 | 0.6099 | 0.5926 |
61
+ | 0.7763 | 9.0 | 189 | 0.8041 | 0.5106 | 0.448 | 0.6383 | 0.5984 |
62
+ | 0.7685 | 10.0 | 210 | 0.8015 | 0.5106 | 0.3670 | 0.6454 | 0.6094 |
63
+ | 0.7652 | 11.0 | 231 | 0.7990 | 0.5035 | 0.4262 | 0.6383 | 0.6165 |
64
+ | 0.7597 | 12.0 | 252 | 0.7979 | 0.5106 | 0.4651 | 0.6241 | 0.6074 |
65
+ | 0.753 | 13.0 | 273 | 0.7955 | 0.4894 | 0.4286 | 0.6454 | 0.6212 |
66
+ | 0.7475 | 14.0 | 294 | 0.7930 | 0.5035 | 0.4167 | 0.6596 | 0.6190 |
67
+ | 0.7465 | 15.0 | 315 | 0.7923 | 0.4965 | 0.4132 | 0.6525 | 0.6202 |
68
+ | 0.7425 | 16.0 | 336 | 0.7914 | 0.5035 | 0.4167 | 0.6454 | 0.6032 |
69
+ | 0.7407 | 17.0 | 357 | 0.7905 | 0.4965 | 0.4132 | 0.6525 | 0.6202 |
70
+ | 0.7397 | 18.0 | 378 | 0.7901 | 0.5248 | 0.4174 | 0.6525 | 0.6202 |
71
+ | 0.7362 | 19.0 | 399 | 0.7897 | 0.5035 | 0.4167 | 0.6525 | 0.6202 |
72
+ | 0.7358 | 20.0 | 420 | 0.7896 | 0.5035 | 0.4167 | 0.6525 | 0.6202 |
73
+
74
+
75
+ ### Framework versions
76
+
77
+ - Transformers 4.57.1
78
+ - Pytorch 2.5.1+cu121
79
+ - Datasets 4.3.0
80
+ - Tokenizers 0.22.1
all_task_heads.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f2cd4b6f426c8d70ea4cda6f0e46bf7d714a4de47b88b46df0653936628c20f
3
+ size 8520
checkpoint-1320/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e32d4ad8ae637fdbd984570d15f5561bda6f57f9e0a886f8d35c192331729fee
3
+ size 437959496
checkpoint-1320/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68bbcce324d0f608337adea146787bdd6d10c16afa5f44957d0a9d2b3ad4c078
3
+ size 16658
checkpoint-1320/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a582225e6c4c2d8175a31cc490b583753045333a08eb50c9a6441ea656788ef9
3
+ size 14244
checkpoint-1320/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9be02f9b01f58d3823071730362dbeb21fad4741731ac81caf91af97ec55ee2
3
+ size 1064
checkpoint-1320/special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
checkpoint-1320/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-1320/tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "BertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
checkpoint-1320/trainer_state.json ADDED
@@ -0,0 +1,423 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 1320,
3
+ "best_metric": 0.7580272555351257,
4
+ "best_model_checkpoint": "./multitask_model2/checkpoint-1320",
5
+ "epoch": 20.0,
6
+ "eval_steps": 500,
7
+ "global_step": 1320,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 1.0,
14
+ "grad_norm": 3.3046741485595703,
15
+ "learning_rate": 1.9015151515151516e-05,
16
+ "loss": 0.7555,
17
+ "step": 66
18
+ },
19
+ {
20
+ "epoch": 1.0,
21
+ "eval_disagreement_accuracy": 0.3986636971046771,
22
+ "eval_disagreement_f1": 0.3382352941176471,
23
+ "eval_loss": 0.7626323699951172,
24
+ "eval_runtime": 2.5766,
25
+ "eval_samples_per_second": 174.262,
26
+ "eval_steps_per_second": 5.822,
27
+ "eval_target_accuracy": 0.34521158129175944,
28
+ "eval_target_f1": 0.4024390243902439,
29
+ "step": 66
30
+ },
31
+ {
32
+ "epoch": 2.0,
33
+ "grad_norm": 2.7593727111816406,
34
+ "learning_rate": 1.8015151515151516e-05,
35
+ "loss": 0.7608,
36
+ "step": 132
37
+ },
38
+ {
39
+ "epoch": 2.0,
40
+ "eval_disagreement_accuracy": 0.41648106904231624,
41
+ "eval_disagreement_f1": 0.40454545454545454,
42
+ "eval_loss": 0.7616363763809204,
43
+ "eval_runtime": 2.1011,
44
+ "eval_samples_per_second": 213.695,
45
+ "eval_steps_per_second": 7.139,
46
+ "eval_target_accuracy": 0.36525612472160357,
47
+ "eval_target_f1": 0.38177874186550975,
48
+ "step": 132
49
+ },
50
+ {
51
+ "epoch": 3.0,
52
+ "grad_norm": 2.1166303157806396,
53
+ "learning_rate": 1.7015151515151515e-05,
54
+ "loss": 0.7554,
55
+ "step": 198
56
+ },
57
+ {
58
+ "epoch": 3.0,
59
+ "eval_disagreement_accuracy": 0.42316258351893093,
60
+ "eval_disagreement_f1": 0.45010615711252655,
61
+ "eval_loss": 0.760993480682373,
62
+ "eval_runtime": 2.5676,
63
+ "eval_samples_per_second": 174.871,
64
+ "eval_steps_per_second": 5.842,
65
+ "eval_target_accuracy": 0.37861915367483295,
66
+ "eval_target_f1": 0.3673469387755102,
67
+ "step": 198
68
+ },
69
+ {
70
+ "epoch": 4.0,
71
+ "grad_norm": 2.2470221519470215,
72
+ "learning_rate": 1.6015151515151518e-05,
73
+ "loss": 0.7577,
74
+ "step": 264
75
+ },
76
+ {
77
+ "epoch": 4.0,
78
+ "eval_disagreement_accuracy": 0.42761692650334077,
79
+ "eval_disagreement_f1": 0.4589473684210526,
80
+ "eval_loss": 0.7605924010276794,
81
+ "eval_runtime": 2.5387,
82
+ "eval_samples_per_second": 176.863,
83
+ "eval_steps_per_second": 5.909,
84
+ "eval_target_accuracy": 0.40757238307349664,
85
+ "eval_target_f1": 0.3696682464454976,
86
+ "step": 264
87
+ },
88
+ {
89
+ "epoch": 5.0,
90
+ "grad_norm": 2.436269521713257,
91
+ "learning_rate": 1.5015151515151516e-05,
92
+ "loss": 0.756,
93
+ "step": 330
94
+ },
95
+ {
96
+ "epoch": 5.0,
97
+ "eval_disagreement_accuracy": 0.43429844097995546,
98
+ "eval_disagreement_f1": 0.483739837398374,
99
+ "eval_loss": 0.760177731513977,
100
+ "eval_runtime": 2.5789,
101
+ "eval_samples_per_second": 174.106,
102
+ "eval_steps_per_second": 5.816,
103
+ "eval_target_accuracy": 0.43429844097995546,
104
+ "eval_target_f1": 0.37438423645320196,
105
+ "step": 330
106
+ },
107
+ {
108
+ "epoch": 6.0,
109
+ "grad_norm": 2.8722598552703857,
110
+ "learning_rate": 1.4015151515151517e-05,
111
+ "loss": 0.7517,
112
+ "step": 396
113
+ },
114
+ {
115
+ "epoch": 6.0,
116
+ "eval_disagreement_accuracy": 0.44543429844098,
117
+ "eval_disagreement_f1": 0.5069306930693069,
118
+ "eval_loss": 0.7598492503166199,
119
+ "eval_runtime": 2.3603,
120
+ "eval_samples_per_second": 190.233,
121
+ "eval_steps_per_second": 6.355,
122
+ "eval_target_accuracy": 0.46993318485523383,
123
+ "eval_target_f1": 0.3769633507853403,
124
+ "step": 396
125
+ },
126
+ {
127
+ "epoch": 7.0,
128
+ "grad_norm": 2.5618972778320312,
129
+ "learning_rate": 1.3015151515151516e-05,
130
+ "loss": 0.757,
131
+ "step": 462
132
+ },
133
+ {
134
+ "epoch": 7.0,
135
+ "eval_disagreement_accuracy": 0.44543429844098,
136
+ "eval_disagreement_f1": 0.5049701789264414,
137
+ "eval_loss": 0.7596745491027832,
138
+ "eval_runtime": 2.3152,
139
+ "eval_samples_per_second": 193.939,
140
+ "eval_steps_per_second": 6.479,
141
+ "eval_target_accuracy": 0.48552338530066813,
142
+ "eval_target_f1": 0.36712328767123287,
143
+ "step": 462
144
+ },
145
+ {
146
+ "epoch": 8.0,
147
+ "grad_norm": 3.659677743911743,
148
+ "learning_rate": 1.2015151515151516e-05,
149
+ "loss": 0.7588,
150
+ "step": 528
151
+ },
152
+ {
153
+ "epoch": 8.0,
154
+ "eval_disagreement_accuracy": 0.44766146993318484,
155
+ "eval_disagreement_f1": 0.515625,
156
+ "eval_loss": 0.7594286203384399,
157
+ "eval_runtime": 2.3137,
158
+ "eval_samples_per_second": 194.064,
159
+ "eval_steps_per_second": 6.483,
160
+ "eval_target_accuracy": 0.49665924276169265,
161
+ "eval_target_f1": 0.35428571428571426,
162
+ "step": 528
163
+ },
164
+ {
165
+ "epoch": 9.0,
166
+ "grad_norm": 2.513981819152832,
167
+ "learning_rate": 1.1015151515151517e-05,
168
+ "loss": 0.7516,
169
+ "step": 594
170
+ },
171
+ {
172
+ "epoch": 9.0,
173
+ "eval_disagreement_accuracy": 0.44543429844098,
174
+ "eval_disagreement_f1": 0.5146198830409356,
175
+ "eval_loss": 0.7590936422348022,
176
+ "eval_runtime": 2.5525,
177
+ "eval_samples_per_second": 175.903,
178
+ "eval_steps_per_second": 5.876,
179
+ "eval_target_accuracy": 0.49665924276169265,
180
+ "eval_target_f1": 0.35795454545454547,
181
+ "step": 594
182
+ },
183
+ {
184
+ "epoch": 10.0,
185
+ "grad_norm": 2.523505926132202,
186
+ "learning_rate": 1.0015151515151516e-05,
187
+ "loss": 0.7532,
188
+ "step": 660
189
+ },
190
+ {
191
+ "epoch": 10.0,
192
+ "eval_disagreement_accuracy": 0.44097995545657015,
193
+ "eval_disagreement_f1": 0.5145067698259188,
194
+ "eval_loss": 0.7589589357376099,
195
+ "eval_runtime": 2.6703,
196
+ "eval_samples_per_second": 168.147,
197
+ "eval_steps_per_second": 5.617,
198
+ "eval_target_accuracy": 0.5256124721603563,
199
+ "eval_target_f1": 0.36795252225519287,
200
+ "step": 660
201
+ },
202
+ {
203
+ "epoch": 11.0,
204
+ "grad_norm": 2.1235299110412598,
205
+ "learning_rate": 9.015151515151516e-06,
206
+ "loss": 0.7582,
207
+ "step": 726
208
+ },
209
+ {
210
+ "epoch": 11.0,
211
+ "eval_disagreement_accuracy": 0.44097995545657015,
212
+ "eval_disagreement_f1": 0.5107212475633528,
213
+ "eval_loss": 0.758784830570221,
214
+ "eval_runtime": 2.5117,
215
+ "eval_samples_per_second": 178.761,
216
+ "eval_steps_per_second": 5.972,
217
+ "eval_target_accuracy": 0.5300668151447662,
218
+ "eval_target_f1": 0.3663663663663664,
219
+ "step": 726
220
+ },
221
+ {
222
+ "epoch": 12.0,
223
+ "grad_norm": 3.0043392181396484,
224
+ "learning_rate": 8.015151515151515e-06,
225
+ "loss": 0.754,
226
+ "step": 792
227
+ },
228
+ {
229
+ "epoch": 12.0,
230
+ "eval_disagreement_accuracy": 0.43207126948775054,
231
+ "eval_disagreement_f1": 0.5009784735812133,
232
+ "eval_loss": 0.7586497068405151,
233
+ "eval_runtime": 2.4635,
234
+ "eval_samples_per_second": 182.261,
235
+ "eval_steps_per_second": 6.089,
236
+ "eval_target_accuracy": 0.5367483296213809,
237
+ "eval_target_f1": 0.36585365853658536,
238
+ "step": 792
239
+ },
240
+ {
241
+ "epoch": 13.0,
242
+ "grad_norm": 2.731289863586426,
243
+ "learning_rate": 7.015151515151516e-06,
244
+ "loss": 0.7524,
245
+ "step": 858
246
+ },
247
+ {
248
+ "epoch": 13.0,
249
+ "eval_disagreement_accuracy": 0.4298440979955457,
250
+ "eval_disagreement_f1": 0.4980392156862745,
251
+ "eval_loss": 0.7585203051567078,
252
+ "eval_runtime": 2.4743,
253
+ "eval_samples_per_second": 181.467,
254
+ "eval_steps_per_second": 6.062,
255
+ "eval_target_accuracy": 0.5412026726057907,
256
+ "eval_target_f1": 0.35625,
257
+ "step": 858
258
+ },
259
+ {
260
+ "epoch": 14.0,
261
+ "grad_norm": 3.0117413997650146,
262
+ "learning_rate": 6.015151515151516e-06,
263
+ "loss": 0.7536,
264
+ "step": 924
265
+ },
266
+ {
267
+ "epoch": 14.0,
268
+ "eval_disagreement_accuracy": 0.43207126948775054,
269
+ "eval_disagreement_f1": 0.5029239766081871,
270
+ "eval_loss": 0.7583419680595398,
271
+ "eval_runtime": 2.434,
272
+ "eval_samples_per_second": 184.467,
273
+ "eval_steps_per_second": 6.163,
274
+ "eval_target_accuracy": 0.5389755011135857,
275
+ "eval_target_f1": 0.3591331269349845,
276
+ "step": 924
277
+ },
278
+ {
279
+ "epoch": 15.0,
280
+ "grad_norm": 2.214229106903076,
281
+ "learning_rate": 5.015151515151515e-06,
282
+ "loss": 0.7492,
283
+ "step": 990
284
+ },
285
+ {
286
+ "epoch": 15.0,
287
+ "eval_disagreement_accuracy": 0.4365256124721604,
288
+ "eval_disagreement_f1": 0.5087378640776699,
289
+ "eval_loss": 0.7582818269729614,
290
+ "eval_runtime": 2.3634,
291
+ "eval_samples_per_second": 189.979,
292
+ "eval_steps_per_second": 6.347,
293
+ "eval_target_accuracy": 0.5456570155902004,
294
+ "eval_target_f1": 0.3584905660377358,
295
+ "step": 990
296
+ },
297
+ {
298
+ "epoch": 16.0,
299
+ "grad_norm": 3.417860984802246,
300
+ "learning_rate": 4.015151515151515e-06,
301
+ "loss": 0.7534,
302
+ "step": 1056
303
+ },
304
+ {
305
+ "epoch": 16.0,
306
+ "eval_disagreement_accuracy": 0.4365256124721604,
307
+ "eval_disagreement_f1": 0.5087378640776699,
308
+ "eval_loss": 0.7582042813301086,
309
+ "eval_runtime": 2.2767,
310
+ "eval_samples_per_second": 197.212,
311
+ "eval_steps_per_second": 6.588,
312
+ "eval_target_accuracy": 0.5501113585746102,
313
+ "eval_target_f1": 0.36075949367088606,
314
+ "step": 1056
315
+ },
316
+ {
317
+ "epoch": 17.0,
318
+ "grad_norm": 3.349172592163086,
319
+ "learning_rate": 3.0151515151515155e-06,
320
+ "loss": 0.7524,
321
+ "step": 1122
322
+ },
323
+ {
324
+ "epoch": 17.0,
325
+ "eval_disagreement_accuracy": 0.4365256124721604,
326
+ "eval_disagreement_f1": 0.5087378640776699,
327
+ "eval_loss": 0.7581369280815125,
328
+ "eval_runtime": 1.3924,
329
+ "eval_samples_per_second": 322.466,
330
+ "eval_steps_per_second": 10.773,
331
+ "eval_target_accuracy": 0.5523385300668151,
332
+ "eval_target_f1": 0.3619047619047619,
333
+ "step": 1122
334
+ },
335
+ {
336
+ "epoch": 18.0,
337
+ "grad_norm": 2.1866793632507324,
338
+ "learning_rate": 2.0151515151515153e-06,
339
+ "loss": 0.7489,
340
+ "step": 1188
341
+ },
342
+ {
343
+ "epoch": 18.0,
344
+ "eval_disagreement_accuracy": 0.43875278396436523,
345
+ "eval_disagreement_f1": 0.5097276264591439,
346
+ "eval_loss": 0.7580736875534058,
347
+ "eval_runtime": 1.4344,
348
+ "eval_samples_per_second": 313.021,
349
+ "eval_steps_per_second": 10.457,
350
+ "eval_target_accuracy": 0.5523385300668151,
351
+ "eval_target_f1": 0.3659305993690852,
352
+ "step": 1188
353
+ },
354
+ {
355
+ "epoch": 19.0,
356
+ "grad_norm": 2.7388525009155273,
357
+ "learning_rate": 1.0151515151515152e-06,
358
+ "loss": 0.75,
359
+ "step": 1254
360
+ },
361
+ {
362
+ "epoch": 19.0,
363
+ "eval_disagreement_accuracy": 0.43875278396436523,
364
+ "eval_disagreement_f1": 0.5078125,
365
+ "eval_loss": 0.7580356001853943,
366
+ "eval_runtime": 1.4257,
367
+ "eval_samples_per_second": 314.942,
368
+ "eval_steps_per_second": 10.521,
369
+ "eval_target_accuracy": 0.5501113585746102,
370
+ "eval_target_f1": 0.36477987421383645,
371
+ "step": 1254
372
+ },
373
+ {
374
+ "epoch": 20.0,
375
+ "grad_norm": 1.7025479078292847,
376
+ "learning_rate": 1.5151515151515152e-08,
377
+ "loss": 0.7527,
378
+ "step": 1320
379
+ },
380
+ {
381
+ "epoch": 20.0,
382
+ "eval_disagreement_accuracy": 0.4365256124721604,
383
+ "eval_disagreement_f1": 0.50682261208577,
384
+ "eval_loss": 0.7580272555351257,
385
+ "eval_runtime": 2.4379,
386
+ "eval_samples_per_second": 184.175,
387
+ "eval_steps_per_second": 6.153,
388
+ "eval_target_accuracy": 0.5501113585746102,
389
+ "eval_target_f1": 0.36477987421383645,
390
+ "step": 1320
391
+ }
392
+ ],
393
+ "logging_steps": 500,
394
+ "max_steps": 1320,
395
+ "num_input_tokens_seen": 0,
396
+ "num_train_epochs": 20,
397
+ "save_steps": 500,
398
+ "stateful_callbacks": {
399
+ "EarlyStoppingCallback": {
400
+ "args": {
401
+ "early_stopping_patience": 5,
402
+ "early_stopping_threshold": 0.0
403
+ },
404
+ "attributes": {
405
+ "early_stopping_patience_counter": 0
406
+ }
407
+ },
408
+ "TrainerControl": {
409
+ "args": {
410
+ "should_epoch_stop": false,
411
+ "should_evaluate": false,
412
+ "should_log": false,
413
+ "should_save": true,
414
+ "should_training_stop": true
415
+ },
416
+ "attributes": {}
417
+ }
418
+ },
419
+ "total_flos": 0.0,
420
+ "train_batch_size": 32,
421
+ "trial_name": null,
422
+ "trial_params": null
423
+ }
checkpoint-1320/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f32b0cf3c27c444da01e7312faa99aadf87ec2a1877a2c08fc94b51f49f45092
3
+ size 5368
checkpoint-1320/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
disagreement_head.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e253eead250118ee8ac98a4dea71e8d6c35ae4fdc2c9b2868508466484aaecfd
3
+ size 4756
emissions.csv ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue,wue
2
+ 2025-12-23T10:37:24,codecarbon,e59d2163-0fff-414d-8954-ede47178b1c0,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,34.87844496779144,0.0019489615861835,5.587868346720616e-05,46.077869508400006,499.9338275694644,70.0,0.000427091073933,0.004817361353858,0.0006486697675039,0.005893122195295,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
3
+ 2025-12-23T10:45:23,codecarbon,5e702762-dfba-4c3d-aa89-985520bc453e,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,29.24782764725387,0.0011193953424845,3.827276869875994e-05,55.06335119414287,293.40352195132203,70.0,0.0004272781586097,0.0024084688712235,0.0005489956889922,0.0033847427188255,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
4
+ 2025-12-23T10:50:33,codecarbon,6efbaf58-131c-4e69-ac10-816730f3d325,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,53.75047469884157,0.002518285342056,4.6851406544142e-05,54.33666809,300.92162948348124,70.0,0.0006771430260794,0.0059323114125078,0.0010051457405866,0.0076146001791739,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
5
+ 2025-12-23T10:54:31,codecarbon,9bbec82e-6839-416a-bf00-bd31a6c0306a,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,78.92634090408683,0.0036728623424934,4.653531761920643e-05,56.98526540500001,285.2102031950449,70.0,0.0009773900821322,0.0086549574795071,0.0014733749647831,0.0111057225264226,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
6
+ 2025-12-23T13:06:06,codecarbon,1bb4fc43-c786-4d22-8d6e-89cf08c6d176,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,87.10759136267006,0.0041644447474116,4.780805762465716e-05,46.028408956,413.4656649634909,70.0,0.001075173137822,0.0098823259614277,0.0016346330115261,0.012592132110776,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
7
+ 2025-12-23T13:09:05,codecarbon,b678ec55-cd27-4b0b-8d5e-121ecf824ceb,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,90.86386202462018,0.0043435641698376,4.780298870260101e-05,46.03451305480001,408.2291856219036,70.0,0.0012153256099812,0.0102144456715507,0.0017039685507046,0.0131337398322365,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
8
+ 2025-12-23T13:11:23,codecarbon,f52fe343-2f04-45a9-9769-501fd1e6d042,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,73.48588255792856,0.0051086886215025,6.951932049636028e-05,46.04403444123077,422.4104431703749,70.0,0.0063043097589664,0.0077633995440606,0.0013795558035063,0.0154472651065334,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
9
+ 2025-12-23T13:14:50,codecarbon,183d200b-b729-4e8f-bf76-5f1d6e8c0595,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,90.32180859148502,0.0042699475308356,4.72748231841566e-05,46.02302298357143,401.4595494831764,70.0,0.0012165457781866,0.0099975860536432,0.0016970115919804,0.0129111434238102,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
10
+ 2025-12-23T13:27:39,codecarbon,4fd0d34d-568d-4a28-b14f-39c3322f064e,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,78.76237196847796,0.003542118939543,4.497222278882894e-05,46.015827220000006,372.2421613754595,70.0,0.0015970928381495,0.0076487497301087,0.0014645479745697,0.010710390542828,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
11
+ 2025-12-23T13:43:19,codecarbon,a1f23191-569d-45ce-8264-5fa452a07e27,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,85.39552418142557,0.0038856624084769,4.55019445775838e-05,58.94266034440001,349.50142761638784,70.0,0.0011055546927818,0.0090425739007145,0.0016010426293489,0.0117491712228452,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
12
+ 2025-12-23T13:48:33,codecarbon,d3fdc4f9-c16e-4ce9-b2e0-73e7b7c3b7c9,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,122.43101214058696,0.0073829399404671,6.030285800455036e-05,46.004408272,373.0607006585514,70.0,0.0066711538559792,0.0133689323618,0.0022838875014231,0.0223239737192024,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
13
+ 2025-12-23T13:52:57,codecarbon,bdf7d142-2b97-4717-a494-0cffbd47514a,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,135.11052935384214,0.0061875869143755,4.579648191719278e-05,72.05785445542858,384.8540449606335,70.0,0.0017598142506422,0.0144188557017344,0.0025308859664895,0.0187095559188661,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
14
+ 2025-12-23T14:01:06,codecarbon,4d889e70-341e-46b1-a9d8-58efa15c73c3,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,158.87868946418166,0.0078110371447338,4.916352955249405e-05,57.68265994525001,393.79721948167287,70.0,0.002120306611315,0.0185181150922488,0.0029799998662744,0.0236184215698384,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
15
+ 2025-12-23T14:08:05,codecarbon,3963c8fd-3318-41a2-a258-72713ee4dd2c,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,173.26289486512542,0.0085620719545119,4.941665069821797e-05,57.20471019325,407.3281443780772,70.0,0.0023902679408817,0.0202501967556827,0.0032488789572794,0.0258893436538439,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
16
+ 2025-12-23T14:13:26,codecarbon,004d04ba-1351-46ec-b263-daaa47d4f84b,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,116.47748952172697,0.0057240659081765,4.914310852406244e-05,75.50763547900003,439.5689290863244,70.0,0.0016256664082666,0.0134967374640524,0.0021855923908917,0.0173079962632108,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
17
+ 2025-12-23T14:17:32,codecarbon,4d350e68-d789-40c2-aa37-16b5f34d9c67,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,103.99149509333074,0.0051375816627462,4.9403863826895744e-05,46.03471617700001,422.1815601723172,70.0,0.0015032616473362,0.0120783785515925,0.0019529894938795,0.0155346296928084,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
18
+ 2025-12-23T14:27:19,codecarbon,196ec1ba-e035-4698-9dc8-0009fce2a348,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,118.47603076696396,0.0060794534132704,5.1313783673496215e-05,59.158979620428575,448.9541384600159,70.0,0.00156146260037,0.014596305288137,0.0022248221527681,0.0183825900412752,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
19
+ 2025-12-23T14:37:19,codecarbon,7ede0045-7707-4e62-a834-4e1ae286be52,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,116.22501818835735,0.0061103761872712,5.257367374525726e-05,171.3733636550909,459.1558846486712,70.0,0.0020884182770109,0.0142068860877202,0.0021807876147536,0.0184760919794849,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
20
+ 2025-12-23T14:40:05,codecarbon,ce6768f5-b6bd-49fe-8869-e3d3aea282d2,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,84.87842439860106,0.0045404640655211,5.349373645531565e-05,46.01654973280001,458.0816977573463,70.0,0.0011667562917286,0.0109709846100969,0.0015913698980739,0.0137291107998996,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
21
+ 2025-12-23T14:43:49,codecarbon,e30d84b7-ea5d-4257-a552-66f093dbe77c,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,86.29132142663002,0.0045306716473621,5.250437207888144e-05,50.72825873472728,442.15714611405065,70.0,0.00112513022884,0.0109655612724566,0.0016088097274303,0.013699501228727,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
22
+ 2025-12-23T14:48:27,codecarbon,7ff0d12b-77b6-475e-8c8f-4d888e435630,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,161.26207901723683,0.0092673562560161,5.74676719566885e-05,55.4797791415,446.2372760760207,70.0,0.0020828271572564,0.0229160391661622,0.0030230624981638,0.0280219288215825,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
23
+ 2025-12-23T14:53:47,codecarbon,a5bdc46f-62ae-43ba-be34-91a5f1c25ca2,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,180.55386474728584,0.0106826993507286,5.916627354214084e-05,46.01514745960001,546.5057576879855,70.0,0.0024497297806838,0.0264590814450258,0.0033927246771701,0.0323015359028798,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
24
+ 2025-12-23T15:01:30,codecarbon,1df0c665-09ee-4c83-8516-00947f8885b0,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,323.31841713003814,0.018536361231211,5.733159711639867e-05,46.021023541000005,531.5948688304567,70.0,0.0043063682903444,0.0456719334819268,0.006070541142874,0.0560488429151454,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
25
+ 2025-12-23T15:07:47,codecarbon,6f58e877-8c0d-4409-9afd-67d2daf4029a,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,317.0770432204008,0.0181521399045213,5.724835743439101e-05,46.022119192000005,539.2299087797594,70.0,0.0047042353243337,0.0442380245570603,0.0059448037332906,0.0548870636146848,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
26
+ 2025-12-23T15:16:14,codecarbon,49fa3080-c968-4e92-a753-d23acc7059b4,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,303.3238886166364,0.017733253728664317,5.846309636059341e-05,46.188218752000004,533.7191723149965,70.0,0.005532440644447099,0.04240604836925854,0.0056819783592958846,0.05362046737300152,0.0,Italy,ITA,tuscany,,,Linux-5.4.0-216-generic-x86_64-with-glibc2.31,3.10.18,3.0.8,96,Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz,2,2 x NVIDIA RTX A6000,10.4087,43.7089,1007.494571685791,machine,N,1.0,0.0
encoder/config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "BertModel"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "classifier_dropout": null,
7
+ "dtype": "float32",
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "transformers_version": "4.57.1",
22
+ "type_vocab_size": 2,
23
+ "use_cache": true,
24
+ "vocab_size": 30522
25
+ }
encoder/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e47716a979def3ee4331621abb95a2a07619cf6428ca798c051201cbbc0ff89
3
+ size 437951328
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:efd2c4b92912015f3863fc7573022e82249c4d3480e070584738dbb7fd0dbb3c
3
+ size 540803928
multitask_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "multitask_roberta",
3
+ "base_model": "google-bert/bert-base-uncased",
4
+ "task_labels_map": {
5
+ "disagreement": 1,
6
+ "target": 1
7
+ },
8
+ "hidden_size": 768,
9
+ "dropout_prob": 0.4
10
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
target_head.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c273742707858fdd9c24be5b665a85e05e0ba519e4527cf554391cf7be61a366
3
+ size 4656
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer/sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
3
+ size 5069051
tokenizer/special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "BertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
tokenizer/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,339 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "+ا",
5
+ "lstrip": false,
6
+ "normalized": true,
7
+ "rstrip": false,
8
+ "single_word": true,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "+ك",
13
+ "lstrip": false,
14
+ "normalized": true,
15
+ "rstrip": false,
16
+ "single_word": true,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "ب+",
21
+ "lstrip": false,
22
+ "normalized": true,
23
+ "rstrip": false,
24
+ "single_word": true,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "+هم",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": true,
33
+ "special": true
34
+ },
35
+ "4": {
36
+ "content": "+ات",
37
+ "lstrip": false,
38
+ "normalized": true,
39
+ "rstrip": false,
40
+ "single_word": true,
41
+ "special": true
42
+ },
43
+ "5": {
44
+ "content": "+ي",
45
+ "lstrip": false,
46
+ "normalized": true,
47
+ "rstrip": false,
48
+ "single_word": true,
49
+ "special": true
50
+ },
51
+ "6": {
52
+ "content": "ل+",
53
+ "lstrip": false,
54
+ "normalized": true,
55
+ "rstrip": false,
56
+ "single_word": true,
57
+ "special": true
58
+ },
59
+ "7": {
60
+ "content": "+هما",
61
+ "lstrip": false,
62
+ "normalized": true,
63
+ "rstrip": false,
64
+ "single_word": true,
65
+ "special": true
66
+ },
67
+ "8": {
68
+ "content": "+نا",
69
+ "lstrip": false,
70
+ "normalized": true,
71
+ "rstrip": false,
72
+ "single_word": true,
73
+ "special": true
74
+ },
75
+ "9": {
76
+ "content": "+ن",
77
+ "lstrip": false,
78
+ "normalized": true,
79
+ "rstrip": false,
80
+ "single_word": true,
81
+ "special": true
82
+ },
83
+ "10": {
84
+ "content": "+ها",
85
+ "lstrip": false,
86
+ "normalized": true,
87
+ "rstrip": false,
88
+ "single_word": true,
89
+ "special": true
90
+ },
91
+ "11": {
92
+ "content": "+كما",
93
+ "lstrip": false,
94
+ "normalized": true,
95
+ "rstrip": false,
96
+ "single_word": true,
97
+ "special": true
98
+ },
99
+ "12": {
100
+ "content": "+ة",
101
+ "lstrip": false,
102
+ "normalized": true,
103
+ "rstrip": false,
104
+ "single_word": true,
105
+ "special": true
106
+ },
107
+ "13": {
108
+ "content": "ف+",
109
+ "lstrip": false,
110
+ "normalized": true,
111
+ "rstrip": false,
112
+ "single_word": true,
113
+ "special": true
114
+ },
115
+ "14": {
116
+ "content": "+كم",
117
+ "lstrip": false,
118
+ "normalized": true,
119
+ "rstrip": false,
120
+ "single_word": true,
121
+ "special": true
122
+ },
123
+ "15": {
124
+ "content": "+كن",
125
+ "lstrip": false,
126
+ "normalized": true,
127
+ "rstrip": false,
128
+ "single_word": true,
129
+ "special": true
130
+ },
131
+ "16": {
132
+ "content": "+ت",
133
+ "lstrip": false,
134
+ "normalized": true,
135
+ "rstrip": false,
136
+ "single_word": true,
137
+ "special": true
138
+ },
139
+ "17": {
140
+ "content": "[بريد]",
141
+ "lstrip": false,
142
+ "normalized": true,
143
+ "rstrip": false,
144
+ "single_word": true,
145
+ "special": true
146
+ },
147
+ "18": {
148
+ "content": "[مستخدم]",
149
+ "lstrip": false,
150
+ "normalized": true,
151
+ "rstrip": false,
152
+ "single_word": true,
153
+ "special": true
154
+ },
155
+ "19": {
156
+ "content": "لل+",
157
+ "lstrip": false,
158
+ "normalized": true,
159
+ "rstrip": false,
160
+ "single_word": true,
161
+ "special": true
162
+ },
163
+ "20": {
164
+ "content": "ال+",
165
+ "lstrip": false,
166
+ "normalized": true,
167
+ "rstrip": false,
168
+ "single_word": true,
169
+ "special": true
170
+ },
171
+ "21": {
172
+ "content": "[رابط]",
173
+ "lstrip": false,
174
+ "normalized": true,
175
+ "rstrip": false,
176
+ "single_word": true,
177
+ "special": true
178
+ },
179
+ "22": {
180
+ "content": "س+",
181
+ "lstrip": false,
182
+ "normalized": true,
183
+ "rstrip": false,
184
+ "single_word": true,
185
+ "special": true
186
+ },
187
+ "23": {
188
+ "content": "+ان",
189
+ "lstrip": false,
190
+ "normalized": true,
191
+ "rstrip": false,
192
+ "single_word": true,
193
+ "special": true
194
+ },
195
+ "24": {
196
+ "content": "+وا",
197
+ "lstrip": false,
198
+ "normalized": true,
199
+ "rstrip": false,
200
+ "single_word": true,
201
+ "special": true
202
+ },
203
+ "25": {
204
+ "content": "+ه",
205
+ "lstrip": false,
206
+ "normalized": true,
207
+ "rstrip": false,
208
+ "single_word": true,
209
+ "special": true
210
+ },
211
+ "26": {
212
+ "content": "+ون",
213
+ "lstrip": false,
214
+ "normalized": true,
215
+ "rstrip": false,
216
+ "single_word": true,
217
+ "special": true
218
+ },
219
+ "27": {
220
+ "content": "+هن",
221
+ "lstrip": false,
222
+ "normalized": true,
223
+ "rstrip": false,
224
+ "single_word": true,
225
+ "special": true
226
+ },
227
+ "28": {
228
+ "content": "+ين",
229
+ "lstrip": false,
230
+ "normalized": true,
231
+ "rstrip": false,
232
+ "single_word": true,
233
+ "special": true
234
+ },
235
+ "29": {
236
+ "content": "��+",
237
+ "lstrip": false,
238
+ "normalized": true,
239
+ "rstrip": false,
240
+ "single_word": true,
241
+ "special": true
242
+ },
243
+ "30": {
244
+ "content": "ك+",
245
+ "lstrip": false,
246
+ "normalized": true,
247
+ "rstrip": false,
248
+ "single_word": true,
249
+ "special": true
250
+ },
251
+ "31": {
252
+ "content": "[PAD]",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "32": {
260
+ "content": "[UNK]",
261
+ "lstrip": false,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "33": {
268
+ "content": "[CLS]",
269
+ "lstrip": false,
270
+ "normalized": false,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": true
274
+ },
275
+ "34": {
276
+ "content": "[SEP]",
277
+ "lstrip": false,
278
+ "normalized": false,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": true
282
+ },
283
+ "35": {
284
+ "content": "[MASK]",
285
+ "lstrip": false,
286
+ "normalized": false,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": true
290
+ }
291
+ },
292
+ "clean_up_tokenization_spaces": false,
293
+ "cls_token": "[CLS]",
294
+ "do_basic_tokenize": true,
295
+ "do_lower_case": false,
296
+ "extra_special_tokens": {},
297
+ "mask_token": "[MASK]",
298
+ "max_len": 512,
299
+ "model_max_length": 512,
300
+ "never_split": [
301
+ "+ك",
302
+ "+كما",
303
+ "ك+",
304
+ "+وا",
305
+ "+ين",
306
+ "و+",
307
+ "+كن",
308
+ "+ان",
309
+ "+هم",
310
+ "+ة",
311
+ "[بريد]",
312
+ "لل+",
313
+ "+ي",
314
+ "+ت",
315
+ "+ن",
316
+ "س+",
317
+ "ل+",
318
+ "[مستخدم]",
319
+ "+كم",
320
+ "+ا",
321
+ "ب+",
322
+ "ف+",
323
+ "+نا",
324
+ "+ها",
325
+ "+ون",
326
+ "+هما",
327
+ "ال+",
328
+ "+ه",
329
+ "+هن",
330
+ "+ات",
331
+ "[رابط]"
332
+ ],
333
+ "pad_token": "[PAD]",
334
+ "sep_token": "[SEP]",
335
+ "strip_accents": null,
336
+ "tokenize_chinese_chars": true,
337
+ "tokenizer_class": "BertTokenizer",
338
+ "unk_token": "[UNK]"
339
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26bfbe33c29932d840225402890b9bde95c580f54a384ebd546694f8ac544323
3
+ size 5432
vocab.txt ADDED
The diff for this file is too large to render. See raw diff