Hugging Face's logo

LucileFavero
/
out_base_W

Text Classification
Transformers
Safetensors
longformer
Generated from Trainer
Model card Files
xet
 Community
LucileFavero commited on
Commit
42f7f77
·
verified ·
1 Parent(s): 5f75113

LucileFavero/LF_Word

Browse files
Files changed (11) hide show
  1. README.md +71 -0
  2. config.json +52 -0
  3. eval_W.csv +2 -0
  4. merges.txt +0 -0
  5. model.safetensors +3 -0
  6. pred_W.csv +358 -0
  7. special_tokens_map.json +15 -0
  8. tokenizer.json +0 -0
  9. tokenizer_config.json +58 -0
  10. training_args.bin +3 -0
  11. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: allenai/longformer-base-4096
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - accuracy
9
+ - f1
10
+ model-index:
11
+ - name: out_base_W
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # out_base_W
19
+
20
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the None dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 1.0009
23
+ - Accuracy: 0.6471
24
+ - F1: 0.6242
25
+ - Cohen Kappa: 0.4456
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 2e-05
45
+ - train_batch_size: 1
46
+ - eval_batch_size: 1
47
+ - seed: 42
48
+ - gradient_accumulation_steps: 8
49
+ - total_train_batch_size: 8
50
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
51
+ - lr_scheduler_type: linear
52
+ - num_epochs: 5
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Cohen Kappa |
58
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:-----------:|
59
+ | 0.7956 | 1.0 | 134 | 0.7777 | 0.6499 | 0.5946 | 0.4277 |
60
+ | 0.7351 | 2.0 | 268 | 0.7918 | 0.5854 | 0.5813 | 0.3401 |
61
+ | 0.5332 | 3.0 | 402 | 0.7313 | 0.6471 | 0.6191 | 0.4461 |
62
+ | 0.5739 | 4.0 | 536 | 0.9479 | 0.6359 | 0.5965 | 0.4022 |
63
+ | 0.3323 | 5.0 | 670 | 1.0009 | 0.6471 | 0.6242 | 0.4456 |
64
+
65
+
66
+ ### Framework versions
67
+
68
+ - Transformers 4.57.3
69
+ - Pytorch 2.9.0+cu126
70
+ - Datasets 4.0.0
71
+ - Tokenizers 0.22.2
config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LongformerForSequenceClassification"
4
+ ],
5
+ "attention_mode": "longformer",
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "attention_window": [
8
+ 512,
9
+ 512,
10
+ 512,
11
+ 512,
12
+ 512,
13
+ 512,
14
+ 512,
15
+ 512,
16
+ 512,
17
+ 512,
18
+ 512,
19
+ 512
20
+ ],
21
+ "bos_token_id": 0,
22
+ "dtype": "float32",
23
+ "eos_token_id": 2,
24
+ "gradient_checkpointing": false,
25
+ "hidden_act": "gelu",
26
+ "hidden_dropout_prob": 0.1,
27
+ "hidden_size": 768,
28
+ "id2label": {
29
+ "0": "Fair",
30
+ "1": "Strong",
31
+ "2": "Weak"
32
+ },
33
+ "ignore_attention_mask": false,
34
+ "initializer_range": 0.02,
35
+ "intermediate_size": 3072,
36
+ "label2id": {
37
+ "Fair": 0,
38
+ "Strong": 1,
39
+ "Weak": 2
40
+ },
41
+ "layer_norm_eps": 1e-05,
42
+ "max_position_embeddings": 4098,
43
+ "model_type": "longformer",
44
+ "num_attention_heads": 12,
45
+ "num_hidden_layers": 12,
46
+ "onnx_export": false,
47
+ "pad_token_id": 1,
48
+ "sep_token_id": 2,
49
+ "transformers_version": "4.57.3",
50
+ "type_vocab_size": 1,
51
+ "vocab_size": 50265
52
+ }
eval_W.csv ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ qwk,cor,exact
2
+ 0.4883778985909242,0.47916077821084685,0.6778711484593838
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d4120e43c4c534f4de6d3680008ca5ca2c9cacde4f7bfddd091db9d9ab51766
3
+ size 594681260
pred_W.csv ADDED
@@ -0,0 +1,358 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ W_score
2
+ Weak
3
+ Fair
4
+ Weak
5
+ Strong
6
+ Fair
7
+ Weak
8
+ Fair
9
+ Fair
10
+ Weak
11
+ Strong
12
+ Weak
13
+ Fair
14
+ Weak
15
+ Weak
16
+ Fair
17
+ Weak
18
+ Weak
19
+ Weak
20
+ Weak
21
+ Fair
22
+ Weak
23
+ Weak
24
+ Fair
25
+ Fair
26
+ Weak
27
+ Fair
28
+ Weak
29
+ Fair
30
+ Fair
31
+ Weak
32
+ Fair
33
+ Weak
34
+ Weak
35
+ Weak
36
+ Weak
37
+ Weak
38
+ Weak
39
+ Weak
40
+ Weak
41
+ Fair
42
+ Fair
43
+ Weak
44
+ Fair
45
+ Weak
46
+ Fair
47
+ Weak
48
+ Fair
49
+ Weak
50
+ Weak
51
+ Fair
52
+ Strong
53
+ Strong
54
+ Fair
55
+ Fair
56
+ Weak
57
+ Weak
58
+ Fair
59
+ Fair
60
+ Weak
61
+ Weak
62
+ Weak
63
+ Strong
64
+ Fair
65
+ Weak
66
+ Fair
67
+ Weak
68
+ Strong
69
+ Weak
70
+ Fair
71
+ Weak
72
+ Weak
73
+ Fair
74
+ Weak
75
+ Weak
76
+ Fair
77
+ Weak
78
+ Weak
79
+ Fair
80
+ Weak
81
+ Fair
82
+ Weak
83
+ Weak
84
+ Weak
85
+ Fair
86
+ Weak
87
+ Weak
88
+ Weak
89
+ Strong
90
+ Weak
91
+ Weak
92
+ Fair
93
+ Fair
94
+ Fair
95
+ Weak
96
+ Fair
97
+ Weak
98
+ Weak
99
+ Weak
100
+ Fair
101
+ Fair
102
+ Fair
103
+ Weak
104
+ Weak
105
+ Strong
106
+ Weak
107
+ Strong
108
+ Weak
109
+ Weak
110
+ Weak
111
+ Fair
112
+ Fair
113
+ Fair
114
+ Fair
115
+ Fair
116
+ Fair
117
+ Fair
118
+ Weak
119
+ Weak
120
+ Weak
121
+ Fair
122
+ Fair
123
+ Weak
124
+ Weak
125
+ Weak
126
+ Weak
127
+ Strong
128
+ Weak
129
+ Weak
130
+ Strong
131
+ Weak
132
+ Weak
133
+ Fair
134
+ Strong
135
+ Strong
136
+ Weak
137
+ Weak
138
+ Weak
139
+ Weak
140
+ Fair
141
+ Weak
142
+ Strong
143
+ Weak
144
+ Weak
145
+ Weak
146
+ Weak
147
+ Strong
148
+ Fair
149
+ Fair
150
+ Strong
151
+ Weak
152
+ Weak
153
+ Weak
154
+ Weak
155
+ Weak
156
+ Strong
157
+ Weak
158
+ Fair
159
+ Weak
160
+ Fair
161
+ Weak
162
+ Weak
163
+ Weak
164
+ Fair
165
+ Fair
166
+ Fair
167
+ Fair
168
+ Weak
169
+ Weak
170
+ Fair
171
+ Fair
172
+ Weak
173
+ Weak
174
+ Weak
175
+ Weak
176
+ Fair
177
+ Fair
178
+ Fair
179
+ Weak
180
+ Fair
181
+ Fair
182
+ Fair
183
+ Weak
184
+ Fair
185
+ Strong
186
+ Weak
187
+ Weak
188
+ Fair
189
+ Fair
190
+ Weak
191
+ Weak
192
+ Weak
193
+ Fair
194
+ Weak
195
+ Fair
196
+ Fair
197
+ Fair
198
+ Weak
199
+ Weak
200
+ Weak
201
+ Strong
202
+ Fair
203
+ Weak
204
+ Weak
205
+ Weak
206
+ Fair
207
+ Fair
208
+ Weak
209
+ Fair
210
+ Weak
211
+ Fair
212
+ Weak
213
+ Weak
214
+ Weak
215
+ Weak
216
+ Weak
217
+ Fair
218
+ Fair
219
+ Fair
220
+ Fair
221
+ Weak
222
+ Weak
223
+ Fair
224
+ Fair
225
+ Weak
226
+ Fair
227
+ Fair
228
+ Fair
229
+ Fair
230
+ Weak
231
+ Weak
232
+ Weak
233
+ Weak
234
+ Strong
235
+ Fair
236
+ Fair
237
+ Weak
238
+ Weak
239
+ Fair
240
+ Fair
241
+ Strong
242
+ Weak
243
+ Weak
244
+ Weak
245
+ Weak
246
+ Weak
247
+ Weak
248
+ Fair
249
+ Weak
250
+ Weak
251
+ Strong
252
+ Strong
253
+ Weak
254
+ Weak
255
+ Strong
256
+ Fair
257
+ Fair
258
+ Weak
259
+ Weak
260
+ Weak
261
+ Weak
262
+ Fair
263
+ Fair
264
+ Weak
265
+ Weak
266
+ Weak
267
+ Fair
268
+ Weak
269
+ Fair
270
+ Weak
271
+ Fair
272
+ Weak
273
+ Fair
274
+ Strong
275
+ Fair
276
+ Weak
277
+ Fair
278
+ Strong
279
+ Fair
280
+ Fair
281
+ Strong
282
+ Weak
283
+ Fair
284
+ Fair
285
+ Fair
286
+ Fair
287
+ Weak
288
+ Weak
289
+ Fair
290
+ Strong
291
+ Fair
292
+ Fair
293
+ Weak
294
+ Weak
295
+ Weak
296
+ Weak
297
+ Weak
298
+ Weak
299
+ Fair
300
+ Weak
301
+ Weak
302
+ Fair
303
+ Fair
304
+ Weak
305
+ Weak
306
+ Strong
307
+ Fair
308
+ Weak
309
+ Fair
310
+ Weak
311
+ Weak
312
+ Weak
313
+ Fair
314
+ Weak
315
+ Fair
316
+ Fair
317
+ Fair
318
+ Fair
319
+ Fair
320
+ Weak
321
+ Weak
322
+ Weak
323
+ Weak
324
+ Weak
325
+ Weak
326
+ Weak
327
+ Weak
328
+ Weak
329
+ Weak
330
+ Weak
331
+ Fair
332
+ Strong
333
+ Fair
334
+ Weak
335
+ Weak
336
+ Fair
337
+ Fair
338
+ Weak
339
+ Weak
340
+ Weak
341
+ Fair
342
+ Weak
343
+ Weak
344
+ Fair
345
+ Weak
346
+ Weak
347
+ Weak
348
+ Fair
349
+ Fair
350
+ Weak
351
+ Fair
352
+ Fair
353
+ Fair
354
+ Weak
355
+ Weak
356
+ Weak
357
+ Weak
358
+ Strong
special_tokens_map.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<s>",
3
+ "cls_token": "<s>",
4
+ "eos_token": "</s>",
5
+ "mask_token": {
6
+ "content": "<mask>",
7
+ "lstrip": true,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "pad_token": "<pad>",
13
+ "sep_token": "</s>",
14
+ "unk_token": "<unk>"
15
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<s>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<pad>",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<unk>",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50264": {
37
+ "content": "<mask>",
38
+ "lstrip": true,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ }
44
+ },
45
+ "bos_token": "<s>",
46
+ "clean_up_tokenization_spaces": false,
47
+ "cls_token": "<s>",
48
+ "eos_token": "</s>",
49
+ "errors": "replace",
50
+ "extra_special_tokens": {},
51
+ "mask_token": "<mask>",
52
+ "model_max_length": 1000000000000000019884624838656,
53
+ "pad_token": "<pad>",
54
+ "sep_token": "</s>",
55
+ "tokenizer_class": "LongformerTokenizer",
56
+ "trim_offsets": true,
57
+ "unk_token": "<unk>"
58
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaf188a6dfeed7a0eebeafb070239016194fe81b3ffa17bf2565d9147a24ab44
3
+ size 5841
vocab.json ADDED
The diff for this file is too large to render. See raw diff