Hugging Face's logo

LucileFavero
/
out_base_V

Text Classification
Transformers
Safetensors
longformer
Generated from Trainer
Model card Files
xet
Community
LucileFavero commited on
Commit
ac37219
·
verified ·
1 Parent(s): 9432d5b

LucileFavero/LF_Convention

Browse files
Files changed (11) hide show
  1. README.md +71 -0
  2. config.json +52 -0
  3. eval_V.csv +2 -0
  4. merges.txt +0 -0
  5. model.safetensors +3 -0
  6. pred_V.csv +358 -0
  7. special_tokens_map.json +15 -0
  8. tokenizer.json +0 -0
  9. tokenizer_config.json +58 -0
  10. training_args.bin +3 -0
  11. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: allenai/longformer-base-4096
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - accuracy
9
+ - f1
10
+ model-index:
11
+ - name: out_base_V
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # out_base_V
19
+
20
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the None dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 0.9725
23
+ - Accuracy: 0.6443
24
+ - F1: 0.6330
25
+ - Cohen Kappa: 0.4632
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 2e-05
45
+ - train_batch_size: 1
46
+ - eval_batch_size: 1
47
+ - seed: 42
48
+ - gradient_accumulation_steps: 8
49
+ - total_train_batch_size: 8
50
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
51
+ - lr_scheduler_type: linear
52
+ - num_epochs: 5
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Cohen Kappa |
58
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:-----------:|
59
+ | 0.7313 | 1.0 | 134 | 0.7851 | 0.6667 | 0.5973 | 0.5339 |
60
+ | 0.9013 | 2.0 | 268 | 0.7168 | 0.6723 | 0.6588 | 0.4886 |
61
+ | 0.5642 | 3.0 | 402 | 0.7978 | 0.6555 | 0.6411 | 0.4756 |
62
+ | 0.4911 | 4.0 | 536 | 0.9718 | 0.6611 | 0.6177 | 0.5085 |
63
+ | 0.32 | 5.0 | 670 | 0.9725 | 0.6443 | 0.6330 | 0.4632 |
64
+
65
+
66
+ ### Framework versions
67
+
68
+ - Transformers 4.57.3
69
+ - Pytorch 2.9.0+cu126
70
+ - Datasets 4.0.0
71
+ - Tokenizers 0.22.2
config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LongformerForSequenceClassification"
4
+ ],
5
+ "attention_mode": "longformer",
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "attention_window": [
8
+ 512,
9
+ 512,
10
+ 512,
11
+ 512,
12
+ 512,
13
+ 512,
14
+ 512,
15
+ 512,
16
+ 512,
17
+ 512,
18
+ 512,
19
+ 512
20
+ ],
21
+ "bos_token_id": 0,
22
+ "dtype": "float32",
23
+ "eos_token_id": 2,
24
+ "gradient_checkpointing": false,
25
+ "hidden_act": "gelu",
26
+ "hidden_dropout_prob": 0.1,
27
+ "hidden_size": 768,
28
+ "id2label": {
29
+ "0": "Fair",
30
+ "1": "Strong",
31
+ "2": "Weak"
32
+ },
33
+ "ignore_attention_mask": false,
34
+ "initializer_range": 0.02,
35
+ "intermediate_size": 3072,
36
+ "label2id": {
37
+ "Fair": 0,
38
+ "Strong": 1,
39
+ "Weak": 2
40
+ },
41
+ "layer_norm_eps": 1e-05,
42
+ "max_position_embeddings": 4098,
43
+ "model_type": "longformer",
44
+ "num_attention_heads": 12,
45
+ "num_hidden_layers": 12,
46
+ "onnx_export": false,
47
+ "pad_token_id": 1,
48
+ "sep_token_id": 2,
49
+ "transformers_version": "4.57.3",
50
+ "type_vocab_size": 1,
51
+ "vocab_size": 50265
52
+ }
eval_V.csv ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ qwk,cor,exact
2
+ 0.44538471930061563,0.40446135888046936,0.6106442577030813
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:771750ccbf2e6a415bc741ea94a2f2b8012673cb3bf051cc656ea10582b377cc
3
+ size 594681260
pred_V.csv ADDED
@@ -0,0 +1,358 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ V_score
2
+ Weak
3
+ Fair
4
+ Weak
5
+ Fair
6
+ Strong
7
+ Fair
8
+ Fair
9
+ Fair
10
+ Fair
11
+ Strong
12
+ Weak
13
+ Fair
14
+ Weak
15
+ Weak
16
+ Strong
17
+ Weak
18
+ Weak
19
+ Fair
20
+ Fair
21
+ Strong
22
+ Weak
23
+ Weak
24
+ Fair
25
+ Fair
26
+ Fair
27
+ Strong
28
+ Fair
29
+ Fair
30
+ Fair
31
+ Fair
32
+ Strong
33
+ Weak
34
+ Fair
35
+ Weak
36
+ Fair
37
+ Weak
38
+ Fair
39
+ Weak
40
+ Weak
41
+ Fair
42
+ Strong
43
+ Fair
44
+ Fair
45
+ Weak
46
+ Fair
47
+ Weak
48
+ Fair
49
+ Fair
50
+ Fair
51
+ Fair
52
+ Strong
53
+ Strong
54
+ Fair
55
+ Fair
56
+ Weak
57
+ Weak
58
+ Strong
59
+ Strong
60
+ Weak
61
+ Fair
62
+ Fair
63
+ Strong
64
+ Fair
65
+ Weak
66
+ Strong
67
+ Weak
68
+ Strong
69
+ Weak
70
+ Fair
71
+ Weak
72
+ Weak
73
+ Strong
74
+ Weak
75
+ Weak
76
+ Fair
77
+ Fair
78
+ Weak
79
+ Strong
80
+ Weak
81
+ Fair
82
+ Fair
83
+ Weak
84
+ Weak
85
+ Fair
86
+ Weak
87
+ Weak
88
+ Weak
89
+ Strong
90
+ Weak
91
+ Weak
92
+ Fair
93
+ Fair
94
+ Fair
95
+ Fair
96
+ Fair
97
+ Weak
98
+ Weak
99
+ Fair
100
+ Fair
101
+ Fair
102
+ Strong
103
+ Weak
104
+ Weak
105
+ Strong
106
+ Weak
107
+ Strong
108
+ Weak
109
+ Weak
110
+ Weak
111
+ Fair
112
+ Strong
113
+ Fair
114
+ Fair
115
+ Fair
116
+ Strong
117
+ Strong
118
+ Weak
119
+ Weak
120
+ Weak
121
+ Fair
122
+ Fair
123
+ Weak
124
+ Weak
125
+ Fair
126
+ Weak
127
+ Strong
128
+ Weak
129
+ Weak
130
+ Strong
131
+ Weak
132
+ Weak
133
+ Fair
134
+ Strong
135
+ Strong
136
+ Weak
137
+ Fair
138
+ Fair
139
+ Weak
140
+ Fair
141
+ Weak
142
+ Strong
143
+ Weak
144
+ Weak
145
+ Weak
146
+ Fair
147
+ Strong
148
+ Fair
149
+ Strong
150
+ Strong
151
+ Fair
152
+ Fair
153
+ Weak
154
+ Fair
155
+ Weak
156
+ Strong
157
+ Weak
158
+ Fair
159
+ Weak
160
+ Fair
161
+ Weak
162
+ Fair
163
+ Fair
164
+ Fair
165
+ Fair
166
+ Strong
167
+ Fair
168
+ Fair
169
+ Fair
170
+ Fair
171
+ Fair
172
+ Weak
173
+ Weak
174
+ Weak
175
+ Weak
176
+ Strong
177
+ Strong
178
+ Fair
179
+ Fair
180
+ Strong
181
+ Fair
182
+ Strong
183
+ Weak
184
+ Fair
185
+ Strong
186
+ Weak
187
+ Fair
188
+ Fair
189
+ Fair
190
+ Weak
191
+ Fair
192
+ Weak
193
+ Fair
194
+ Weak
195
+ Fair
196
+ Fair
197
+ Fair
198
+ Weak
199
+ Weak
200
+ Weak
201
+ Strong
202
+ Fair
203
+ Fair
204
+ Fair
205
+ Weak
206
+ Fair
207
+ Fair
208
+ Fair
209
+ Strong
210
+ Weak
211
+ Strong
212
+ Weak
213
+ Weak
214
+ Weak
215
+ Weak
216
+ Weak
217
+ Fair
218
+ Fair
219
+ Fair
220
+ Fair
221
+ Weak
222
+ Weak
223
+ Fair
224
+ Fair
225
+ Weak
226
+ Fair
227
+ Strong
228
+ Strong
229
+ Fair
230
+ Fair
231
+ Weak
232
+ Fair
233
+ Weak
234
+ Strong
235
+ Fair
236
+ Fair
237
+ Weak
238
+ Weak
239
+ Fair
240
+ Fair
241
+ Strong
242
+ Fair
243
+ Weak
244
+ Weak
245
+ Weak
246
+ Weak
247
+ Fair
248
+ Fair
249
+ Weak
250
+ Weak
251
+ Strong
252
+ Strong
253
+ Weak
254
+ Fair
255
+ Strong
256
+ Fair
257
+ Fair
258
+ Weak
259
+ Fair
260
+ Weak
261
+ Weak
262
+ Fair
263
+ Strong
264
+ Fair
265
+ Weak
266
+ Fair
267
+ Fair
268
+ Weak
269
+ Fair
270
+ Fair
271
+ Fair
272
+ Fair
273
+ Fair
274
+ Strong
275
+ Fair
276
+ Weak
277
+ Fair
278
+ Strong
279
+ Fair
280
+ Strong
281
+ Strong
282
+ Weak
283
+ Strong
284
+ Fair
285
+ Fair
286
+ Fair
287
+ Weak
288
+ Fair
289
+ Strong
290
+ Strong
291
+ Fair
292
+ Fair
293
+ Fair
294
+ Weak
295
+ Weak
296
+ Weak
297
+ Fair
298
+ Weak
299
+ Fair
300
+ Weak
301
+ Fair
302
+ Fair
303
+ Fair
304
+ Weak
305
+ Weak
306
+ Strong
307
+ Fair
308
+ Weak
309
+ Fair
310
+ Weak
311
+ Fair
312
+ Fair
313
+ Fair
314
+ Weak
315
+ Strong
316
+ Fair
317
+ Strong
318
+ Fair
319
+ Strong
320
+ Weak
321
+ Weak
322
+ Weak
323
+ Fair
324
+ Fair
325
+ Weak
326
+ Weak
327
+ Weak
328
+ Weak
329
+ Fair
330
+ Weak
331
+ Fair
332
+ Strong
333
+ Strong
334
+ Weak
335
+ Fair
336
+ Fair
337
+ Fair
338
+ Weak
339
+ Weak
340
+ Weak
341
+ Fair
342
+ Fair
343
+ Fair
344
+ Fair
345
+ Weak
346
+ Fair
347
+ Weak
348
+ Strong
349
+ Fair
350
+ Weak
351
+ Strong
352
+ Fair
353
+ Fair
354
+ Fair
355
+ Weak
356
+ Weak
357
+ Weak
358
+ Strong
special_tokens_map.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<s>",
3
+ "cls_token": "<s>",
4
+ "eos_token": "</s>",
5
+ "mask_token": {
6
+ "content": "<mask>",
7
+ "lstrip": true,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "pad_token": "<pad>",
13
+ "sep_token": "</s>",
14
+ "unk_token": "<unk>"
15
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<s>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<pad>",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<unk>",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50264": {
37
+ "content": "<mask>",
38
+ "lstrip": true,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ }
44
+ },
45
+ "bos_token": "<s>",
46
+ "clean_up_tokenization_spaces": false,
47
+ "cls_token": "<s>",
48
+ "eos_token": "</s>",
49
+ "errors": "replace",
50
+ "extra_special_tokens": {},
51
+ "mask_token": "<mask>",
52
+ "model_max_length": 1000000000000000019884624838656,
53
+ "pad_token": "<pad>",
54
+ "sep_token": "</s>",
55
+ "tokenizer_class": "LongformerTokenizer",
56
+ "trim_offsets": true,
57
+ "unk_token": "<unk>"
58
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75e35cf13d31d8c0081765680721bc9fe9a73a4bc2d29e0f273348c2b66b93f0
3
+ size 5841
vocab.json ADDED
The diff for this file is too large to render. See raw diff