Hugging Face's logo

LucileFavero
/
out_base_S

Text Classification
Transformers
Safetensors
longformer
Generated from Trainer
Model card Files
xet
Community
LucileFavero commited on
Commit
48b4edf
·
verified ·
1 Parent(s): 6342c20

LucileFavero/LF_Sentence

Browse files
Files changed (11) hide show
  1. README.md +71 -0
  2. config.json +52 -0
  3. eval_S.csv +2 -0
  4. merges.txt +0 -0
  5. model.safetensors +3 -0
  6. pred_S.csv +358 -0
  7. special_tokens_map.json +15 -0
  8. tokenizer.json +0 -0
  9. tokenizer_config.json +58 -0
  10. training_args.bin +3 -0
  11. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: allenai/longformer-base-4096
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - accuracy
9
+ - f1
10
+ model-index:
11
+ - name: out_base_S
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # out_base_S
19
+
20
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the None dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 0.9425
23
+ - Accuracy: 0.6583
24
+ - F1: 0.6496
25
+ - Cohen Kappa: 0.4906
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 2e-05
45
+ - train_batch_size: 1
46
+ - eval_batch_size: 1
47
+ - seed: 42
48
+ - gradient_accumulation_steps: 8
49
+ - total_train_batch_size: 8
50
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
51
+ - lr_scheduler_type: linear
52
+ - num_epochs: 5
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Cohen Kappa |
58
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:-----------:|
59
+ | 0.7488 | 1.0 | 134 | 0.7795 | 0.6218 | 0.4799 | 0.5341 |
60
+ | 0.7346 | 2.0 | 268 | 0.7503 | 0.6275 | 0.6221 | 0.4471 |
61
+ | 0.617 | 3.0 | 402 | 0.7870 | 0.6667 | 0.6556 | 0.4862 |
62
+ | 0.5551 | 4.0 | 536 | 0.9548 | 0.6583 | 0.6407 | 0.4721 |
63
+ | 0.3533 | 5.0 | 670 | 0.9425 | 0.6583 | 0.6496 | 0.4906 |
64
+
65
+
66
+ ### Framework versions
67
+
68
+ - Transformers 4.57.3
69
+ - Pytorch 2.9.0+cu126
70
+ - Datasets 4.0.0
71
+ - Tokenizers 0.22.2
config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LongformerForSequenceClassification"
4
+ ],
5
+ "attention_mode": "longformer",
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "attention_window": [
8
+ 512,
9
+ 512,
10
+ 512,
11
+ 512,
12
+ 512,
13
+ 512,
14
+ 512,
15
+ 512,
16
+ 512,
17
+ 512,
18
+ 512,
19
+ 512
20
+ ],
21
+ "bos_token_id": 0,
22
+ "dtype": "float32",
23
+ "eos_token_id": 2,
24
+ "gradient_checkpointing": false,
25
+ "hidden_act": "gelu",
26
+ "hidden_dropout_prob": 0.1,
27
+ "hidden_size": 768,
28
+ "id2label": {
29
+ "0": "Fair",
30
+ "1": "Strong",
31
+ "2": "Weak"
32
+ },
33
+ "ignore_attention_mask": false,
34
+ "initializer_range": 0.02,
35
+ "intermediate_size": 3072,
36
+ "label2id": {
37
+ "Fair": 0,
38
+ "Strong": 1,
39
+ "Weak": 2
40
+ },
41
+ "layer_norm_eps": 1e-05,
42
+ "max_position_embeddings": 4098,
43
+ "model_type": "longformer",
44
+ "num_attention_heads": 12,
45
+ "num_hidden_layers": 12,
46
+ "onnx_export": false,
47
+ "pad_token_id": 1,
48
+ "sep_token_id": 2,
49
+ "transformers_version": "4.57.3",
50
+ "type_vocab_size": 1,
51
+ "vocab_size": 50265
52
+ }
eval_S.csv ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ qwk,cor,exact
2
+ 0.42631530398729967,0.39104318796665744,0.6106442577030813
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4113cbe1a38a8e2530a95391f211397443de2bc03880eba14a4656a4db81b185
3
+ size 594681260
pred_S.csv ADDED
@@ -0,0 +1,358 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ S_score
2
+ Weak
3
+ Fair
4
+ Weak
5
+ Fair
6
+ Fair
7
+ Fair
8
+ Fair
9
+ Fair
10
+ Weak
11
+ Strong
12
+ Weak
13
+ Fair
14
+ Weak
15
+ Weak
16
+ Strong
17
+ Weak
18
+ Weak
19
+ Fair
20
+ Weak
21
+ Strong
22
+ Weak
23
+ Fair
24
+ Fair
25
+ Fair
26
+ Fair
27
+ Strong
28
+ Fair
29
+ Fair
30
+ Fair
31
+ Weak
32
+ Strong
33
+ Weak
34
+ Fair
35
+ Weak
36
+ Weak
37
+ Weak
38
+ Weak
39
+ Weak
40
+ Weak
41
+ Fair
42
+ Strong
43
+ Fair
44
+ Fair
45
+ Weak
46
+ Strong
47
+ Weak
48
+ Fair
49
+ Fair
50
+ Fair
51
+ Fair
52
+ Strong
53
+ Strong
54
+ Fair
55
+ Fair
56
+ Fair
57
+ Weak
58
+ Strong
59
+ Strong
60
+ Weak
61
+ Fair
62
+ Fair
63
+ Strong
64
+ Fair
65
+ Weak
66
+ Strong
67
+ Weak
68
+ Strong
69
+ Weak
70
+ Fair
71
+ Weak
72
+ Weak
73
+ Fair
74
+ Weak
75
+ Weak
76
+ Fair
77
+ Fair
78
+ Fair
79
+ Strong
80
+ Fair
81
+ Fair
82
+ Fair
83
+ Weak
84
+ Weak
85
+ Fair
86
+ Weak
87
+ Weak
88
+ Weak
89
+ Strong
90
+ Weak
91
+ Weak
92
+ Fair
93
+ Fair
94
+ Fair
95
+ Fair
96
+ Fair
97
+ Weak
98
+ Weak
99
+ Fair
100
+ Fair
101
+ Weak
102
+ Strong
103
+ Fair
104
+ Weak
105
+ Fair
106
+ Weak
107
+ Strong
108
+ Weak
109
+ Weak
110
+ Weak
111
+ Fair
112
+ Fair
113
+ Fair
114
+ Fair
115
+ Fair
116
+ Strong
117
+ Strong
118
+ Weak
119
+ Weak
120
+ Fair
121
+ Fair
122
+ Fair
123
+ Weak
124
+ Weak
125
+ Fair
126
+ Weak
127
+ Strong
128
+ Weak
129
+ Weak
130
+ Strong
131
+ Weak
132
+ Weak
133
+ Fair
134
+ Strong
135
+ Strong
136
+ Weak
137
+ Fair
138
+ Fair
139
+ Weak
140
+ Strong
141
+ Weak
142
+ Strong
143
+ Weak
144
+ Weak
145
+ Weak
146
+ Fair
147
+ Fair
148
+ Fair
149
+ Strong
150
+ Strong
151
+ Fair
152
+ Fair
153
+ Weak
154
+ Fair
155
+ Weak
156
+ Strong
157
+ Weak
158
+ Fair
159
+ Weak
160
+ Fair
161
+ Fair
162
+ Fair
163
+ Fair
164
+ Fair
165
+ Fair
166
+ Strong
167
+ Fair
168
+ Fair
169
+ Weak
170
+ Fair
171
+ Fair
172
+ Weak
173
+ Weak
174
+ Weak
175
+ Weak
176
+ Strong
177
+ Strong
178
+ Fair
179
+ Fair
180
+ Fair
181
+ Fair
182
+ Strong
183
+ Weak
184
+ Fair
185
+ Strong
186
+ Weak
187
+ Weak
188
+ Strong
189
+ Fair
190
+ Weak
191
+ Fair
192
+ Weak
193
+ Strong
194
+ Weak
195
+ Fair
196
+ Fair
197
+ Fair
198
+ Weak
199
+ Weak
200
+ Weak
201
+ Strong
202
+ Strong
203
+ Fair
204
+ Fair
205
+ Weak
206
+ Strong
207
+ Fair
208
+ Fair
209
+ Strong
210
+ Weak
211
+ Strong
212
+ Weak
213
+ Weak
214
+ Weak
215
+ Weak
216
+ Weak
217
+ Strong
218
+ Fair
219
+ Fair
220
+ Fair
221
+ Weak
222
+ Weak
223
+ Fair
224
+ Fair
225
+ Weak
226
+ Fair
227
+ Strong
228
+ Strong
229
+ Fair
230
+ Fair
231
+ Weak
232
+ Fair
233
+ Weak
234
+ Strong
235
+ Fair
236
+ Fair
237
+ Weak
238
+ Fair
239
+ Fair
240
+ Fair
241
+ Strong
242
+ Fair
243
+ Weak
244
+ Weak
245
+ Weak
246
+ Weak
247
+ Fair
248
+ Fair
249
+ Fair
250
+ Fair
251
+ Strong
252
+ Strong
253
+ Weak
254
+ Fair
255
+ Strong
256
+ Fair
257
+ Fair
258
+ Weak
259
+ Fair
260
+ Weak
261
+ Weak
262
+ Fair
263
+ Strong
264
+ Fair
265
+ Weak
266
+ Fair
267
+ Fair
268
+ Weak
269
+ Fair
270
+ Fair
271
+ Fair
272
+ Fair
273
+ Fair
274
+ Strong
275
+ Fair
276
+ Weak
277
+ Fair
278
+ Strong
279
+ Fair
280
+ Strong
281
+ Strong
282
+ Weak
283
+ Strong
284
+ Fair
285
+ Weak
286
+ Fair
287
+ Fair
288
+ Fair
289
+ Fair
290
+ Strong
291
+ Fair
292
+ Fair
293
+ Fair
294
+ Fair
295
+ Weak
296
+ Weak
297
+ Fair
298
+ Weak
299
+ Fair
300
+ Weak
301
+ Fair
302
+ Fair
303
+ Fair
304
+ Weak
305
+ Weak
306
+ Strong
307
+ Fair
308
+ Weak
309
+ Fair
310
+ Weak
311
+ Weak
312
+ Weak
313
+ Fair
314
+ Fair
315
+ Strong
316
+ Fair
317
+ Strong
318
+ Fair
319
+ Strong
320
+ Weak
321
+ Weak
322
+ Weak
323
+ Fair
324
+ Fair
325
+ Weak
326
+ Weak
327
+ Weak
328
+ Fair
329
+ Fair
330
+ Weak
331
+ Fair
332
+ Strong
333
+ Fair
334
+ Weak
335
+ Fair
336
+ Fair
337
+ Fair
338
+ Weak
339
+ Weak
340
+ Weak
341
+ Fair
342
+ Weak
343
+ Fair
344
+ Fair
345
+ Weak
346
+ Fair
347
+ Weak
348
+ Fair
349
+ Fair
350
+ Weak
351
+ Strong
352
+ Fair
353
+ Fair
354
+ Fair
355
+ Weak
356
+ Weak
357
+ Weak
358
+ Strong
special_tokens_map.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<s>",
3
+ "cls_token": "<s>",
4
+ "eos_token": "</s>",
5
+ "mask_token": {
6
+ "content": "<mask>",
7
+ "lstrip": true,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "pad_token": "<pad>",
13
+ "sep_token": "</s>",
14
+ "unk_token": "<unk>"
15
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<s>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<pad>",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<unk>",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50264": {
37
+ "content": "<mask>",
38
+ "lstrip": true,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ }
44
+ },
45
+ "bos_token": "<s>",
46
+ "clean_up_tokenization_spaces": false,
47
+ "cls_token": "<s>",
48
+ "eos_token": "</s>",
49
+ "errors": "replace",
50
+ "extra_special_tokens": {},
51
+ "mask_token": "<mask>",
52
+ "model_max_length": 1000000000000000019884624838656,
53
+ "pad_token": "<pad>",
54
+ "sep_token": "</s>",
55
+ "tokenizer_class": "LongformerTokenizer",
56
+ "trim_offsets": true,
57
+ "unk_token": "<unk>"
58
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f503ffb3ed2bab92595efe9a84062ea15a6304521af0d71446dfbb9125ccf55
3
+ size 5841
vocab.json ADDED
The diff for this file is too large to render. See raw diff