AbdullahBarayan commited on
Commit
3c29f5f
·
verified ·
1 Parent(s): 891f92f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +92 -73
README.md CHANGED
@@ -1,73 +1,92 @@
1
- ---
2
- library_name: transformers
3
- license: apache-2.0
4
- base_model: answerdotai/ModernBERT-base
5
- tags:
6
- - generated_from_trainer
7
- metrics:
8
- - f1
9
- model-index:
10
- - name: ModernBERT-base-doc_en-Cefr
11
- results: []
12
- ---
13
-
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
-
17
- # ModernBERT-base-doc_en-Cefr
18
-
19
- This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the None dataset.
20
- It achieves the following results on the evaluation set:
21
- - Loss: 0.7333
22
- - F1: 0.8103
23
-
24
- ## Model description
25
-
26
- More information needed
27
-
28
- ## Intended uses & limitations
29
-
30
- More information needed
31
-
32
- ## Training and evaluation data
33
-
34
- More information needed
35
-
36
- ## Training procedure
37
-
38
- ### Training hyperparameters
39
-
40
- The following hyperparameters were used during training:
41
- - learning_rate: 3.6e-05
42
- - train_batch_size: 3
43
- - eval_batch_size: 3
44
- - seed: 42
45
- - gradient_accumulation_steps: 16
46
- - total_train_batch_size: 48
47
- - optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
48
- - lr_scheduler_type: linear
49
- - lr_scheduler_warmup_ratio: 0.1
50
- - num_epochs: 10
51
-
52
- ### Training results
53
-
54
- | Training Loss | Epoch | Step | Validation Loss | F1 |
55
- |:-------------:|:-----:|:----:|:---------------:|:------:|
56
- | 28.6059 | 1.0 | 14 | 1.2701 | 0.3637 |
57
- | 18.3081 | 2.0 | 28 | 0.9493 | 0.6123 |
58
- | 8.3371 | 3.0 | 42 | 0.7826 | 0.6860 |
59
- | 4.5689 | 4.0 | 56 | 0.6426 | 0.7304 |
60
- | 2.3147 | 5.0 | 70 | 0.8673 | 0.7149 |
61
- | 1.4668 | 6.0 | 84 | 0.7212 | 0.7819 |
62
- | 0.7954 | 7.0 | 98 | 0.8906 | 0.7633 |
63
- | 0.3336 | 8.0 | 112 | 0.8336 | 0.7877 |
64
- | 0.1416 | 9.0 | 126 | 0.7927 | 0.7921 |
65
- | 0.0155 | 10.0 | 140 | 0.7333 | 0.8103 |
66
-
67
-
68
- ### Framework versions
69
-
70
- - Transformers 4.53.1
71
- - Pytorch 2.6.0+cu124
72
- - Datasets 3.6.0
73
- - Tokenizers 0.21.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: answerdotai/ModernBERT-base
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - f1
9
+ model-index:
10
+ - name: ModernBERT-base-doc_en-Cefr
11
+ results: []
12
+ language:
13
+ - en
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # ModernBERT-base-doc_en-Cefr
20
+
21
+ This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the None dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 0.7333
24
+ - F1: 0.8103
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 3.6e-05
44
+ - train_batch_size: 3
45
+ - eval_batch_size: 3
46
+ - seed: 42
47
+ - gradient_accumulation_steps: 16
48
+ - total_train_batch_size: 48
49
+ - optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
50
+ - lr_scheduler_type: linear
51
+ - lr_scheduler_warmup_ratio: 0.1
52
+ - num_epochs: 10
53
+
54
+ ### Training results
55
+
56
+ | Training Loss | Epoch | Step | Validation Loss | F1 |
57
+ |:-------------:|:-----:|:----:|:---------------:|:------:|
58
+ | 28.6059 | 1.0 | 14 | 1.2701 | 0.3637 |
59
+ | 18.3081 | 2.0 | 28 | 0.9493 | 0.6123 |
60
+ | 8.3371 | 3.0 | 42 | 0.7826 | 0.6860 |
61
+ | 4.5689 | 4.0 | 56 | 0.6426 | 0.7304 |
62
+ | 2.3147 | 5.0 | 70 | 0.8673 | 0.7149 |
63
+ | 1.4668 | 6.0 | 84 | 0.7212 | 0.7819 |
64
+ | 0.7954 | 7.0 | 98 | 0.8906 | 0.7633 |
65
+ | 0.3336 | 8.0 | 112 | 0.8336 | 0.7877 |
66
+ | 0.1416 | 9.0 | 126 | 0.7927 | 0.7921 |
67
+ | 0.0155 | 10.0 | 140 | 0.7333 | 0.8103 |
68
+
69
+
70
+ ### Framework versions
71
+
72
+ - Transformers 4.53.1
73
+ - Pytorch 2.6.0+cu124
74
+ - Datasets 3.6.0
75
+ - Tokenizers 0.21.2
76
+
77
+ ### Citation
78
+ ```bibtex
79
+ @inproceedings{alva-manchego-etal-2025-findings,
80
+ title = "Findings of the {TSAR} 2025 Shared Task on Readability-Controlled Text Simplification",
81
+ author = "Alva-Manchego, Fernando and Stodden, Regina and Imperial, Joseph Marvin and Barayan, Abdullah and North, Kai and Tayyar Madabushi, Harish",
82
+ editor = "Shardlow, Matthew and Alva-Manchego, Fernando and North, Kai and Stodden, Regina and Saggion, Horacio and Khallaf, Nouran and Hayakawa, Akio",
83
+ booktitle = "Proceedings of the Fourth Workshop on Text Simplification, Accessibility and Readability (TSAR 2025)",
84
+ month = nov,
85
+ year = "2025",
86
+ address = "Suzhou, China",
87
+ publisher = "Association for Computational Linguistics",
88
+ url = "https://aclanthology.org/2025.tsar-1.8/",
89
+ doi = "10.18653/v1/2025.tsar-1.8",
90
+ pages = "116--130",
91
+ ISBN = "979-8-89176-176-6"
92
+ }