Hugging Face's logo

DayCardoso
/
valueeval24-modern-bert-cos

Text Classification
Transformers
Safetensors
English
modernbert
multi-label-classification
human-values
Generated from Trainer
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
Community
DayCardoso commited on
Commit
dda1705
·
verified ·
1 Parent(s): b67dcde

Model save

Browse files
Files changed (4) hide show
  1. README.md +19 -22
  2. best_model/model.safetensors +1 -1
  3. best_model/training_args.bin +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -8,21 +8,21 @@ metrics:
8
  - f1
9
  - accuracy
10
  model-index:
11
- - name: valueeval24-modern-bert
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
- # valueeval24-modern-bert
19
 
20
  This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 0.1613
23
- - F1: 0.3178
24
- - Roc Auc: 0.6190
25
- - Accuracy: 0.1954
26
 
27
  ## Model description
28
 
@@ -41,12 +41,12 @@ More information needed
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
- - learning_rate: 5e-06
45
  - train_batch_size: 8
46
  - eval_batch_size: 8
47
- - seed: 2024
48
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
- - lr_scheduler_type: linear
50
  - lr_scheduler_warmup_ratio: 0.01
51
  - num_epochs: 20
52
  - mixed_precision_training: Native AMP
@@ -55,19 +55,16 @@ The following hyperparameters were used during training:
55
 
56
  | Training Loss | Epoch | Step | Validation Loss | F1 | Roc Auc | Accuracy |
57
  |:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:|:--------:|
58
- | 0.1463 | 1.0 | 2883 | 0.1052 | 0.1633 | 0.5464 | 0.0854 |
59
- | 0.1003 | 2.0 | 5766 | 0.0995 | 0.2146 | 0.5640 | 0.1188 |
60
- | 0.0907 | 3.0 | 8649 | 0.0981 | 0.2777 | 0.5899 | 0.1662 |
61
- | 0.0806 | 4.0 | 11532 | 0.1001 | 0.3038 | 0.6035 | 0.1804 |
62
- | 0.0685 | 5.0 | 14415 | 0.1048 | 0.3099 | 0.6094 | 0.1914 |
63
- | 0.0549 | 6.0 | 17298 | 0.1104 | 0.3209 | 0.6177 | 0.1968 |
64
- | 0.0412 | 7.0 | 20181 | 0.1158 | 0.3197 | 0.6198 | 0.1934 |
65
- | 0.0285 | 8.0 | 23064 | 0.1232 | 0.3226 | 0.6210 | 0.1974 |
66
- | 0.0184 | 9.0 | 25947 | 0.1312 | 0.3157 | 0.6186 | 0.1943 |
67
- | 0.0114 | 10.0 | 28830 | 0.1381 | 0.3176 | 0.6192 | 0.1951 |
68
- | 0.0071 | 11.0 | 31713 | 0.1463 | 0.3216 | 0.6216 | 0.1972 |
69
- | 0.0047 | 12.0 | 34596 | 0.1542 | 0.3153 | 0.6168 | 0.1959 |
70
- | 0.0032 | 13.0 | 37479 | 0.1613 | 0.3178 | 0.6190 | 0.1954 |
71
 
72
 
73
  ### Framework versions
 
8
  - f1
9
  - accuracy
10
  model-index:
11
+ - name: valueeval24-modern-bert-cos
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
+ # valueeval24-modern-bert-cos
19
 
20
  This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.1822
23
+ - F1: 0.4181
24
+ - Roc Auc: 0.6703
25
+ - Accuracy: 0.28
26
 
27
  ## Model description
28
 
 
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
+ - learning_rate: 3e-05
45
  - train_batch_size: 8
46
  - eval_batch_size: 8
47
+ - seed: 42
48
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
+ - lr_scheduler_type: cosine
50
  - lr_scheduler_warmup_ratio: 0.01
51
  - num_epochs: 20
52
  - mixed_precision_training: Native AMP
 
55
 
56
  | Training Loss | Epoch | Step | Validation Loss | F1 | Roc Auc | Accuracy |
57
  |:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:|:--------:|
58
+ | 0.1158 | 1.0 | 2883 | 0.0913 | 0.3100 | 0.6016 | 0.1966 |
59
+ | 0.0771 | 2.0 | 5766 | 0.0871 | 0.3997 | 0.6464 | 0.2675 |
60
+ | 0.0494 | 3.0 | 8649 | 0.0970 | 0.3983 | 0.6513 | 0.2757 |
61
+ | 0.025 | 4.0 | 11532 | 0.1187 | 0.4085 | 0.6699 | 0.2805 |
62
+ | 0.0132 | 5.0 | 14415 | 0.1315 | 0.4296 | 0.6834 | 0.2849 |
63
+ | 0.0077 | 6.0 | 17298 | 0.1478 | 0.4066 | 0.6707 | 0.2668 |
64
+ | 0.0044 | 7.0 | 20181 | 0.1619 | 0.4123 | 0.6716 | 0.2734 |
65
+ | 0.0029 | 8.0 | 23064 | 0.1705 | 0.4124 | 0.6653 | 0.275 |
66
+ | 0.0022 | 9.0 | 25947 | 0.1766 | 0.4082 | 0.6626 | 0.2764 |
67
+ | 0.0014 | 10.0 | 28830 | 0.1822 | 0.4181 | 0.6703 | 0.28 |
 
 
 
68
 
69
 
70
  ### Framework versions
best_model/model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6ac4c652bf257481a022bc3aefeab6eaae819ad2d89a9eb908115902861a430e
3
  size 598667264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:483255e8069b321564d7bff8b2845a9353aa983cc2cc59757e03fb2aac391709
3
  size 598667264
best_model/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:134dc5edc9623c243c8f008d5aaaa232b0a3f9e9ba9e310061f9286c7b0b80e8
3
  size 5304
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a05c28f342fffdf73e6d8f8176fb534ae720281cf47547ecf9a1a7feb9ced23
3
  size 5304
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fff05b9d1995de71c5384244936a737df7da03c90ca74083a13840e8dcf7f620
3
  size 598667264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:483255e8069b321564d7bff8b2845a9353aa983cc2cc59757e03fb2aac391709
3
  size 598667264