metadata
library_name: transformers
tags:
- generated_from_trainer
model-index:
- name: sal-base
results: []
sal-base
This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.8697
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 1
- eval_batch_size: 8
- seed: 45242
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine_with_restarts
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100.0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.0797 | 1.0 | 807 | 1.8427 |
| 1.4525 | 2.0 | 1614 | 1.0320 |
| 1.0207 | 3.0 | 2421 | 0.8592 |
| 1.0664 | 4.0 | 3228 | 0.7038 |
| 0.9961 | 5.0 | 4035 | 0.5068 |
| 0.6692 | 6.0 | 4842 | 0.3971 |
| 0.3041 | 7.0 | 5649 | 0.3844 |
| 0.4348 | 8.0 | 6456 | 0.3507 |
| 0.3898 | 9.0 | 7263 | 0.3439 |
| 0.1955 | 10.0 | 8070 | 0.2859 |
| 0.3438 | 11.0 | 8877 | 0.3049 |
| 0.2301 | 12.0 | 9684 | 0.2657 |
| 0.1325 | 13.0 | 10491 | 0.2427 |
| 0.1481 | 14.0 | 11298 | 0.2547 |
| 0.0214 | 15.0 | 12105 | 0.2966 |
| 0.1104 | 16.0 | 12912 | 0.2886 |
| 0.065 | 17.0 | 13719 | 0.2805 |
| 0.1236 | 18.0 | 14526 | 0.2881 |
| 0.011 | 19.0 | 15333 | 0.2888 |
| 0.2482 | 20.0 | 16140 | 0.3615 |
| 0.1819 | 21.0 | 16947 | 0.3657 |
| 0.0461 | 22.0 | 17754 | 0.3575 |
| 0.1013 | 23.0 | 18561 | 0.3397 |
| 0.0623 | 24.0 | 19368 | 0.3244 |
| 0.0177 | 25.0 | 20175 | 0.3442 |
| 0.0085 | 26.0 | 20982 | 0.3931 |
| 0.034 | 27.0 | 21789 | 0.3658 |
| 0.0587 | 28.0 | 22596 | 0.3712 |
| 0.1045 | 29.0 | 23403 | 0.4020 |
| 0.0367 | 30.0 | 24210 | 0.5245 |
| 0.0688 | 31.0 | 25017 | 0.5180 |
| 0.0468 | 32.0 | 25824 | 0.3551 |
| 0.0226 | 33.0 | 26631 | 0.3973 |
| 0.0788 | 34.0 | 27438 | 0.4155 |
| 0.0628 | 35.0 | 28245 | 0.4288 |
| 0.0232 | 36.0 | 29052 | 0.4475 |
| 0.0148 | 37.0 | 29859 | 0.4464 |
| 0.1581 | 38.0 | 30666 | 0.5728 |
| 0.102 | 39.0 | 31473 | 0.4071 |
| 0.086 | 40.0 | 32280 | 0.4359 |
| 0.082 | 41.0 | 33087 | 0.3960 |
| 0.0338 | 42.0 | 33894 | 0.5187 |
| 0.0656 | 43.0 | 34701 | 0.5074 |
| 0.0546 | 44.0 | 35508 | 0.4921 |
| 0.006 | 45.0 | 36315 | 0.5108 |
| 0.0042 | 46.0 | 37122 | 0.5130 |
| 0.0541 | 47.0 | 37929 | 0.5102 |
| 0.0351 | 48.0 | 38736 | 0.4801 |
| 0.0082 | 49.0 | 39543 | 0.5563 |
| 0.1399 | 50.0 | 40350 | 0.5250 |
| 0.0371 | 51.0 | 41157 | 0.5732 |
| 0.0125 | 52.0 | 41964 | 0.6164 |
| 0.0729 | 53.0 | 42771 | 0.6178 |
| 0.0705 | 54.0 | 43578 | 0.6195 |
| 0.1058 | 55.0 | 44385 | 0.6180 |
| 0.1061 | 56.0 | 45192 | 0.6778 |
| 0.1624 | 57.0 | 45999 | 0.6777 |
| 0.0078 | 58.0 | 46806 | 0.5889 |
| 0.0101 | 59.0 | 47613 | 0.6608 |
| 0.0125 | 60.0 | 48420 | 0.6582 |
| 0.0084 | 61.0 | 49227 | 0.6625 |
| 0.0632 | 62.0 | 50034 | 0.6881 |
| 0.0104 | 63.0 | 50841 | 0.6781 |
| 0.0104 | 64.0 | 51648 | 0.6790 |
| 0.0188 | 65.0 | 52455 | 0.6319 |
| 0.0369 | 66.0 | 53262 | 0.6444 |
| 0.0042 | 67.0 | 54069 | 0.7006 |
| 0.0615 | 68.0 | 54876 | 0.6862 |
| 0.0585 | 69.0 | 55683 | 0.6895 |
| 0.0033 | 70.0 | 56490 | 0.7316 |
| 0.0458 | 71.0 | 57297 | 0.7493 |
| 0.0462 | 72.0 | 58104 | 0.7519 |
| 0.0081 | 73.0 | 58911 | 0.7512 |
| 0.0234 | 74.0 | 59718 | 0.6921 |
| 0.013 | 75.0 | 60525 | 0.7668 |
| 0.0438 | 76.0 | 61332 | 0.7913 |
| 0.0117 | 77.0 | 62139 | 0.7236 |
| 0.0658 | 78.0 | 62946 | 0.7462 |
| 0.007 | 79.0 | 63753 | 0.7387 |
| 0.0177 | 80.0 | 64560 | 0.7444 |
| 0.0052 | 81.0 | 65367 | 0.7462 |
| 0.0005 | 82.0 | 66174 | 0.7463 |
| 0.005 | 83.0 | 66981 | 0.8667 |
| 0.0824 | 84.0 | 67788 | 0.7736 |
| 0.0067 | 85.0 | 68595 | 0.7529 |
| 0.014 | 86.0 | 69402 | 0.7829 |
| 0.0019 | 87.0 | 70209 | 0.8120 |
| 0.0091 | 88.0 | 71016 | 0.8180 |
| 0.0039 | 89.0 | 71823 | 0.8255 |
| 0.0085 | 90.0 | 72630 | 0.8275 |
| 0.0019 | 91.0 | 73437 | 0.8279 |
| 0.0419 | 92.0 | 74244 | 0.8148 |
| 0.0182 | 93.0 | 75051 | 0.8169 |
| 0.0152 | 94.0 | 75858 | 0.8195 |
| 0.0044 | 95.0 | 76665 | 0.8245 |
| 0.0027 | 96.0 | 77472 | 0.8645 |
| 0.0161 | 97.0 | 78279 | 0.8314 |
| 0.0028 | 98.0 | 79086 | 0.8691 |
| 0.0141 | 99.0 | 79893 | 0.8696 |
| 0.0053 | 100.0 | 80700 | 0.8697 |
Framework versions
- Transformers 4.48.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0