|
|
--- |
|
|
library_name: transformers |
|
|
license: mit |
|
|
base_model: FacebookAI/roberta-base |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
model-index: |
|
|
- name: eternis_router_encoder_sft_9Sep |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# eternis_router_encoder_sft_9Sep |
|
|
|
|
|
This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on the None dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 0.7641 |
|
|
- Mse: 0.4879 |
|
|
- Mae: 0.4952 |
|
|
|
|
|
## Model description |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 5e-05 |
|
|
- train_batch_size: 8 |
|
|
- eval_batch_size: 16 |
|
|
- seed: 42 |
|
|
- gradient_accumulation_steps: 4 |
|
|
- total_train_batch_size: 32 |
|
|
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
|
- lr_scheduler_type: linear |
|
|
- lr_scheduler_warmup_ratio: 0.1 |
|
|
- num_epochs: 20 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Mse | Mae | |
|
|
|:-------------:|:-------:|:-----:|:---------------:|:------:|:------:| |
|
|
| 8.5061 | 0.3429 | 300 | 1.7282 | 1.0945 | 0.8522 | |
|
|
| 8.1139 | 0.6857 | 600 | 1.6352 | 1.0425 | 0.8249 | |
|
|
| 7.7043 | 1.0286 | 900 | 1.5126 | 0.9853 | 0.8051 | |
|
|
| 6.9404 | 1.3714 | 1200 | 1.3621 | 0.8932 | 0.7625 | |
|
|
| 6.6003 | 1.7143 | 1500 | 1.2822 | 0.8509 | 0.7413 | |
|
|
| 5.8303 | 2.0571 | 1800 | 1.1538 | 0.7647 | 0.7005 | |
|
|
| 5.4533 | 2.4 | 2100 | 1.1136 | 0.7337 | 0.6812 | |
|
|
| 5.0087 | 2.7429 | 2400 | 1.0895 | 0.7149 | 0.6684 | |
|
|
| 4.8301 | 3.0857 | 2700 | 1.0786 | 0.7060 | 0.6610 | |
|
|
| 4.676 | 3.4286 | 3000 | 1.0654 | 0.6971 | 0.6550 | |
|
|
| 4.6187 | 3.7714 | 3300 | 1.0473 | 0.6842 | 0.6447 | |
|
|
| 4.3955 | 4.1143 | 3600 | 1.0129 | 0.6608 | 0.6308 | |
|
|
| 4.4388 | 4.4571 | 3900 | 1.0255 | 0.6688 | 0.6355 | |
|
|
| 4.3574 | 4.8 | 4200 | 0.9843 | 0.6405 | 0.6154 | |
|
|
| 4.384 | 5.1429 | 4500 | 0.9835 | 0.6403 | 0.6153 | |
|
|
| 4.4066 | 5.4857 | 4800 | 1.0027 | 0.6535 | 0.6256 | |
|
|
| 4.4052 | 5.8286 | 5100 | 0.9643 | 0.6275 | 0.6050 | |
|
|
| 4.3846 | 6.1714 | 5400 | 0.9507 | 0.6175 | 0.5982 | |
|
|
| 4.31 | 6.5143 | 5700 | 0.9536 | 0.6193 | 0.5989 | |
|
|
| 4.2701 | 6.8571 | 6000 | 0.9481 | 0.6159 | 0.5943 | |
|
|
| 4.2284 | 7.2 | 6300 | 0.9181 | 0.5951 | 0.5815 | |
|
|
| 4.1039 | 7.5429 | 6600 | 0.9036 | 0.5857 | 0.5731 | |
|
|
| 4.2027 | 7.8857 | 6900 | 0.8906 | 0.5758 | 0.5655 | |
|
|
| 4.1092 | 8.2286 | 7200 | 0.8899 | 0.5761 | 0.5653 | |
|
|
| 4.213 | 8.5714 | 7500 | 0.8903 | 0.5778 | 0.5658 | |
|
|
| 4.1775 | 8.9143 | 7800 | 0.8766 | 0.5663 | 0.5582 | |
|
|
| 4.1293 | 9.2571 | 8100 | 0.8891 | 0.5739 | 0.5637 | |
|
|
| 4.135 | 9.6 | 8400 | 0.8823 | 0.5701 | 0.5599 | |
|
|
| 4.1324 | 9.9429 | 8700 | 0.8633 | 0.5590 | 0.5546 | |
|
|
| 4.0383 | 10.2857 | 9000 | 0.8562 | 0.5524 | 0.5462 | |
|
|
| 4.0424 | 10.6286 | 9300 | 0.8390 | 0.5403 | 0.5376 | |
|
|
| 4.0851 | 10.9714 | 9600 | 0.8455 | 0.5446 | 0.5421 | |
|
|
| 3.9225 | 11.3143 | 9900 | 0.8190 | 0.5283 | 0.5305 | |
|
|
| 4.0771 | 11.6571 | 10200 | 0.8195 | 0.5261 | 0.5268 | |
|
|
| 3.9188 | 12.0 | 10500 | 0.8112 | 0.5218 | 0.5233 | |
|
|
| 3.9961 | 12.3429 | 10800 | 0.8157 | 0.5247 | 0.5241 | |
|
|
| 3.9793 | 12.6857 | 11100 | 0.7877 | 0.5059 | 0.5097 | |
|
|
| 4.0327 | 13.0286 | 11400 | 0.8180 | 0.5256 | 0.5242 | |
|
|
| 3.9755 | 13.3714 | 11700 | 0.8043 | 0.5173 | 0.5183 | |
|
|
| 3.9531 | 13.7143 | 12000 | 0.7860 | 0.5035 | 0.5060 | |
|
|
| 3.9182 | 14.0571 | 12300 | 0.7914 | 0.5070 | 0.5123 | |
|
|
| 3.8201 | 14.4 | 12600 | 0.7873 | 0.5033 | 0.5095 | |
|
|
| 3.8887 | 14.7429 | 12900 | 0.7868 | 0.5039 | 0.5077 | |
|
|
| 3.9021 | 15.0857 | 13200 | 0.7736 | 0.4964 | 0.5016 | |
|
|
| 3.8241 | 15.4286 | 13500 | 0.7677 | 0.4903 | 0.5002 | |
|
|
| 3.8387 | 15.7714 | 13800 | 0.7751 | 0.4953 | 0.5025 | |
|
|
| 3.8344 | 16.1143 | 14100 | 0.7802 | 0.5004 | 0.5023 | |
|
|
| 3.8173 | 16.4571 | 14400 | 0.7588 | 0.4853 | 0.4958 | |
|
|
| 3.9998 | 16.8 | 14700 | 0.7556 | 0.4828 | 0.4915 | |
|
|
| 3.838 | 17.1429 | 15000 | 0.7690 | 0.4916 | 0.4988 | |
|
|
| 3.7549 | 17.4857 | 15300 | 0.7545 | 0.4819 | 0.4920 | |
|
|
| 3.8553 | 17.8286 | 15600 | 0.7901 | 0.5058 | 0.5077 | |
|
|
| 3.7462 | 18.1714 | 15900 | 0.7558 | 0.4829 | 0.4947 | |
|
|
| 3.7608 | 18.5143 | 16200 | 0.7518 | 0.4796 | 0.4890 | |
|
|
| 3.9487 | 18.8571 | 16500 | 0.7590 | 0.4836 | 0.4915 | |
|
|
| 3.9489 | 19.2 | 16800 | 0.7278 | 0.4639 | 0.4758 | |
|
|
| 3.8653 | 19.5429 | 17100 | 0.7498 | 0.4786 | 0.4914 | |
|
|
| 3.7533 | 19.8857 | 17400 | 0.7641 | 0.4879 | 0.4952 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.56.1 |
|
|
- Pytorch 2.7.0 |
|
|
- Datasets 4.0.0 |
|
|
- Tokenizers 0.22.0 |
|
|
|
