metadata
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- generated_from_trainer
model-index:
- name: results
results: []
results
This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.6970
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 3
- eval_batch_size: 3
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.4077 | 0.0782 | 10000 | 1.2127 |
| 1.3837 | 0.1565 | 20000 | 1.1769 |
| 1.264 | 0.2347 | 30000 | 1.1395 |
| 1.2336 | 0.3130 | 40000 | 1.2410 |
| 1.1166 | 0.3912 | 50000 | 1.1913 |
| 1.2681 | 0.4695 | 60000 | 1.1694 |
| 1.1209 | 0.5477 | 70000 | 1.1809 |
| 1.1211 | 0.6259 | 80000 | 1.1599 |
| 1.0544 | 0.7042 | 90000 | 1.1876 |
| 1.1172 | 0.7824 | 100000 | 1.1239 |
| 0.9684 | 0.8607 | 110000 | 1.1358 |
| 1.0254 | 0.9389 | 120000 | 1.1637 |
| 1.0335 | 1.0171 | 130000 | 1.1263 |
| 1.0661 | 1.0954 | 140000 | 1.1546 |
| 0.9775 | 1.1736 | 150000 | 1.0895 |
| 0.9167 | 1.2519 | 160000 | 0.8886 |
| 0.8994 | 1.3301 | 170000 | 0.8791 |
| 0.9155 | 1.4084 | 180000 | 1.0548 |
| 0.9194 | 1.4866 | 190000 | 1.1017 |
| 0.8942 | 1.5648 | 200000 | 1.0410 |
| 0.865 | 1.6431 | 210000 | 1.0301 |
| 0.9295 | 1.7213 | 220000 | nan |
| 0.9483 | 1.7996 | 230000 | 1.0066 |
| 0.8371 | 1.8778 | 240000 | 1.0697 |
| 0.8137 | 1.9560 | 250000 | 1.0518 |
| 0.8199 | 2.0343 | 260000 | 0.9879 |
| 0.774 | 2.1125 | 270000 | 1.0009 |
| 0.8802 | 2.1908 | 280000 | 1.0175 |
| 0.7443 | 2.2690 | 290000 | 1.0213 |
| 0.7648 | 2.3473 | 300000 | 0.9495 |
| 0.8057 | 2.4255 | 310000 | 0.9668 |
| 0.7384 | 2.5037 | 320000 | 0.9304 |
| 0.7542 | 2.5820 | 330000 | 0.9320 |
| 0.7462 | 2.6602 | 340000 | 0.9513 |
| 0.7154 | 2.7385 | 350000 | 0.9557 |
| 0.7571 | 2.8167 | 360000 | 0.9425 |
| 0.7021 | 2.8949 | 370000 | 0.9010 |
| 0.7114 | 2.9732 | 380000 | 0.9036 |
Framework versions
- Transformers 4.49.0
- Pytorch 2.6.0+cu124
- Datasets 3.3.1
- Tokenizers 0.21.0