demo-oss-xlmr

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7735

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.4115	200	2.7150
No log	0.8230	400	2.4276
No log	1.2346	600	2.2905
No log	1.6461	800	2.1311
No log	2.0576	1000	2.0614
No log	2.4691	1200	1.9768
No log	2.8807	1400	1.9533
No log	3.2922	1600	1.9050
No log	3.7037	1800	1.8647
No log	4.1152	2000	1.8106
No log	4.5267	2200	1.8017
No log	4.9383	2400	1.7735

Framework versions

• Transformers 4.57.1
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.1