Revision_XLMRoBERTa_Lexical_Dataset_46k_NewType

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4730
• Accuracy: 0.8828
• F1: 0.8722

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2770	200	0.4823	0.7583	0.7452
No log	0.5540	400	0.5181	0.7915	0.7434
No log	0.8310	600	0.3669	0.8389	0.8254
0.4864	1.1080	800	0.3475	0.8475	0.8308
0.4864	1.3850	1000	0.3577	0.8293	0.8239
0.4864	1.6620	1200	0.3062	0.8692	0.8539
0.4864	1.9391	1400	0.2959	0.8721	0.8621
0.3534	2.2161	1600	0.3268	0.8663	0.8576
0.3534	2.4931	1800	0.3067	0.8780	0.8655
0.3534	2.7701	2000	0.2833	0.8803	0.8717
0.2898	3.0471	2200	0.2778	0.8850	0.8742
0.2898	3.3241	2400	0.2892	0.8822	0.8688
0.2898	3.6011	2600	0.2676	0.8839	0.8760
0.2898	3.8781	2800	0.2785	0.8857	0.8754
0.2547	4.1551	3000	0.2999	0.8836	0.8738
0.2547	4.4321	3200	0.2896	0.8831	0.8738
0.2547	4.7091	3400	0.2877	0.8885	0.8793
0.2547	4.9861	3600	0.2775	0.8876	0.8754
0.2274	5.2632	3800	0.2842	0.8850	0.8768
0.2274	5.5402	4000	0.3091	0.8791	0.8695
0.2274	5.8172	4200	0.2676	0.8886	0.8808
0.2017	6.0942	4400	0.3033	0.8894	0.8783
0.2017	6.3712	4600	0.3274	0.8863	0.8753
0.2017	6.6482	4800	0.3238	0.8851	0.8761
0.2017	6.9252	5000	0.3197	0.8872	0.8778
0.1794	7.2022	5200	0.3446	0.8822	0.8731
0.1794	7.4792	5400	0.3409	0.8869	0.8771
0.1794	7.7562	5600	0.3257	0.8871	0.8763
0.1577	8.0332	5800	0.3625	0.8856	0.8743
0.1577	8.3102	6000	0.3924	0.8820	0.8706
0.1577	8.5873	6200	0.3439	0.8888	0.8787
0.1577	8.8643	6400	0.3541	0.8848	0.8748
0.1407	9.1413	6600	0.3880	0.8849	0.8748
0.1407	9.4183	6800	0.3764	0.8833	0.8722
0.1407	9.6953	7000	0.3744	0.8837	0.8732
0.1407	9.9723	7200	0.3950	0.8815	0.8692
0.1257	10.2493	7400	0.4004	0.8849	0.8751
0.1257	10.5263	7600	0.4374	0.8831	0.8729
0.1257	10.8033	7800	0.4106	0.8844	0.8733
0.1086	11.0803	8000	0.4207	0.8819	0.8723
0.1086	11.3573	8200	0.4518	0.8843	0.8740
0.1086	11.6343	8400	0.4324	0.8861	0.8749
0.1086	11.9114	8600	0.4153	0.8839	0.8728
0.0992	12.1884	8800	0.4506	0.8830	0.8728
0.0992	12.4654	9000	0.4281	0.8850	0.8745
0.0992	12.7424	9200	0.4477	0.8798	0.8702
0.0903	13.0194	9400	0.4374	0.8834	0.8736
0.0903	13.2964	9600	0.4329	0.8845	0.8733
0.0903	13.5734	9800	0.4572	0.8850	0.8739
0.0903	13.8504	10000	0.4665	0.8828	0.8719
0.0807	14.1274	10200	0.4645	0.8833	0.8731
0.0807	14.4044	10400	0.4703	0.8842	0.8738
0.0807	14.6814	10600	0.4702	0.8825	0.8724
0.0807	14.9584	10800	0.4730	0.8828	0.8722

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2