Revision_XLMRoBERTa_Lexical_Dataset_52k

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5369
• Accuracy: 0.8678
• F1: 0.8611

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2421	200	0.4928	0.7595	0.7497
No log	0.4843	400	0.4409	0.7850	0.7796
No log	0.7264	600	0.3714	0.8242	0.8161
No log	0.9685	800	0.3523	0.8377	0.8274
0.5143	1.2107	1000	0.3573	0.8293	0.8241
0.5143	1.4528	1200	0.3325	0.8442	0.8365
0.5143	1.6949	1400	0.3264	0.8611	0.8483
0.5143	1.9370	1600	0.3023	0.8505	0.8445
0.3673	2.1792	1800	0.2878	0.8685	0.8609
0.3673	2.4213	2000	0.2932	0.8684	0.8595
0.3673	2.6634	2200	0.3163	0.8469	0.8414
0.3673	2.9056	2400	0.2874	0.8669	0.8590
0.3039	3.1477	2600	0.2998	0.8591	0.8531
0.3039	3.3898	2800	0.3076	0.8600	0.8544
0.3039	3.6320	3000	0.3091	0.8625	0.8567
0.3039	3.8741	3200	0.3335	0.8536	0.8483
0.2700	4.1162	3400	0.3292	0.8651	0.8589
0.2700	4.3584	3600	0.3018	0.8733	0.8659
0.2700	4.6005	3800	0.2970	0.8778	0.8699
0.2700	4.8426	4000	0.2990	0.8692	0.8626
0.2405	5.0847	4200	0.3537	0.8602	0.8545
0.2405	5.3269	4400	0.3316	0.8731	0.8667
0.2405	5.5690	4600	0.3296	0.8712	0.8648
0.2405	5.8111	4800	0.3900	0.8476	0.8429
0.2117	6.0533	5000	0.3456	0.8693	0.8626
0.2117	6.2954	5200	0.3669	0.8632	0.8575
0.2117	6.5375	5400	0.3592	0.8671	0.8606
0.2117	6.7797	5600	0.3413	0.8690	0.8632
0.1875	7.0218	5800	0.3686	0.8686	0.8617
0.1875	7.2639	6000	0.4161	0.8629	0.8575
0.1875	7.5061	6200	0.3601	0.8779	0.8716
0.1875	7.7482	6400	0.3508	0.8772	0.8705
0.1658	7.9903	6600	0.3804	0.8658	0.8604
0.1658	8.2324	6800	0.3951	0.8749	0.8681
0.1658	8.4746	7000	0.4027	0.8758	0.8693
0.1658	8.7167	7200	0.4352	0.8613	0.8557
0.1658	8.9588	7400	0.4107	0.8683	0.8619
0.1462	9.2010	7600	0.4389	0.8665	0.8605
0.1462	9.4431	7800	0.4402	0.8663	0.8602
0.1462	9.6852	8000	0.4104	0.8743	0.8677
0.1462	9.9274	8200	0.4018	0.8721	0.8657
0.1272	10.1695	8400	0.4486	0.8695	0.8630
0.1272	10.4116	8600	0.4651	0.8685	0.8621
0.1272	10.6538	8800	0.4658	0.8701	0.8634
0.1272	10.8959	9000	0.4603	0.8672	0.8608
0.1126	11.1380	9200	0.4581	0.8711	0.8648
0.1126	11.3801	9400	0.4629	0.8715	0.8649
0.1126	11.6223	9600	0.4983	0.8624	0.8565
0.1126	11.8644	9800	0.4868	0.8701	0.8637
0.1028	12.1065	10000	0.5103	0.8665	0.8605
0.1028	12.3487	10200	0.4814	0.8712	0.8642
0.1028	12.5908	10400	0.5268	0.8657	0.8596
0.1028	12.8329	10600	0.5266	0.8672	0.8611
0.0921	13.0751	10800	0.5216	0.8679	0.8615
0.0921	13.3172	11000	0.5051	0.8731	0.8662
0.0921	13.5593	11200	0.5522	0.8670	0.8608
0.0921	13.8015	11400	0.5347	0.8660	0.8596
0.0821	14.0436	11600	0.5207	0.8687	0.8620
0.0821	14.2857	11800	0.5256	0.8683	0.8617
0.0821	14.5278	12000	0.5402	0.8666	0.8602
0.0821	14.7700	12200	0.5369	0.8678	0.8611

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2