XMLRoberta_Dataset59KCoDuoi

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2879
• Accuracy: 0.9572
• F1: 0.9573

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.5115	200	0.1969	0.9324	0.9325
No log	1.0230	400	0.1623	0.9463	0.9467
No log	1.5345	600	0.1687	0.9486	0.9487
0.2066	2.0460	800	0.1706	0.9541	0.9544
0.2066	2.5575	1000	0.1454	0.9548	0.9550
0.2066	3.0691	1200	0.1511	0.9569	0.9571
0.2066	3.5806	1400	0.1495	0.9564	0.9565
0.1117	4.0921	1600	0.1576	0.9568	0.9568
0.1117	4.6036	1800	0.1455	0.9551	0.9553
0.1117	5.1151	2000	0.1526	0.9615	0.9616
0.1117	5.6266	2200	0.1521	0.9582	0.9583
0.0855	6.1381	2400	0.1516	0.9585	0.9587
0.0855	6.6496	2600	0.1610	0.9577	0.9580
0.0855	7.1611	2800	0.1592	0.9597	0.9599
0.0855	7.6726	3000	0.1707	0.9565	0.9565
0.0675	8.1841	3200	0.1708	0.9560	0.9562
0.0675	8.6957	3400	0.1833	0.9542	0.9546
0.0675	9.2072	3600	0.1713	0.9579	0.9579
0.0675	9.7187	3800	0.1749	0.9586	0.9587
0.0519	10.2302	4000	0.1781	0.9585	0.9587
0.0519	10.7417	4200	0.1996	0.9575	0.9576
0.0519	11.2532	4400	0.2032	0.9557	0.9560
0.0519	11.7647	4600	0.1813	0.9573	0.9576
0.0419	12.2762	4800	0.2248	0.9580	0.9582
0.0419	12.7877	5000	0.2166	0.9574	0.9576
0.0419	13.2992	5200	0.2183	0.9555	0.9557
0.0419	13.8107	5400	0.2312	0.9559	0.9561
0.0326	14.3223	5600	0.2248	0.9585	0.9586
0.0326	14.8338	5800	0.2627	0.9555	0.9557
0.0326	15.3453	6000	0.2449	0.9582	0.9583
0.0326	15.8568	6200	0.2393	0.9595	0.9596
0.0259	16.3683	6400	0.2676	0.9566	0.9568
0.0259	16.8798	6600	0.2590	0.9577	0.9579
0.0259	17.3913	6800	0.2616	0.9587	0.9589
0.0259	17.9028	7000	0.2765	0.9568	0.9568
0.0203	18.4143	7200	0.2862	0.9575	0.9576
0.0203	18.9258	7400	0.2857	0.9581	0.9582
0.0203	19.4373	7600	0.2859	0.9582	0.9583
0.0203	19.9488	7800	0.2879	0.9572	0.9573

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1