XLMRoBERTa_Lexical_Dataset55K

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1438
• Accuracy: 0.7808
• F1: 0.8360

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2317	200	0.5677	0.7620	0.8215
No log	0.4635	400	0.6684	0.6970	0.7782
No log	0.6952	600	0.7522	0.6589	0.7508
No log	0.9270	800	0.5695	0.7446	0.8111
0.4294	1.1587	1000	0.5555	0.7711	0.8287
0.4294	1.3905	1200	0.7692	0.6922	0.7750
0.4294	1.6222	1400	0.5649	0.7700	0.8281
0.4294	1.8540	1600	0.6370	0.7391	0.8076
0.2877	2.0857	1800	0.5635	0.7906	0.8416
0.2877	2.3175	2000	0.5465	0.7768	0.8326
0.2877	2.5492	2200	0.6197	0.7589	0.8210
0.2877	2.7810	2400	0.6086	0.7654	0.8254
0.2426	3.0127	2600	0.8355	0.6897	0.7733
0.2426	3.2445	2800	0.5750	0.7964	0.8454
0.2426	3.4762	3000	0.7292	0.7493	0.8147
0.2426	3.7080	3200	0.6157	0.7691	0.8279
0.2426	3.9397	3400	0.5626	0.8065	0.8524
0.2176	4.1715	3600	0.7033	0.7442	0.8115
0.2176	4.4032	3800	0.4640	0.8398	0.8734
0.2176	4.6350	4000	0.5459	0.8192	0.8605
0.2176	4.8667	4200	0.5352	0.7964	0.8458
0.1938	5.0985	4400	0.6397	0.7887	0.8409
0.1938	5.3302	4600	0.6416	0.8058	0.8521
0.1938	5.5620	4800	0.7778	0.7581	0.8208
0.1938	5.7937	5000	0.8658	0.7172	0.7929
0.1694	6.0255	5200	0.6958	0.7804	0.8357
0.1694	6.2572	5400	0.6935	0.7884	0.8408
0.1694	6.4890	5600	0.7117	0.7957	0.8457
0.1694	6.7207	5800	0.5838	0.8138	0.8572
0.1694	6.9525	6000	0.6154	0.8130	0.8569
0.1493	7.1842	6200	0.7858	0.7755	0.8324
0.1493	7.4160	6400	0.7709	0.7790	0.8348
0.1493	7.6477	6600	0.6795	0.8042	0.8512
0.1493	7.8795	6800	0.6621	0.8096	0.8545
0.1315	8.1112	7000	0.7189	0.8091	0.8543
0.1315	8.3430	7200	0.8954	0.7793	0.8348
0.1315	8.5747	7400	0.7296	0.8054	0.8520
0.1315	8.8065	7600	0.7626	0.8119	0.8563
0.1123	9.0382	7800	0.8372	0.7951	0.8454
0.1123	9.2700	8000	0.9260	0.7791	0.8348
0.1123	9.5017	8200	0.9010	0.7771	0.8336
0.1123	9.7335	8400	0.8107	0.8075	0.8532
0.1123	9.9652	8600	0.9045	0.7921	0.8435
0.1013	10.1970	8800	0.8423	0.8146	0.8581
0.1013	10.4287	9000	0.8836	0.8060	0.8524
0.1013	10.6605	9200	0.9896	0.7779	0.8342
0.1013	10.8922	9400	1.0312	0.7739	0.8314
0.0889	11.1240	9600	1.1268	0.7678	0.8275
0.0889	11.3557	9800	1.0607	0.7653	0.8258
0.0889	11.5875	10000	0.9427	0.7937	0.8445
0.0889	11.8192	10200	1.0853	0.7677	0.8274
0.0766	12.0510	10400	0.9392	0.7862	0.8396
0.0766	12.2827	10600	1.0988	0.7688	0.8281
0.0766	12.5145	10800	0.9423	0.8105	0.8553
0.0766	12.7462	11000	1.0569	0.7710	0.8295
0.0766	12.9780	11200	1.0698	0.7844	0.8384
0.0693	13.2097	11400	0.9822	0.7985	0.8477
0.0693	13.4415	11600	1.0140	0.8062	0.8527
0.0693	13.6732	11800	1.0212	0.7962	0.8461
0.0693	13.9050	12000	1.0648	0.7914	0.8430
0.0644	14.1367	12200	1.1202	0.7885	0.8411
0.0644	14.3685	12400	1.0868	0.7920	0.8434
0.0644	14.6002	12600	1.1419	0.7826	0.8373
0.0644	14.8320	12800	1.1438	0.7808	0.8360

Framework versions

• Transformers 4.44.2
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1