Revision_PhoBert_Lexical_Dataset_52k

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5781
• Accuracy: 0.8625
• F1: 0.8563

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2421	200	0.3674	0.8375	0.8287
No log	0.4843	400	0.3602	0.8327	0.8275
No log	0.7264	600	0.3506	0.8261	0.8216
No log	0.9685	800	0.3331	0.8388	0.8343
0.4364	1.2107	1000	0.2985	0.8626	0.8557
0.4364	1.4528	1200	0.2868	0.8711	0.8633
0.4364	1.6949	1400	0.3085	0.8582	0.8526
0.4364	1.9370	1600	0.3204	0.8398	0.8354
0.3331	2.1792	1800	0.3007	0.8635	0.8576
0.3331	2.4213	2000	0.2833	0.8707	0.8640
0.3331	2.6634	2200	0.3191	0.8430	0.8377
0.3331	2.9056	2400	0.2676	0.8764	0.8691
0.2800	3.1477	2600	0.3069	0.8547	0.8495
0.2800	3.3898	2800	0.2911	0.8743	0.8677
0.2800	3.6320	3000	0.2854	0.8705	0.8648
0.2800	3.8741	3200	0.3088	0.8644	0.8586
0.2423	4.1162	3400	0.3385	0.8553	0.8505
0.2423	4.3584	3600	0.3006	0.8806	0.8739
0.2423	4.6005	3800	0.2974	0.8809	0.8741
0.2423	4.8426	4000	0.2867	0.8783	0.8718
0.2120	5.0847	4200	0.3426	0.8658	0.8599
0.2120	5.3269	4400	0.3727	0.8592	0.8542
0.2120	5.5690	4600	0.3409	0.8699	0.8643
0.2120	5.8111	4800	0.3793	0.8575	0.8524
0.1843	6.0533	5000	0.3463	0.8750	0.8686
0.1843	6.2954	5200	0.4115	0.8578	0.8525
0.1843	6.5375	5400	0.4067	0.8587	0.8535
0.1843	6.7797	5600	0.3832	0.8650	0.8594
0.1584	7.0218	5800	0.4082	0.8647	0.8591
0.1584	7.2639	6000	0.4192	0.8617	0.8562
0.1584	7.5061	6200	0.4261	0.8587	0.8531
0.1584	7.7482	6400	0.4080	0.8686	0.8624
0.1351	7.9903	6600	0.4212	0.8529	0.8474
0.1351	8.2324	6800	0.4599	0.8583	0.8525
0.1351	8.4746	7000	0.4316	0.8636	0.8579
0.1351	8.7167	7200	0.4833	0.8573	0.8515
0.1351	8.9588	7400	0.4489	0.8573	0.8518
0.1172	9.2010	7600	0.4566	0.8604	0.8541
0.1172	9.4431	7800	0.4845	0.8610	0.8554
0.1172	9.6852	8000	0.4640	0.8627	0.8567
0.1172	9.9274	8200	0.4370	0.8684	0.8618
0.0998	10.1695	8400	0.4858	0.8617	0.8559
0.0998	10.4116	8600	0.4793	0.8646	0.8583
0.0998	10.6538	8800	0.4999	0.8617	0.8557
0.0998	10.8959	9000	0.4840	0.8665	0.8603
0.0895	11.1380	9200	0.4993	0.8650	0.8592
0.0895	11.3801	9400	0.5160	0.8625	0.8564
0.0895	11.6223	9600	0.5123	0.8643	0.8583
0.0895	11.8644	9800	0.5381	0.8610	0.8554
0.0780	12.1065	10000	0.5519	0.8615	0.8557
0.0780	12.3487	10200	0.5642	0.8605	0.8548
0.0780	12.5908	10400	0.5579	0.8593	0.8536
0.0780	12.8329	10600	0.5436	0.8626	0.8566
0.0681	13.0751	10800	0.5843	0.8619	0.8561
0.0681	13.3172	11000	0.5579	0.8633	0.8571
0.0681	13.5593	11200	0.5749	0.8602	0.8543
0.0681	13.8015	11400	0.5593	0.8650	0.8587
0.0633	14.0436	11600	0.5817	0.8611	0.8552
0.0633	14.2857	11800	0.5724	0.8624	0.8562
0.0633	14.5278	12000	0.5760	0.8626	0.8565
0.0633	14.7700	12200	0.5781	0.8625	0.8563

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2