bert-30M-uncased-classification-CMC-fqa

This model is a fine-tuned version of vietgpt/bert-30M-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4703
• Accuracy: 0.8739

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	130	5.3723	0.0043
No log	2.0	260	5.3750	0.0043
No log	3.0	390	5.3555	0.0
5.3407	4.0	520	5.2599	0.0174
5.3407	5.0	650	4.9584	0.0261
5.3407	6.0	780	4.7092	0.0609
5.3407	7.0	910	4.4852	0.1087
4.7451	8.0	1040	4.2827	0.1696
4.7451	9.0	1170	4.0794	0.2478
4.7451	10.0	1300	3.8857	0.2913
4.7451	11.0	1430	3.6879	0.4
3.8815	12.0	1560	3.5051	0.4435
3.8815	13.0	1690	3.3316	0.4652
3.8815	14.0	1820	3.1648	0.4913
3.8815	15.0	1950	2.9991	0.5261
3.1326	16.0	2080	2.8421	0.5652
3.1326	17.0	2210	2.6915	0.5913
3.1326	18.0	2340	2.5519	0.6043
3.1326	19.0	2470	2.4192	0.6478
2.4835	20.0	2600	2.2918	0.6870
2.4835	21.0	2730	2.1705	0.7043
2.4835	22.0	2860	2.0567	0.7261
2.4835	23.0	2990	1.9522	0.7261
1.9554	24.0	3120	1.8542	0.7391
1.9554	25.0	3250	1.7546	0.7696
1.9554	26.0	3380	1.6647	0.7609
1.5347	27.0	3510	1.5819	0.7739
1.5347	28.0	3640	1.5082	0.7870
1.5347	29.0	3770	1.4383	0.7957
1.5347	30.0	3900	1.3742	0.8
1.1984	31.0	4030	1.3075	0.8043
1.1984	32.0	4160	1.2476	0.8043
1.1984	33.0	4290	1.1953	0.8043
1.1984	34.0	4420	1.1515	0.8087
0.9448	35.0	4550	1.0959	0.8174
0.9448	36.0	4680	1.0462	0.8174
0.9448	37.0	4810	1.0107	0.8174
0.9448	38.0	4940	0.9778	0.8087
0.7518	39.0	5070	0.9337	0.8217
0.7518	40.0	5200	0.9048	0.8261
0.7518	41.0	5330	0.8726	0.8261
0.7518	42.0	5460	0.8452	0.8348
0.6032	43.0	5590	0.8161	0.8391
0.6032	44.0	5720	0.7944	0.8348
0.6032	45.0	5850	0.7719	0.8565
0.6032	46.0	5980	0.7545	0.8652
0.4828	47.0	6110	0.7288	0.8609
0.4828	48.0	6240	0.7109	0.8652
0.4828	49.0	6370	0.6962	0.8696
0.3959	50.0	6500	0.6831	0.8696
0.3959	51.0	6630	0.6634	0.8652
0.3959	52.0	6760	0.6495	0.8739
0.3959	53.0	6890	0.6453	0.8739
0.3252	54.0	7020	0.6261	0.8783
0.3252	55.0	7150	0.6167	0.8739
0.3252	56.0	7280	0.6052	0.8783
0.3252	57.0	7410	0.5975	0.8870
0.2733	58.0	7540	0.5831	0.8826
0.2733	59.0	7670	0.5768	0.8739
0.2733	60.0	7800	0.5668	0.8783
0.2733	61.0	7930	0.5636	0.8739
0.231	62.0	8060	0.5498	0.8826
0.231	63.0	8190	0.5495	0.8739
0.231	64.0	8320	0.5413	0.8826
0.231	65.0	8450	0.5327	0.8870
0.1956	66.0	8580	0.5300	0.8826
0.1956	67.0	8710	0.5254	0.8783
0.1956	68.0	8840	0.5159	0.8826
0.1956	69.0	8970	0.5158	0.8826
0.1671	70.0	9100	0.5136	0.8870
0.1671	71.0	9230	0.5151	0.8826
0.1671	72.0	9360	0.5118	0.8783
0.1671	73.0	9490	0.5056	0.8783
0.1465	74.0	9620	0.5051	0.8783
0.1465	75.0	9750	0.5023	0.8826
0.1465	76.0	9880	0.4938	0.8783
0.1292	77.0	10010	0.4982	0.8826
0.1292	78.0	10140	0.4951	0.8870
0.1292	79.0	10270	0.4931	0.8826
0.1292	80.0	10400	0.4858	0.8783
0.1153	81.0	10530	0.4854	0.8783
0.1153	82.0	10660	0.4872	0.8826
0.1153	83.0	10790	0.4856	0.8783
0.1153	84.0	10920	0.4862	0.8826
0.1056	85.0	11050	0.4829	0.8783
0.1056	86.0	11180	0.4790	0.8870
0.1056	87.0	11310	0.4757	0.8739
0.1056	88.0	11440	0.4732	0.8783
0.1	89.0	11570	0.4764	0.8783
0.1	90.0	11700	0.4748	0.8739
0.1	91.0	11830	0.4751	0.8739
0.1	92.0	11960	0.4728	0.8739
0.0937	93.0	12090	0.4744	0.8739
0.0937	94.0	12220	0.4738	0.8739
0.0937	95.0	12350	0.4720	0.8739
0.0937	96.0	12480	0.4713	0.8739
0.0883	97.0	12610	0.4703	0.8739
0.0883	98.0	12740	0.4709	0.8739
0.0883	99.0	12870	0.4702	0.8739
0.0854	100.0	13000	0.4703	0.8739

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1