jialicheng/cifar10_vit-base

vit-base

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the cifar10 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0452
• Accuracy: 0.9892

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Accuracy	Validation Loss
No log	1.0	333	0.9599	0.7045
1.2275	2.0	666	0.9721	0.3862
1.2275	3.0	999	0.9771	0.2734
0.4176	4.0	1332	0.9794	0.2127
0.3859	5.0	1665	0.9822	0.1720
0.3859	6.0	1998	0.9834	0.1459
0.32	7.0	2331	0.9843	0.1241
0.2795	8.0	2664	0.9846	0.1106
0.2795	9.0	2997	0.9861	0.0951
0.2489	10.0	3330	0.9856	0.0877
0.2284	11.0	3663	0.987	0.0783
0.2284	12.0	3996	0.9861	0.0743
0.2139	13.0	4329	0.9883	0.0666
0.2019	14.0	4662	0.9862	0.0654
0.2019	15.0	4995	0.9875	0.0608
0.1882	16.0	5328	0.9875	0.0594
0.1845	17.0	5661	0.9878	0.0545
0.1845	18.0	5994	0.9885	0.0534
0.1762	19.0	6327	0.9876	0.0562
0.1629	20.0	6660	0.9879	0.0510
0.1629	21.0	6993	0.9889	0.0488
0.1622	22.0	7326	0.9879	0.0489
0.1621	23.0	7659	0.9881	0.0482
0.1621	24.0	7992	0.9886	0.0464
0.1518	25.0	8325	0.9887	0.0464
0.151	26.0	8658	0.9884	0.0477
0.151	27.0	8991	0.9886	0.0471
0.1486	28.0	9324	0.9882	0.0489
0.147	29.0	9657	0.9884	0.0477
0.147	30.0	9990	0.0494	0.9883
0.1412	31.0	10323	0.0467	0.9881
0.1403	32.0	10656	0.0444	0.9888
0.1403	33.0	10989	0.0451	0.9888
0.1373	34.0	11322	0.0464	0.9887
0.1379	35.0	11655	0.0438	0.9896
0.1379	36.0	11988	0.0440	0.9887
0.1375	37.0	12321	0.0460	0.9881
0.1377	38.0	12654	0.0435	0.9896
0.1377	39.0	12987	0.0461	0.989
0.1332	40.0	13320	0.0442	0.9897
0.1306	41.0	13653	0.0463	0.9894
0.1306	42.0	13986	0.0449	0.9892
0.1289	43.0	14319	0.0456	0.989
0.128	44.0	14652	0.0451	0.9892
0.128	45.0	14985	0.0454	0.9889
0.1321	46.0	15318	0.0445	0.9895
0.1222	47.0	15651	0.0467	0.9893
0.1222	48.0	15984	0.0465	0.9897
0.122	49.0	16317	0.0452	0.9896
0.123	50.0	16650	0.0478	0.9894
0.123	51.0	16983	0.0465	0.9892
0.1194	52.0	17316	0.0488	0.9887
0.1209	53.0	17649	0.0472	0.9892
0.1209	54.0	17982	0.0456	0.9897
0.1212	55.0	18315	0.0466	0.9893
0.1187	56.0	18648	0.0458	0.9894
0.1187	57.0	18981	0.0447	0.9899
0.1193	58.0	19314	0.0419	0.9892
0.119	59.0	19647	0.0431	0.9897
0.119	60.0	19980	0.0437	0.9894
0.1165	61.0	20313	0.0470	0.9889
0.1146	62.0	20646	0.0472	0.989
0.1146	63.0	20979	0.0445	0.9894
0.1147	64.0	21312	0.0454	0.9894
0.1117	65.0	21645	0.0446	0.9899
0.1117	66.0	21978	0.0482	0.989
0.1137	67.0	22311	0.0458	0.9895
0.1145	68.0	22644	0.0462	0.989
0.1145	69.0	22977	0.0461	0.9894
0.1136	70.0	23310	0.0455	0.9894
0.1144	71.0	23643	0.0455	0.9896
0.1144	72.0	23976	0.0458	0.9891
0.1126	73.0	24309	0.0462	0.989
0.1065	74.0	24642	0.0463	0.9894
0.1065	75.0	24975	0.0461	0.9895
0.1136	76.0	25308	0.0462	0.9893
0.1117	77.0	25641	0.0454	0.9886
0.1117	78.0	25974	0.0456	0.9889
0.1106	79.0	26307	0.0454	0.9887
0.1085	80.0	26640	0.0458	0.9893
0.1085	81.0	26973	0.0458	0.9892
0.107	82.0	27306	0.0450	0.9894
0.1112	83.0	27639	0.0438	0.9896
0.1112	84.0	27972	0.0453	0.9891
0.1073	85.0	28305	0.0445	0.9893
0.1103	86.0	28638	0.0444	0.9892
0.1103	87.0	28971	0.0443	0.9891
0.1074	88.0	29304	0.0460	0.9893
0.1041	89.0	29637	0.0455	0.9891
0.1041	90.0	29970	0.0440	0.9894
0.1054	91.0	30303	0.0453	0.9894
0.1069	92.0	30636	0.0451	0.989
0.1069	93.0	30969	0.0449	0.9894
0.1056	94.0	31302	0.0457	0.9892
0.1069	95.0	31635	0.0449	0.9892
0.1069	96.0	31968	0.0450	0.9892
0.1053	97.0	32301	0.0449	0.9896
0.1068	98.0	32634	0.0453	0.9893
0.1068	99.0	32967	0.0453	0.9891
0.1059	100.0	33300	0.0452	0.9892

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu118
• Datasets 2.18.0
• Tokenizers 0.15.2