vit-base-patch16-224-ve-U16-b-80

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5265
• Accuracy: 0.8696

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	8	1.3828	0.4565
1.3846	2.0	16	1.3610	0.5
1.3611	3.0	24	1.2967	0.4348
1.2759	4.0	32	1.1830	0.3913
1.1164	5.0	40	1.0824	0.3696
1.1164	6.0	48	0.9665	0.5
0.98	7.0	56	0.9036	0.5652
0.8533	8.0	64	0.8348	0.7826
0.7321	9.0	72	0.7397	0.8261
0.6075	10.0	80	0.7155	0.7174
0.6075	11.0	88	0.6006	0.8261
0.4901	12.0	96	0.5265	0.8696
0.3967	13.0	104	0.5214	0.8043
0.2746	14.0	112	0.5433	0.7826
0.2366	15.0	120	0.6141	0.7826
0.2366	16.0	128	0.6658	0.7826
0.2247	17.0	136	0.6327	0.7609
0.2047	18.0	144	0.5339	0.8261
0.1592	19.0	152	0.6647	0.8043
0.1349	20.0	160	0.7483	0.7609
0.1349	21.0	168	0.7387	0.8043
0.1285	22.0	176	0.8261	0.7609
0.1104	23.0	184	0.7151	0.8043
0.1191	24.0	192	0.7785	0.7609
0.1074	25.0	200	0.8902	0.7391
0.1074	26.0	208	0.7757	0.7826
0.0947	27.0	216	0.7157	0.7826
0.0973	28.0	224	0.8198	0.7826
0.0992	29.0	232	0.7240	0.8261
0.0766	30.0	240	0.6993	0.8043
0.0766	31.0	248	0.5688	0.8261
0.0606	32.0	256	0.6202	0.8478
0.0633	33.0	264	0.6740	0.8261
0.0681	34.0	272	0.6782	0.8261
0.0591	35.0	280	0.8370	0.7826
0.0591	36.0	288	0.6995	0.8261
0.0731	37.0	296	0.7560	0.8261
0.0618	38.0	304	0.6730	0.8261
0.0543	39.0	312	0.7166	0.8261
0.0574	40.0	320	0.7332	0.8261
0.0574	41.0	328	0.6982	0.8261
0.0707	42.0	336	0.7183	0.7826
0.0646	43.0	344	0.7568	0.8043
0.0476	44.0	352	0.8521	0.8043
0.047	45.0	360	0.8992	0.8043
0.047	46.0	368	0.8749	0.7826
0.0406	47.0	376	0.9928	0.7826
0.0361	48.0	384	0.9659	0.7826
0.042	49.0	392	0.8839	0.8043
0.0421	50.0	400	0.8613	0.7391
0.0421	51.0	408	0.9006	0.7826
0.0396	52.0	416	0.8627	0.7826
0.0255	53.0	424	0.8717	0.7609
0.0359	54.0	432	1.0508	0.7609
0.0424	55.0	440	0.9745	0.7826
0.0424	56.0	448	0.9511	0.8043
0.0364	57.0	456	0.9239	0.8043
0.0444	58.0	464	0.9500	0.7826
0.0445	59.0	472	0.9266	0.8261
0.0368	60.0	480	0.9346	0.8043
0.0368	61.0	488	0.9513	0.8043
0.0278	62.0	496	0.9505	0.8043
0.0324	63.0	504	0.9625	0.8261
0.0308	64.0	512	0.9720	0.8261
0.0185	65.0	520	0.9515	0.8043
0.0185	66.0	528	0.9278	0.8043
0.0323	67.0	536	0.9315	0.8261
0.0251	68.0	544	0.9794	0.8043
0.0297	69.0	552	1.0378	0.7609
0.0257	70.0	560	1.0336	0.7609
0.0257	71.0	568	1.0577	0.7826
0.02	72.0	576	1.0332	0.8043
0.0226	73.0	584	1.0165	0.8043
0.0257	74.0	592	1.0194	0.8043
0.0232	75.0	600	1.0026	0.8043
0.0232	76.0	608	1.0073	0.8043
0.0274	77.0	616	1.0099	0.8043
0.0182	78.0	624	1.0170	0.8043
0.0375	79.0	632	1.0139	0.8043
0.029	80.0	640	1.0128	0.8043

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0