beit-base-patch16-224-dmae-va-U5-42C

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9081
• Accuracy: 0.65

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	2.0475	0.1
2.0435	1.94	15	1.8868	0.1
1.9821	2.97	23	1.6588	0.1333
1.7075	4.0	31	1.4345	0.3
1.7075	4.9	38	1.2973	0.4
1.5153	5.94	46	1.2245	0.4167
1.4239	6.97	54	1.1969	0.5167
1.354	8.0	62	1.1733	0.4833
1.354	8.9	69	1.1552	0.5167
1.2885	9.94	77	1.1274	0.55
1.2549	10.97	85	1.1119	0.55
1.2378	12.0	93	1.1023	0.5667
1.1994	12.9	100	1.0860	0.5833
1.1994	13.94	108	1.0694	0.6
1.1535	14.97	116	1.0548	0.6167
1.1354	16.0	124	1.0402	0.6167
1.1031	16.9	131	1.0266	0.6167
1.1031	17.94	139	1.0156	0.6167
1.1006	18.97	147	1.0024	0.6167
1.0655	20.0	155	0.9867	0.6167
1.0439	20.9	162	0.9770	0.6167
1.0335	21.94	170	0.9662	0.6
1.0335	22.97	178	0.9590	0.6
1.0152	24.0	186	0.9504	0.6167
1.01	24.9	193	0.9433	0.6333
0.9779	25.94	201	0.9395	0.6333
0.9779	26.97	209	0.9354	0.65
0.9792	28.0	217	0.9292	0.65
0.9591	28.9	224	0.9257	0.65
0.9643	29.94	232	0.9211	0.65
0.9453	30.97	240	0.9172	0.65
0.9453	32.0	248	0.9150	0.65
0.938	32.9	255	0.9129	0.65
0.9206	33.94	263	0.9105	0.65
0.9412	34.97	271	0.9096	0.65
0.9412	36.0	279	0.9087	0.65
0.9363	36.9	286	0.9083	0.65
0.9356	37.94	294	0.9081	0.65

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.15.2