swin-ena24

This model is a fine-tuned version of microsoft/swin-base-simmim-window6-192 on the ena24 dataset. It achieves the following results on the evaluation set:

• Loss: 2.4677
• Accuracy: 0.5146
• F1 Macro: 0.4328

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
1.9888	0.2519	100	3.4122	0.1631	0.0893
1.6111	0.5038	200	2.9077	0.2578	0.1535
1.1276	0.7557	300	2.6504	0.3574	0.2827
1.0234	1.0076	400	2.5728	0.3906	0.3156
0.8909	1.2594	500	2.5007	0.4219	0.3388
0.8008	1.5113	600	2.7039	0.4043	0.3619
0.6885	1.7632	700	3.1090	0.3701	0.2926
0.839	2.0151	800	2.5845	0.4844	0.4149
0.325	2.2670	900	2.5143	0.5068	0.4128
0.4501	2.5189	1000	2.7684	0.4482	0.4056
0.3191	2.7708	1100	2.4677	0.5146	0.4328
0.1664	3.0227	1200	2.4777	0.5361	0.4597
0.1469	3.2746	1300	2.6403	0.5205	0.4495
0.3063	3.5264	1400	2.8000	0.5010	0.4415
0.1786	3.7783	1500	2.8165	0.5332	0.4525
0.0687	4.0302	1600	2.9027	0.5684	0.4942
0.0427	4.2821	1700	3.3216	0.4912	0.4362
0.1825	4.5340	1800	3.1456	0.5312	0.4664
0.0758	4.7859	1900	3.2782	0.5547	0.4578
0.0471	5.0378	2000	3.3348	0.5518	0.4725
0.0512	5.2897	2100	3.7182	0.5283	0.4514
0.0095	5.5416	2200	3.9028	0.5342	0.4785
0.0247	5.7935	2300	3.9606	0.5713	0.4879
0.0008	6.0453	2400	4.1290	0.5654	0.4918
0.0024	6.2972	2500	4.4147	0.5654	0.4863
0.0002	6.5491	2600	4.5209	0.5654	0.4913
0.0055	6.8010	2700	4.5154	0.5820	0.5067

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1