distilhubert-finetuned-gtzan

This model is a fine-tuned version of ntu-spml/distilhubert on the GTZAN dataset. It achieves the following results on the evaluation set:

• Loss: 1.2454
• Accuracy: 0.82

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.2
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.2107	1.0	112	2.2411	0.31
2.0193	2.0	225	1.9900	0.53
1.7491	3.0	337	1.6436	0.59
1.5096	4.0	450	1.3625	0.63
0.9801	5.0	562	1.0769	0.75
0.8603	6.0	675	0.9399	0.78
0.5573	7.0	787	0.8290	0.77
0.5776	8.0	900	0.6834	0.82
0.4687	9.0	1012	0.6522	0.82
0.3513	10.0	1125	0.6564	0.82
0.1691	11.0	1237	0.6628	0.84
0.0384	12.0	1350	0.8602	0.81
0.0218	13.0	1462	0.8367	0.85
0.0057	14.0	1575	0.9951	0.83
0.0041	15.0	1687	1.0021	0.84
0.0027	16.0	1800	1.0215	0.82
0.0021	17.0	1912	0.9737	0.83
0.0017	18.0	2025	1.0321	0.85
0.0015	19.0	2137	0.9519	0.81
0.0013	20.0	2250	0.9298	0.82
0.0011	21.0	2362	0.9627	0.83
0.001	22.0	2475	1.1373	0.82
0.0009	23.0	2587	1.0855	0.83
0.0008	24.0	2700	0.9979	0.81
0.0008	25.0	2812	1.0956	0.82
0.0009	26.0	2925	0.9861	0.82
0.0007	27.0	3037	1.1387	0.83
0.0006	28.0	3150	1.1965	0.83
0.0006	29.0	3262	1.1527	0.81
0.0007	30.0	3375	1.0609	0.82
0.0006	31.0	3487	1.1770	0.81
0.0801	32.0	3600	1.2290	0.82
0.0005	33.0	3712	1.1785	0.83
0.0005	34.0	3825	1.2154	0.83
0.0004	35.0	3937	1.2250	0.83
0.0004	36.0	4050	1.2280	0.82
0.0004	37.0	4162	1.2364	0.83
0.0004	38.0	4275	1.2379	0.82
0.0004	39.0	4387	1.2483	0.83
0.0004	39.82	4480	1.2454	0.82

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1
• Datasets 2.12.0
• Tokenizers 0.13.2