distilhubert-finetuned-gtzan

This model is a fine-tuned version of ntu-spml/distilhubert on the GTZAN dataset. It achieves the following results on the evaluation set:

• Loss: 0.5690
• Accuracy: 0.87

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.2968	1.0	57	1.2136	0.7
1.0931	2.0	114	1.1346	0.7
0.9362	3.0	171	0.9992	0.76
0.948	4.0	228	0.9344	0.76
0.7033	5.0	285	0.7802	0.81
0.6625	6.0	342	0.7777	0.79
0.5627	7.0	399	0.7143	0.81
0.5081	8.0	456	0.6232	0.86
0.4635	9.0	513	0.6564	0.85
0.3347	10.0	570	0.6108	0.85
0.2895	11.0	627	0.7139	0.8
0.2493	12.0	684	0.5887	0.84
0.2673	13.0	741	0.5907	0.86
0.1949	14.0	798	0.5798	0.83
0.1541	15.0	855	0.5532	0.87
0.1913	16.0	912	0.5314	0.87
0.1339	17.0	969	0.5337	0.88
0.0876	18.0	1026	0.5815	0.87
0.0713	19.0	1083	0.5847	0.85
0.0869	20.0	1140	0.5456	0.86
0.0587	21.0	1197	0.5480	0.86
0.0524	22.0	1254	0.5534	0.87
0.0621	23.0	1311	0.5707	0.87
0.0452	24.0	1368	0.5748	0.87
0.0464	25.0	1425	0.5690	0.87

Framework versions

• Transformers 4.34.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3