wav2vec2-demo-M01-2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8002
• Wer: 0.9717

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
23.0778	0.9	500	3.3183	1.0
3.3341	1.8	1000	3.1046	1.0
2.8465	2.7	1500	2.7012	1.0
2.4301	3.6	2000	1.9785	1.3025
1.5285	4.5	2500	1.5775	1.4170
1.055	5.41	3000	1.5070	1.3753
0.8075	6.31	3500	1.3811	1.2509
0.6867	7.21	4000	1.2362	1.2007
0.566	8.11	4500	1.6537	1.2064
0.4967	9.01	5000	1.5271	1.1562
0.4337	9.91	5500	1.2359	1.1046
0.3941	10.81	6000	1.5255	1.1237
0.3612	11.71	6500	1.3686	1.0890
0.3237	12.61	7000	1.1992	1.0629
0.3085	13.51	7500	1.5883	1.0834
0.3003	14.41	8000	1.7352	1.0686
0.2859	15.32	8500	1.4790	0.9958
0.2495	16.22	9000	1.6757	1.0155
0.2329	17.12	9500	1.5789	1.0283
0.2241	18.02	10000	1.5147	0.9922
0.215	18.92	10500	1.4684	0.9965
0.202	19.82	11000	1.5099	0.9760
0.1942	20.72	11500	1.8195	0.9837
0.1627	21.62	12000	1.8089	1.0021
0.1782	22.52	12500	1.6944	0.9710
0.1566	23.42	13000	1.5881	0.9682
0.1471	24.32	13500	1.6741	0.9654
0.1355	25.23	14000	1.6183	0.9576
0.1253	26.13	14500	1.5730	0.9696
0.1245	27.03	15000	1.6876	0.9689
0.1258	27.93	15500	1.7535	0.9802
0.1217	28.83	16000	1.7710	0.9661
0.1074	29.73	16500	1.8002	0.9717

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 1.18.3
• Tokenizers 0.13.2