torgo_xlsr_finetune_M05_keep_all

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6032
• Wer: 0.2230

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.6054	0.55	1000	3.4415	1.0
3.1884	1.1	2000	2.7133	1.0
1.25	1.65	3000	1.5085	0.7190
0.8201	2.2	4000	1.5053	0.5696
0.6758	2.75	5000	1.3675	0.5063
0.542	3.3	6000	1.4764	0.4445
0.5344	3.85	7000	1.4226	0.4293
0.4464	4.4	8000	1.5430	0.3816
0.4191	4.95	9000	1.5657	0.3835
0.402	5.5	10000	1.5064	0.3610
0.332	6.05	11000	1.8348	0.3519
0.335	6.6	12000	1.3122	0.3408
0.3045	7.15	13000	1.4435	0.3126
0.2982	7.7	14000	1.4392	0.3347
0.2706	8.25	15000	1.7165	0.3481
0.2618	8.8	16000	1.3418	0.2856
0.2422	9.35	17000	1.4707	0.3122
0.2396	9.9	18000	1.6035	0.3035
0.248	10.45	19000	1.3132	0.2375
0.2272	10.99	20000	1.5060	0.2722
0.194	11.54	21000	1.4504	0.2650
0.2096	12.09	22000	1.7180	0.2871
0.2137	12.64	23000	1.6053	0.2676
0.1929	13.19	24000	1.5596	0.2577
0.1811	13.74	25000	1.6539	0.2600
0.1749	14.29	26000	1.7676	0.2570
0.1697	14.84	27000	1.7116	0.2577
0.1659	15.39	28000	1.7506	0.2551
0.1625	15.94	29000	1.6863	0.2566
0.1603	16.49	30000	1.6332	0.2406
0.1497	17.04	31000	1.4480	0.2307
0.1503	17.59	32000	1.5508	0.2238
0.1349	18.14	33000	1.6107	0.2280
0.1342	18.69	34000	1.5869	0.2280
0.1309	19.24	35000	1.5790	0.2268
0.1319	19.79	36000	1.6032	0.2230

Framework versions

• Transformers 4.26.1
• Pytorch 2.2.0
• Datasets 2.16.1
• Tokenizers 0.13.3