wav2vec2-darija-tamazigh-test-41000-rows-test-aya-0

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1948
• Wer: 0.4682
• Cer: 0.1239

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.4778	1.0	1215	3.4581	1.0	1.0
2.7994	2.0	2430	2.6753	0.9991	0.9291
1.7791	3.0	3645	1.3744	0.9259	0.3614
1.4077	4.0	4860	0.9562	0.8190	0.2729
1.3074	5.0	6075	0.7549	0.7649	0.2454
1.1844	6.0	7290	0.6490	0.7274	0.2283
0.951	7.0	8505	0.5654	0.6889	0.2147
0.9808	8.0	9720	0.5096	0.6691	0.2088
0.8549	9.0	10935	0.4653	0.6494	0.1970
0.8519	10.0	12150	0.4340	0.6322	0.1896
0.7373	11.0	13365	0.4118	0.6156	0.1847
0.8522	12.0	14580	0.3851	0.6075	0.1769
0.8136	13.0	15795	0.3668	0.5965	0.1718
0.7852	14.0	17010	0.3506	0.5892	0.1674
0.7249	15.0	18225	0.3321	0.5761	0.1633
0.6739	16.0	19440	0.3200	0.5692	0.1609
0.7185	17.0	20655	0.3103	0.5633	0.1581
0.8025	18.0	21870	0.2980	0.5461	0.1527
0.6638	19.0	23085	0.2852	0.5385	0.1508
0.6429	20.0	24300	0.2795	0.5323	0.1478
0.684	21.0	25515	0.2726	0.5228	0.1445
0.623	22.0	26730	0.2642	0.5192	0.1421
0.698	23.0	27945	0.2603	0.5194	0.1442
0.5648	24.0	29160	0.2528	0.5158	0.1427
0.6505	25.0	30375	0.2457	0.5091	0.1389
0.6514	26.0	31590	0.2406	0.5060	0.1376
0.5719	27.0	32805	0.2364	0.4957	0.1341
0.5836	28.0	34020	0.2318	0.5007	0.1356
0.5665	29.0	35235	0.2289	0.4993	0.1353
0.6011	30.0	36450	0.2251	0.4852	0.1311
0.5222	31.0	37665	0.2245	0.4868	0.1312
0.5416	32.0	38880	0.2201	0.4881	0.1309
0.5853	33.0	40095	0.2168	0.4857	0.1297
0.5465	34.0	41310	0.2144	0.4878	0.1304
0.5231	35.0	42525	0.2132	0.4889	0.1320
0.5181	36.0	43740	0.2099	0.4811	0.1281
0.49	37.0	44955	0.2092	0.4821	0.1278
0.5351	38.0	46170	0.2073	0.4773	0.1277
0.5507	39.0	47385	0.2042	0.4743	0.1255
0.5776	40.0	48600	0.2019	0.4777	0.1266
0.4353	41.0	49815	0.2011	0.4736	0.1265
0.5359	42.0	51030	0.2001	0.4719	0.1249
0.4706	43.0	52245	0.1986	0.4707	0.1250
0.4466	44.0	53460	0.1980	0.4726	0.1255
0.5137	45.0	54675	0.1966	0.4717	0.1242
0.5533	46.0	55890	0.1961	0.4677	0.1237
0.5464	47.0	57105	0.1950	0.4690	0.1235
0.4453	48.0	58320	0.1949	0.4664	0.1229
0.4856	49.0	59535	0.1952	0.4682	0.1241
0.5914	50.0	60750	0.1948	0.4682	0.1239

Framework versions

• Transformers 4.48.3
• Pytorch 2.1.0+cu121
• Datasets 2.21.0
• Tokenizers 0.21.2