wav2vec2-darija-tamazigh-test-1000-rows-test-2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.4860
• Wer: 1.0
• Cer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
13.7722	1.0	15	14.9896	1.0	1.7062
13.6893	2.0	30	14.9863	1.0	1.7081
13.7519	3.0	45	14.9802	1.0	1.7104
13.4976	4.0	60	14.9715	1.0	1.7187
13.3519	5.0	75	14.9601	1.0	1.7284
13.2141	6.0	90	14.9461	1.0026	1.7279
13.5479	7.0	105	14.9295	1.0026	1.7344
13.79	8.0	120	14.9103	1.0077	1.7709
13.2426	9.0	135	14.8878	1.0153	1.7949
13.3078	10.0	150	14.8632	1.0588	1.8055
13.545	11.0	165	14.8358	1.1253	1.8120
13.2942	12.0	180	14.8058	1.3018	1.8319
13.3046	13.0	195	14.7736	1.7647	1.8494
14.0132	14.0	210	14.7377	2.4297	1.8351
13.2155	15.0	225	14.6978	3.1841	1.7843
13.1875	16.0	240	14.6533	3.5090	1.5875
13.1263	17.0	255	14.6048	3.4348	1.3376
13.6729	18.0	270	14.5528	3.1995	1.1464
13.3298	19.0	285	14.4966	2.6701	0.9316
13.2721	20.0	300	14.4354	1.9591	0.8494
12.7355	21.0	315	14.3687	1.3248	0.8707
13.1534	22.0	330	14.2947	1.1049	0.9284
12.5264	23.0	345	14.2119	1.0384	0.9718
13.0372	24.0	360	14.1252	1.0205	0.9871
12.5199	25.0	375	14.0310	1.0	0.9963
12.7322	26.0	390	13.9320	1.0	0.9991
12.538	27.0	405	13.8167	1.0	1.0
12.491	28.0	420	13.6856	1.0	1.0
12.0785	29.0	435	13.5270	1.0	1.0
11.8788	30.0	450	13.3362	1.0	1.0
11.752	31.0	465	13.1166	1.0	1.0
11.1579	32.0	480	12.8463	1.0	1.0
11.5395	33.0	495	12.4823	1.0	1.0
10.9719	34.0	510	12.1452	1.0	1.0
10.8553	35.0	525	11.8524	1.0	1.0
10.5334	36.0	540	11.6063	1.0	1.0
10.6811	37.0	555	11.4928	1.0	1.0
11.6902	37.3448	560	11.4860	1.0	1.0

Framework versions

• Transformers 4.48.3
• Pytorch 2.1.0+cu121
• Datasets 2.21.0
• Tokenizers 0.21.2