khier12/800min_wav2vec_xls-r53_FT_Algerian_Dialect

800min_wav2vec_xls-r53_FT

This model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-arabic on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7240
• Wer: 0.4972
• Cer: 0.1568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
1.7059	0.9153	100	0.2935	1.0710	0.8356
1.1356	1.8307	200	0.2471	0.8798	0.7414
0.9796	2.7460	300	0.2196	0.7998	0.6757
0.9021	3.6613	400	0.2032	0.7767	0.6317
0.8241	4.5767	500	0.1919	0.7197	0.5982
0.7813	5.4920	600	0.1872	0.7161	0.5866
0.7297	6.4073	700	0.1809	0.6938	0.5681
0.6929	7.3227	800	0.1763	0.7047	0.5579
0.6638	8.2380	900	0.1727	0.7087	0.5484
0.6292	9.1533	1000	0.1707	0.6831	0.5406
0.6087	10.0892	1100	0.1684	0.6981	0.5379
0.5878	11.0046	1200	0.1658	0.6850	0.5306
0.566	11.9199	1300	0.1658	0.6720	0.5288
0.5547	12.8352	1400	0.1625	0.6950	0.5236
0.5359	13.7506	1500	0.1622	0.7058	0.5168
0.5274	14.6659	1600	0.1621	0.6932	0.5176
0.5071	15.5812	1700	0.1614	0.6984	0.5147
0.492	16.4966	1800	0.1616	0.6918	0.5139
0.4932	17.4119	1900	0.1595	0.7154	0.5060
0.4719	18.3272	2000	0.1593	0.7011	0.5096
0.4631	19.2632	2100	0.7115	0.5058	0.1590
0.462	20.1785	2200	0.7084	0.5051	0.1577
0.4538	21.0938	2300	0.7388	0.5014	0.1572
0.4506	22.0092	2400	0.7228	0.5016	0.1576
0.4416	22.9245	2500	0.7204	0.5009	0.1575
0.4437	23.8398	2600	0.7201	0.5002	0.1577
0.444	24.7551	2700	0.7240	0.4972	0.1568

Framework versions

• Transformers 4.41.1
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.19.1