480min_wav2vec_xls-r53_FT

This model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-arabic on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7631
• Wer: 0.5522
• Cer: 0.1752

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
1.7166	1.5253	100	0.2935	1.0749	0.8271
1.1245	3.0505	200	0.2443	0.9033	0.7330
0.9648	4.6015	300	0.8433	0.6700	0.2202
0.8567	6.1268	400	0.7753	0.6359	0.2062
0.769	7.6520	500	0.7855	0.6054	0.1952
0.7087	9.1773	600	0.7694	0.5903	0.1898
0.6554	10.7026	700	0.7280	0.5789	0.1856
0.6226	12.2278	800	0.7751	0.5685	0.1830
0.5922	13.7531	900	0.7454	0.5607	0.1794
0.5657	15.2784	1000	0.7614	0.5639	0.1788
0.5537	16.8036	1100	0.7438	0.5585	0.1778
0.5383	18.3289	1200	0.7551	0.5555	0.1762
0.5255	19.8541	1300	0.7631	0.5522	0.1752

Framework versions

• Transformers 4.41.1
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.19.1