sulaimank/wav2vec2-large-mms-1b-ttj

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2330
• Wer: 0.2797
• Cer: 0.0551

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-03
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Step	Training Loss	Validation Loss	Wer	Cer
100	7.9083	3.7621	0.999886	0.891796
200	0.6899	0.2827	0.340568	0.065918
300	0.3870	0.2672	0.341250	0.065194
400	0.3613	0.2500	0.310795	0.059507
500	0.3395	0.2438	0.299545	0.058291
600	0.3127	0.2432	0.289545	0.057061
700	0.3186	0.2330	0.279659	0.055137
800	0.3000	0.2337	0.282273	0.055658
900	0.2955	0.2298	0.280568	0.055325

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.1+cu128
• Datasets 4.4.1
• Tokenizers 0.22.1