arun100/whisper-base-tr-2

Whisper Base Turkish

This model is a fine-tuned version of arun100/whisper-base-tr-1 on the google/fleurs tr_tr dataset. It achieves the following results on the evaluation set:

• Loss: 0.5195
• Wer: 28.3845

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3503	45.0	500	0.4874	29.1817
0.0695	90.0	1000	0.4960	28.4597
0.0243	136.0	1500	0.5195	28.3845
0.0145	181.0	2000	0.5334	28.6477
0.0101	227.0	2500	0.5454	28.6778
0.0077	272.0	3000	0.5548	28.6928
0.0063	318.0	3500	0.5625	28.7079
0.0054	363.0	4000	0.5684	29.0238
0.0048	409.0	4500	0.5727	28.9260
0.0046	454.0	5000	0.5743	28.9260

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0