denysdios/whisper-med-tr-tuned

Whisper Medium Tr - denysdios

This model is a fine-tuned version of openai/whisper-medium on the Common Voice 13.0 & Fleurs dataset.

It achieves the following results on the evaluation set:

• Loss: 0.1618
• Wer: 14.3825

Model Description

The model took about nine hours to train on a single A100 GPU.

⚠️ Production Status: This specific artifact is an experimental research iteration designed to analyze training dynamics and is not production-ready. The fully optimized, production-grade model with mitigated overfitting remains private.

Intended Uses & Limitations

This repository serves as a benchmark checkpoint for studying fine-tuning constraints on Turkish ASR. The primary objective was increasing Turkish labeled data exposure. However, because the dataset contains only 49.945 hours of data (roughly 1.1% of the original Whisper pre-training distribution), the model exhibits standard fine-tuning constraints under out-of-domain distribution shifts.

Post-Evaluation Insights (Generalization Metrics)

Independent diagnostic evaluations reveal a high variance between clean in-domain data and noisy out-of-domain speech:

• In-Domain Memorization (FLEURS Train Split): Natively matches audio profiles with an exceptional 1.15% WER.
• Out-of-Domain Performance (MediaSpeech/OpenSLR): Degrades to a 30.29% WER due to acoustic domain mismatches (broadcast TV/radio vs. clean reading speech).

Strategic Takeaway: The training logs show clear optimization divergence past Step 2000. Beyond this threshold, the model ceased mapping generalized Turkish phonetics and began memorizing the acoustic signatures of the training subset. For downstream applications, weights should be pulled from the Step 2000 checkpoint to maintain general zero-shot capacity.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training Results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1803	0.36	1000	0.2089	18.6326
0.1428	0.71	2000	0.1821	16.3912
0.0535	1.07	3000	0.1693	14.9132
0.0491	1.43	4000	0.1618	14.3825

Framework Versions

• Transformers 4.38.1
• Pytorch 2.1.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2