whisper-small-sc

This model is a fine-tuned version of openai/whisper-small on janaab/supreme-court-speech dataset. It achieves the following results on the evaluation set:

• Loss: 0.2992
• Wer Ortho: 10.7269
• Wer: 10.2317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 250
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
0.2799	0.7704	250	0.2614	12.1130	11.6261
0.1454	1.5408	500	0.2367	11.0523	10.6016
0.072	2.3112	750	0.2428	10.6736	10.2154
0.0488	3.0817	1000	0.2638	10.8335	10.3623
0.0294	3.8521	1250	0.2689	11.2821	10.7793
0.012	4.6225	1500	0.2992	10.7269	10.2317

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1