FilippoLampa/whisper-small-dv

dysarthria_emo_enhancer_0_0

This model is a fine-tuned version of openai/whisper-small on the custom_torgo_0_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5371
• Wer Ortho: 50.5634
• Wer: 33.5172

And the following results on the TORGO + UAS training set:

• Acc: 0.30
• Wer: 80.63

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
3.4478	0.35	50	3.6845	67.3239	50.4828
2.3364	0.7	100	2.4193	60.5634	44.0
0.2024	1.05	150	0.7172	56.4789	36.6897
0.3609	1.39	200	0.5792	54.9296	36.8276
0.2227	1.74	250	0.5763	53.9437	35.4483
0.0752	2.09	300	0.5516	53.6620	34.8966
0.0249	2.44	350	0.5511	46.7606	29.5172
0.17	2.79	400	0.5289	50.4225	31.8621
0.0736	3.14	450	0.5618	51.5493	32.5517
0.0375	3.48	500	0.5371	50.5634	33.5172

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1