whisper-hf-lora

This model is a fine-tuned version of openai/whisper-large-v3-turbo on compulsion/heart-failure-audio. It achieves the following results on the evaluation set:

• Loss: 0.7572
• Wer: 0.2624

Model description

A PEFT LoRA adapter of whisper-large-v3-turbo finetuned on heart failure audio data that is conversational, longitudinal, and focused on chronic illness management and care coordination in a community-based healthcare setting.

Intended uses & limitations

To be used in ASR tasks specifically in the heart failure domain.

Benchmark (base whisper-large-v3-turbo vs. finetuned LoRA adapter)

Normalized for PHI redactions and throught Transformer's BasicTextNormalizer.

Model	Raw WER (%)	Normalised WER (%)
Baseline	35.00	26.71
LoRA	28.61	22.68

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 500
• num_epochs: 8
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.5146	1.0	92	2.3205	0.2746
2.4798	2.0	184	2.1513	0.2734
2.1105	3.0	276	1.7245	0.2647
1.6642	4.0	368	1.2785	0.2463
1.2627	5.0	460	1.0579	0.2395
1.076	6.0	552	0.9416	0.2771
1.0108	7.0	644	0.8642	0.2746
0.9166	8.0	736	0.7572	0.2624

Framework versions

• PEFT 0.15.2
• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1