MathildeB3/52Hz-small-LORA-fr

52Hz Small Fr - IMT Atlantique X 52 Hertz

This model is a fine-tuned version of openai/whisper-small on the Premier dataset organisé de 52 Hertz dataset. It achieves the following results on the evaluation set:

• Loss: 0.3100
• Wer: 19.5896

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.4225	1.0	21	0.9856	61.0075
1.1531	2.0	42	0.5645	34.5149
0.6599	3.0	63	0.3796	29.8507
0.4498	4.0	84	0.3136	26.6791
0.3502	5.0	105	0.2993	26.6791
0.2559	6.0	126	0.3062	25.5597
0.1846	7.0	147	0.2905	21.6418
0.1538	8.0	168	0.3110	23.6940
0.1498	9.0	189	0.2954	21.0821
0.1431	10.0	210	0.2963	20.7090
0.0861	11.0	231	0.2945	19.5896
0.082	12.0	252	0.3092	22.2015
0.0786	13.0	273	0.2977	19.0299
0.072	14.0	294	0.2997	21.2687
0.0613	15.0	315	0.3030	20.3358
0.0489	16.0	336	0.3104	20.3358
0.0527	17.0	357	0.3075	19.4030
0.052	18.0	378	0.3099	19.7761
0.0518	19.0	399	0.3101	19.5896
0.0491	20.0	420	0.3100	19.5896

Framework versions

• PEFT 0.18.1
• Transformers 4.57.3
• Pytorch 2.9.1+cu130
• Datasets 4.4.2
• Tokenizers 0.22.2