arun100/whisper-base-ko-2

Whisper Base Korean

This model is a fine-tuned version of openai/whisper-base on the google/fleurs ko_kr dataset. It achieves the following results on the evaluation set:

• Loss: 0.4901
• Wer: 27.4344

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3225	66.0	500	0.5002	27.9275
0.1185	133.0	1000	0.4901	27.4344
0.0468	199.0	1500	0.5047	27.4696
0.0268	266.0	2000	0.5147	27.8746
0.0189	333.0	2500	0.5218	28.0507
0.0145	399.0	3000	0.5273	28.4733
0.0121	466.0	3500	0.5318	28.6318
0.0107	533.0	4000	0.5352	28.6846
0.0098	599.0	4500	0.5376	28.8079
0.0095	666.0	5000	0.5385	28.8079

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0