CasperMuz/wav2vec2-base-sna-cleaned

Wav2Vec2 Base Shona - Cleaned

This model is a fine-tuned version of facebook/wav2vec2-base on the Cleaned Google WAXAL Shona dataset. It achieves the following results on the evaluation set:

• Loss: 0.3005
• Wer: 37.7326

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 400
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
11.8984	0.2998	250	2.9574	99.9973
11.5445	0.5995	500	2.8993	99.9973
3.8440	0.8993	750	0.8141	98.7660
2.3326	1.1990	1000	0.5056	56.1276
1.9937	1.4988	1250	0.4354	49.7725
1.8639	1.7986	1500	0.3815	45.0764
1.5769	2.0983	1750	0.3492	43.6845
1.4288	2.3981	2000	0.3384	41.2247
1.3319	2.6978	2250	0.3376	40.4620
1.2553	2.9976	2500	0.3212	39.5059
1.2150	3.2974	2750	0.3165	39.0700
1.1343	3.5971	3000	0.3103	38.2229
1.1755	3.8969	3250	0.3044	38.5361
1.1932	4.1966	3500	0.3002	37.8551
1.3583	4.4964	3750	0.2995	37.7135
1.3055	4.7962	4000	0.3005	37.7326

Framework versions

• Transformers 5.14.1
• Pytorch 2.12.0+cu130
• Datasets 5.0.0
• Tokenizers 0.22.2