w2vbert-lingala-sd2

This model is a fine-tuned version of sulaimank/w2vbert-lingala-waxal-punct-v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0545
• Wer Keep: 0.0915
• Cer Keep: 0.0214
• Zindi Keep: 0.9436
• Wer Strip: 0.0403
• Zindi Strip: 0.9746
• Zindi Lower: 0.9829

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 8.0

Training results

Training Loss	Epoch	Step	Cer Keep	Validation Loss	Wer Keep	Wer Strip	Zindi Keep	Zindi Lower	Zindi Strip
3.6428	0.1758	200	0.9997	3.2376	1.0	1.0	0.0001	0.0001	0.0001
1.6433	0.3516	400	0.0408	0.1868	0.1654	0.1120	0.8969	0.9459	0.9292
0.5927	0.5275	600	0.0336	0.1295	0.1356	0.0799	0.9154	0.9661	0.9494
0.5845	0.7033	800	0.0314	0.1090	0.1237	0.0728	0.9224	0.9662	0.9533
0.9751	0.8791	1000	0.0296	0.1028	0.1163	0.0651	0.9270	0.9686	0.9583
0.3809	1.0545	1200	0.0282	0.0974	0.1109	0.0594	0.9304	0.9718	0.9620
0.5085	1.2303	1400	0.0301	0.0975	0.1158	0.0632	0.9271	0.9684	0.9591
0.2797	1.4062	1600	0.0286	0.0970	0.1128	0.0606	0.9293	0.9714	0.9611
0.442	1.5820	1800	0.0274	0.0912	0.1092	0.0588	0.9317	0.9727	0.9624
0.6423	1.7578	2000	0.0272	0.0906	0.1108	0.0607	0.9310	0.9710	0.9613
0.2759	1.9336	2200	0.0274	0.0824	0.1080	0.0556	0.9323	0.9734	0.9643
0.3426	2.1090	2400	0.0262	0.0833	0.1049	0.0541	0.9344	0.9745	0.9653
0.1779	2.2848	2600	0.0269	0.0845	0.1073	0.0557	0.9329	0.9734	0.9642
0.3073	2.4607	2800	0.0265	0.0838	0.1065	0.0552	0.9335	0.9740	0.9646
0.3184	2.6365	3000	0.0267	0.0742	0.1074	0.0553	0.9330	0.9740	0.9645
0.3589	2.8123	3200	0.0262	0.0726	0.1045	0.0535	0.9346	0.9755	0.9656
0.3602	2.9881	3400	0.0254	0.0720	0.1028	0.0517	0.9359	0.9761	0.9669
0.259	3.1635	3600	0.0259	0.0754	0.1029	0.0516	0.9356	0.9764	0.9669
0.1736	3.3393	3800	0.0251	0.0722	0.1012	0.0492	0.9368	0.9783	0.9686
0.1477	3.5152	4000	0.0699	0.0992	0.0242	0.9383	0.0486	0.9690	0.9780
0.1615	3.6910	4200	0.0676	0.0994	0.0240	0.9383	0.0485	0.9692	0.9786
0.2897	3.8668	4400	0.0686	0.0984	0.0241	0.9388	0.0479	0.9695	0.9784
0.3471	4.0422	4600	0.0647	0.0993	0.0241	0.9383	0.0480	0.9695	0.9786
0.5524	4.2180	4800	0.0659	0.0994	0.0236	0.9385	0.0468	0.9703	0.9789
0.3344	4.3938	5000	0.0647	0.0984	0.0235	0.9390	0.0471	0.9702	0.9792
0.3536	4.5697	5200	0.0641	0.1005	0.0237	0.9379	0.0470	0.9702	0.9796
0.1587	4.7455	5400	0.0638	0.0961	0.0230	0.9405	0.0451	0.9713	0.9801
0.3888	4.9213	5600	0.0615	0.0958	0.0235	0.9404	0.0447	0.9715	0.9801
0.2487	5.0967	5800	0.0614	0.0975	0.0232	0.9397	0.0459	0.9709	0.9799
0.0854	5.2725	6000	0.0604	0.0950	0.0226	0.9412	0.0442	0.9720	0.9808
0.2548	5.4484	6200	0.0618	0.0948	0.0226	0.9413	0.0439	0.9722	0.9809
0.1518	5.6242	6400	0.0599	0.0951	0.0225	0.9412	0.0443	0.9719	0.9807
0.4898	5.8	6600	0.0586	0.0946	0.0223	0.9416	0.0431	0.9727	0.9812
0.2699	5.9758	6800	0.0572	0.0936	0.0222	0.9421	0.0420	0.9734	0.9820
0.316	6.1512	7000	0.0576	0.0942	0.0219	0.9420	0.0432	0.9727	0.9815
0.3514	6.3270	7200	0.0562	0.0921	0.0218	0.9431	0.0413	0.9738	0.9823
0.1803	6.5029	7400	0.0560	0.0922	0.0219	0.9429	0.0413	0.9739	0.9822
0.1222	6.6787	7600	0.0564	0.0931	0.0219	0.9425	0.0414	0.9738	0.9823
0.1351	6.8545	7800	0.0552	0.0917	0.0216	0.9433	0.0403	0.9744	0.9830
0.2344	7.0299	8000	0.0548	0.0917	0.0214	0.9434	0.0405	0.9744	0.9828
0.4722	7.2057	8200	0.0547	0.0911	0.0213	0.9438	0.0403	0.9745	0.9829
0.1888	7.3815	8400	0.0545	0.0912	0.0214	0.9437	0.0405	0.9744	0.9828
0.2479	7.5574	8600	0.0548	0.0920	0.0216	0.9432	0.0407	0.9743	0.9828
0.1695	7.7332	8800	0.0544	0.0919	0.0215	0.9433	0.0408	0.9742	0.9826
0.3146	7.9090	9000	0.0545	0.0915	0.0214	0.9436	0.0403	0.9746	0.9829

Framework versions

• Transformers 4.57.6
• Pytorch 2.13.0+cu130
• Datasets 3.6.0
• Tokenizers 0.22.2