w2vbert-lingala-sd3

This model is a fine-tuned version of sulaimank/w2vbert-lingala-waxal-punct-v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0758
• Wer Keep: 0.1038
• Cer Keep: 0.0301
• Zindi Keep: 0.9331
• Wer Strip: 0.0530
• Zindi Strip: 0.9663
• Zindi Lower: 0.9746

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Keep	Cer Keep	Zindi Keep	Wer Strip	Zindi Strip	Zindi Lower
3.9843	0.0635	200	2.8641	1.0332	0.6587	0.1541	1.0328	0.1560	0.1562
1.857	0.1271	400	0.5767	0.2807	0.1058	0.8068	0.2567	0.8232	0.8283
0.7646	0.1906	600	0.4026	0.2280	0.0743	0.8489	0.1725	0.8833	0.8892
0.792	0.2542	800	0.2842	0.2074	0.0612	0.8657	0.1556	0.8998	0.9054
1.1558	0.3177	1000	0.2448	0.1751	0.0552	0.8849	0.1255	0.9190	0.9255
1.2207	0.3813	1200	0.2150	0.1599	0.0461	0.8970	0.1110	0.9289	0.9355
0.7727	0.4448	1400	0.2092	0.1657	0.0472	0.8935	0.1084	0.9308	0.9373
0.4014	0.5083	1600	0.1645	0.1523	0.0458	0.9009	0.1023	0.9343	0.9413
0.5154	0.5719	1800	0.1285	0.1430	0.0388	0.9091	0.0940	0.9403	0.9483
0.2078	0.6354	2000	0.1193	0.1314	0.0380	0.9153	0.0823	0.9473	0.9551
0.1819	0.6990	2200	0.1171	0.1322	0.0371	0.9154	0.0823	0.9475	0.9553
0.5426	0.7625	2400	0.1119	0.1291	0.0375	0.9167	0.0779	0.9502	0.9582
0.508	0.8261	2600	0.1123	0.1256	0.0372	0.9186	0.0754	0.9518	0.9597
0.1858	0.8896	2800	0.1090	0.1260	0.0373	0.9183	0.0757	0.9517	0.9597
0.4101	0.9531	3000	0.1068	0.1252	0.0359	0.9194	0.0745	0.9525	0.9603
0.4611	1.0165	3200	0.1053	0.1230	0.0353	0.9209	0.0727	0.9538	0.9619
0.3242	1.0801	3400	0.1041	0.1250	0.0367	0.9192	0.0731	0.9535	0.9617
0.2125	1.1436	3600	0.1013	0.1208	0.0343	0.9224	0.0696	0.9557	0.9638
0.1616	1.2071	3800	0.1011	0.1203	0.0351	0.9223	0.0702	0.9554	0.9637
0.2845	1.2707	4000	0.0999	0.1190	0.0346	0.9232	0.0686	0.9563	0.9648
0.3022	1.3342	4200	0.0963	0.1166	0.0342	0.9246	0.0669	0.9575	0.9659
0.3556	1.3978	4400	0.0947	0.1186	0.0338	0.9238	0.0669	0.9574	0.9656
0.3133	1.4613	4600	0.0990	0.1182	0.0337	0.9240	0.0682	0.9566	0.9650
0.3633	1.5249	4800	0.0961	0.1175	0.0345	0.9240	0.0668	0.9573	0.9655
0.2762	1.5884	5000	0.0993	0.1200	0.0352	0.9224	0.0696	0.9557	0.9652
0.3428	1.6519	5200	0.0932	0.1147	0.0341	0.9256	0.0638	0.9593	0.9679
0.3717	1.7155	5400	0.0961	0.1156	0.0334	0.9255	0.0650	0.9584	0.9669
0.2016	1.7790	5600	0.0964	0.1164	0.0325	0.9255	0.0649	0.9588	0.9674
0.2162	1.8426	5800	0.0940	0.1131	0.0322	0.9274	0.0626	0.9600	0.9685
0.3036	1.9061	6000	0.0952	0.1143	0.0334	0.9262	0.0638	0.9592	0.9673
0.2259	1.9697	6200	0.0943	0.1136	0.0336	0.9264	0.0627	0.9600	0.9683
0.2473	2.0330	6400	0.0890	0.1113	0.0314	0.9287	0.0606	0.9614	0.9696
0.3503	2.0966	6600	0.0917	0.1124	0.0324	0.9276	0.0616	0.9607	0.9693
0.1326	2.1601	6800	0.0881	0.1086	0.0306	0.9304	0.0582	0.9629	0.9710
0.2621	2.2237	7000	0.0897	0.1126	0.0324	0.9275	0.0598	0.9619	0.9703
0.1797	2.2872	7200	0.0864	0.1086	0.0310	0.9302	0.0580	0.9631	0.9713
0.1742	2.3508	7400	0.0848	0.1078	0.0313	0.9304	0.0579	0.9631	0.9711
0.1161	2.4143	7600	0.0841	0.1110	0.0320	0.9285	0.0598	0.9619	0.9704
0.2284	2.4778	7800	0.0853	0.1090	0.0316	0.9297	0.0585	0.9628	0.9709
0.2763	2.5414	8000	0.0823	0.1091	0.0312	0.9298	0.0587	0.9627	0.9708
0.1775	2.6049	8200	0.0839	0.1094	0.0312	0.9297	0.0591	0.9625	0.9716
0.2852	2.6685	8400	0.0828	0.1095	0.0317	0.9294	0.0581	0.9631	0.9715
0.1337	2.7320	8600	0.0818	0.1094	0.0309	0.9299	0.0578	0.9633	0.9717
0.2429	2.7956	8800	0.0806	0.1075	0.0308	0.9308	0.0571	0.9638	0.9723
0.2814	2.8591	9000	0.0806	0.1071	0.0308	0.9310	0.0566	0.9640	0.9723
0.2304	2.9226	9200	0.0815	0.1063	0.0304	0.9316	0.0559	0.9644	0.9725
0.2363	2.9862	9400	0.0807	0.1066	0.0311	0.9311	0.0550	0.9650	0.9733
0.2167	3.0496	9600	0.0814	0.1059	0.0305	0.9318	0.0548	0.9652	0.9737
0.3208	3.1131	9800	0.0828	0.1060	0.0306	0.9317	0.0547	0.9652	0.9739
0.3189	3.1766	10000	0.0801	0.1061	0.0309	0.9315	0.0548	0.9652	0.9735
0.2116	3.2402	10200	0.0797	0.1055	0.0309	0.9318	0.0550	0.9651	0.9734
0.2195	3.3037	10400	0.0789	0.1064	0.0307	0.9314	0.0547	0.9653	0.9736
0.2198	3.3673	10600	0.0803	0.1053	0.0303	0.9322	0.0544	0.9654	0.9739
0.1865	3.4308	10800	0.0785	0.1045	0.0300	0.9327	0.0537	0.9659	0.9742
0.3211	3.4944	11000	0.0781	0.1065	0.0306	0.9315	0.0546	0.9653	0.9742
0.3764	3.5579	11200	0.0786	0.1061	0.0309	0.9315	0.0544	0.9654	0.9741
0.272	3.6214	11400	0.0768	0.1043	0.0306	0.9326	0.0535	0.9660	0.9743
0.1589	3.6850	11600	0.0769	0.1039	0.0298	0.9331	0.0530	0.9663	0.9745
0.1082	3.7485	11800	0.0759	0.1040	0.0300	0.9330	0.0535	0.9660	0.9743
0.1823	3.8121	12000	0.0767	0.1039	0.0301	0.9330	0.0531	0.9662	0.9745
0.1889	3.8756	12200	0.0756	0.1038	0.0300	0.9331	0.0531	0.9662	0.9745
0.3766	3.9392	12400	0.0758	0.1038	0.0301	0.9331	0.0530	0.9663	0.9746

Framework versions

• Transformers 4.57.6
• Pytorch 2.11.0+cu128
• Datasets 3.6.0
• Tokenizers 0.22.2