w2vbert-waxal-p2b

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4981
• Wer Ach: 0.3353
• Cer Ach: 0.1347
• Zindi Ach: 0.7650
• Wer Mas: 0.5025
• Cer Mas: 0.1108
• Zindi Mas: 0.6934
• Wer Nyn: 0.3598
• Cer Nyn: 0.0849
• Zindi Nyn: 0.7777
• Wer: 0.4020
• Cer: 0.1063
• Zindi: 0.7458
• Zindi Strip: 0.7742
• Zindi Phase2: 0.7453
• Lang Token Acc: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.05
• num_epochs: 8.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ach	Cer Ach	Zindi Ach	Wer Mas	Cer Mas	Zindi Mas	Wer Nyn	Cer Nyn	Zindi Nyn	Wer	Cer	Zindi	Zindi Strip	Zindi Phase2	Lang Token Acc
1.2655	0.1285	200	0.4851	0.3359	0.1352	0.7645	0.5012	0.1109	0.6940	0.3596	0.0856	0.7774	0.4017	0.1067	0.7458	0.7737	0.7453	0.9995
1.3051	0.2570	400	0.4995	0.3342	0.1349	0.7655	0.5009	0.1109	0.6941	0.3617	0.0860	0.7761	0.4017	0.1069	0.7457	0.7736	0.7452	0.9995
1.3676	0.3854	600	0.4891	0.3362	0.1344	0.7647	0.4983	0.1107	0.6955	0.3604	0.0855	0.7770	0.4010	0.1065	0.7463	0.7741	0.7458	0.9990
1.3406	0.5139	800	0.4893	0.3394	0.1355	0.7626	0.5050	0.1117	0.6916	0.3587	0.0856	0.7779	0.4038	0.1072	0.7445	0.7723	0.7440	0.9985
1.3109	0.6424	1000	0.4895	0.3403	0.1355	0.7621	0.5022	0.1112	0.6933	0.3585	0.0854	0.7780	0.4031	0.1069	0.7450	0.7732	0.7445	1.0
1.2962	0.7709	1200	0.4860	0.3373	0.1354	0.7636	0.5029	0.1113	0.6929	0.3559	0.0845	0.7798	0.4015	0.1066	0.7460	0.7744	0.7454	1.0
1.4258	0.8994	1400	0.4895	0.3426	0.1383	0.7595	0.4995	0.1104	0.6951	0.3579	0.0853	0.7784	0.4026	0.1072	0.7451	0.7734	0.7443	0.9995
1.2745	1.0276	1600	0.4921	0.3400	0.1372	0.7614	0.5075	0.1126	0.6900	0.3586	0.0849	0.7783	0.4049	0.1076	0.7438	0.7719	0.7432	1.0
1.2506	1.1561	1800	0.4981	0.3353	0.1347	0.7650	0.5025	0.1108	0.6934	0.3598	0.0849	0.7777	0.4020	0.1063	0.7458	0.7742	0.7453	1.0

Framework versions

• Transformers 5.14.1
• Pytorch 2.13.0+cu130
• Datasets 5.0.1
• Tokenizers 0.22.2