Rakib/whisper-small-bn-all-600-v2

Rakib/whisper-small-bn-all-600

This model is a fine-tuned version of Rakib/whisper-small-bn-all-600 on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0796
• Cer: 7.9281
• Wer: 14.3762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 128
• total_train_batch_size: 512
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 58
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
No log	0.36	20	29.4196	0.5030	51.1675
2.2164	0.72	40	17.1332	0.2866	32.3807
1.1372	1.08	60	14.6765	0.2368	28.1070
0.6747	1.43	80	12.3966	0.2014	24.4490
0.4837	1.79	100	11.1684	0.1774	22.1318
0.4837	2.15	120	10.3942	0.1645	20.6119
0.3824	2.51	140	11.1069	0.1524	21.4795
0.3091	2.9	160	0.1425	12.1593	22.8154
0.2898	3.25	180	0.1352	12.1491	22.9727
0.265	3.61	200	0.1312	12.6732	23.5769
0.265	3.97	220	0.1235	12.8681	23.4950
0.2436	4.33	240	0.1221	12.0239	22.4498
0.2288	4.69	260	0.1179	12.2401	22.1606
0.2181	5.05	280	0.1145	11.7777	22.0505
0.2051	5.4	300	0.1124	11.4862	21.1575
0.2051	5.76	320	0.1101	11.0824	20.4293
0.1967	6.12	340	0.1087	11.3310	20.6669
0.1894	6.48	360	0.1066	10.7207	19.7527
0.1808	6.84	380	0.1041	10.1898	19.1048
0.1741	7.2	400	0.1029	10.4854	19.1311
0.1741	7.56	420	0.1022	10.0271	18.4430
0.1702	7.91	440	0.0999	9.8511	18.3329
0.1645	8.27	460	0.0992	10.7231	18.9317
0.1595	8.63	480	0.0979	10.4477	18.6613
0.1542	8.99	500	0.0957	10.2274	18.4698
0.1542	9.35	520	0.0956	10.2842	18.2585
0.1493	9.71	540	0.0938	10.2244	18.0303
0.1481	10.06	560	0.0932	10.3663	18.1245
0.1437	10.42	580	0.0926	10.2147	18.1722
0.1392	10.78	600	0.0916	10.9133	18.6320
0.1392	11.14	620	0.0913	10.8184	18.6345
0.1375	11.5	640	0.0904	9.6980	17.2376
0.1328	11.86	660	0.0895	9.7480	17.3755
0.1318	12.21	680	0.0879	11.3823	18.7273
0.1282	12.57	700	0.0883	9.3186	16.7355
0.1282	12.93	720	0.0869	9.1750	16.4895
0.1266	13.29	740	0.0875	9.7876	17.0213
0.1225	13.65	760	0.0861	10.1410	17.3303
0.1213	14.01	780	0.0848	9.8175	16.9722
0.1183	14.36	800	0.0853	9.8500	16.9727
0.1183	14.72	820	0.0841	9.0481	15.9736
0.1171	15.08	840	0.0846	9.1555	16.1854
0.1145	15.44	860	0.0838	9.2261	16.1700
0.1129	15.8	880	0.0835	8.9634	15.7895
0.1114	16.16	900	0.0827	8.7861	15.6392
0.1114	16.51	920	0.0824	8.8835	15.7523
0.1094	16.87	940	0.0818	8.6445	15.2597
0.107	17.23	960	0.0815	9.2495	15.9091
0.1065	17.59	980	0.0811	10.1371	16.8844
0.1048	17.95	1000	0.0805	8.9065	15.6080
0.1048	18.31	1020	0.0812	8.2908	14.7914
0.1021	18.67	1040	0.0795	8.2616	14.8614
0.1011	19.02	1060	0.0802	8.6007	14.9467
0.099	19.38	1080	0.0802	8.6523	15.0881
0.0974	19.74	1100	0.0790	8.2233	14.8564
0.0974	20.1	1120	0.0796	7.9281	14.3762
0.0971	20.46	1140	0.0787	8.3144	14.6768
0.0943	20.82	1160	0.0779	8.7677	15.1263
0.094	21.17	1180	0.0783	9.0204	15.4165
0.0924	21.53	1200	0.0778	11.5092	17.7178
0.0924	21.89	1220	0.0778	8.1642	14.3767
0.0918	22.25	1240	0.0783	8.6886	14.9904
0.0897	22.61	1260	0.0772	11.9568	18.1285
0.0886	22.97	1280	0.0771	9.6865	15.8644
0.0874	23.32	1300	0.0769	8.4833	14.5955
0.0874	23.68	1320	0.0765	8.8560	15.0816
0.0873	24.04	1340	0.0763	8.7034	14.7646
0.0849	24.4	1360	0.0766	8.4351	14.4784
0.084	24.76	1380	0.0754	10.2015	16.2142
0.0826	25.12	1400	0.0759	10.0267	16.1100
0.0826	25.47	1420	0.0759	11.6649	17.8229

Framework versions

• Transformers 4.28.0.dev0
• Pytorch 2.0.0+cu117
• Datasets 2.10.2.dev0
• Tokenizers 0.13.2