w2v2-lmk_original

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.3823
• Wer: 0.5436
• Cer: 0.1752

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
8.7555	0.9217	100	1.0	4.5183	1.0
3.2686	1.8387	200	1.0	2.9922	1.0
3.081	2.7558	300	1.0	2.9402	1.0
2.9798	3.6728	400	1.0	2.8604	1.0
2.9457	4.5899	500	1.0	2.7142	1.0
2.6169	5.5069	600	0.9292	2.1802	1.0
2.0416	6.4240	700	0.4821	1.4700	0.9547
1.7328	7.3410	800	0.3199	1.1744	0.8362
1.6054	8.2581	900	0.2993	1.0605	0.8223
1.4377	9.1751	1000	0.2612	0.9728	0.7178
1.3523	10.0922	1100	0.2475	0.9209	0.6969
1.2561	11.0092	1200	0.2384	0.9213	0.6585
1.1447	11.9309	1300	0.2308	0.9006	0.6028
1.1125	12.8479	1400	0.2140	0.9020	0.6132
1.0296	13.7650	1500	0.2125	0.8684	0.5923
0.955	14.6820	1600	0.2110	0.8873	0.5679
0.9307	15.5991	1700	0.2064	0.8307	0.5819
0.8542	16.5161	1800	0.2041	0.9681	0.5470
0.8282	17.4332	1900	0.1950	0.9368	0.5819
0.8439	18.3502	2000	0.1935	0.8414	0.5645
0.7883	19.2673	2100	0.1889	0.9085	0.5401
0.7347	20.1843	2200	0.1874	0.8952	0.5679
0.7454	21.1014	2300	0.1912	0.8924	0.5540
0.7236	22.0184	2400	0.1995	0.9485	0.5889
0.6953	22.9401	2500	0.1813	0.9403	0.5331
0.6605	23.8571	2600	0.1889	0.8801	0.5505
0.6406	24.7742	2700	0.1957	0.9006	0.5679
0.6101	25.6912	2800	0.1942	0.9417	0.5575
0.564	26.6083	2900	0.1805	0.9250	0.5436
0.5788	27.5253	3000	0.2018	0.9609	0.5784
0.5531	28.4424	3100	0.1889	0.9829	0.5749
0.5559	29.3594	3200	0.1851	0.9122	0.5540
0.5452	30.2765	3300	0.1973	1.0399	0.5645
0.4801	31.1935	3400	0.2034	1.0395	0.5923
0.5206	32.1106	3500	0.1851	1.0093	0.5575
0.5219	33.0276	3600	0.1889	1.0413	0.5679
0.4822	33.9493	3700	0.1896	0.9724	0.5540
0.4808	34.8664	3800	0.2011	1.1110	0.5923
0.4729	35.7834	3900	0.1973	1.0083	0.5645
0.4437	36.7005	4000	0.1896	1.0383	0.5679
0.44	37.6175	4100	0.1957	1.0961	0.5749
0.42	38.5346	4200	0.1950	1.1664	0.5679
0.389	39.4516	4300	0.1995	1.1686	0.5958
0.4146	40.3687	4400	0.1995	1.1471	0.5993
0.3715	41.2857	4500	0.1904	1.1480	0.5714
0.3753	42.2028	4600	0.1988	1.1812	0.5784
0.4049	43.1198	4700	0.1927	1.2219	0.5819
0.3731	44.0369	4800	0.1927	1.2076	0.5819
0.345	44.9585	4900	0.2049	1.2494	0.5993
0.3664	45.8756	5000	0.1881	1.0780	0.5679
0.3811	46.7926	5100	0.2003	1.2551	0.5749
0.327	47.7097	5200	0.2018	1.2526	0.5749
0.3156	48.6267	5300	0.1973	1.2325	0.5749
0.3394	49.5438	5400	0.1889	1.2548	0.5610
0.3343	50.4608	5500	0.1919	1.2031	0.5610
0.3427	51.3779	5600	0.1843	1.1861	0.5436
0.3223	52.2949	5700	0.1896	1.1878	0.5575
0.2747	53.2120	5800	0.1919	1.2358	0.5645
0.3128	54.1290	5900	0.1889	1.2146	0.5645
0.299	55.0461	6000	0.1851	1.2575	0.5540
0.2905	55.9677	6100	0.1858	1.3072	0.5610
0.2909	56.8848	6200	0.1965	1.3107	0.5784
0.2831	57.8018	6300	0.1912	1.2443	0.5714
0.26	58.7189	6400	0.1942	1.3205	0.5784
0.2638	59.6359	6500	0.1889	1.2863	0.5645
0.2714	60.5530	6600	0.1957	1.3860	0.5401
0.2473	61.4700	6700	0.1858	1.3188	0.5366
0.2507	62.3871	6800	0.1851	1.3305	0.5366
0.292	63.3041	6900	1.3343	0.5366	0.1828
0.231	64.2212	7000	1.2796	0.5505	0.1904
0.2665	65.1382	7100	1.2489	0.5366	0.1835
0.2572	66.0553	7200	1.2689	0.5575	0.1858
0.2318	66.9770	7300	1.3170	0.5505	0.1851
0.2265	67.8940	7400	1.3452	0.5610	0.1980
0.2342	68.8111	7500	1.3586	0.5366	0.1828
0.21	69.7281	7600	1.3115	0.5505	0.1835
0.2072	70.6452	7700	1.3487	0.5436	0.1828
0.2258	71.5622	7800	1.3543	0.5366	0.1820
0.2207	72.4793	7900	1.3404	0.5226	0.1759
0.2169	73.3963	8000	1.3788	0.5610	0.1881
0.2623	74.3134	8100	1.3656	0.5470	0.1874
0.2063	75.2304	8200	1.3811	0.5505	0.1858
0.2127	76.1475	8300	1.3472	0.5331	0.1820
0.2212	77.0645	8400	1.3495	0.5192	0.1782
0.206	77.9862	8500	1.3442	0.5436	0.1866
0.1957	78.9032	8600	1.3710	0.5226	0.1813
0.191	79.8203	8700	1.3939	0.5505	0.1866
0.2056	80.7373	8800	1.3876	0.5401	0.1805
0.193	81.6544	8900	1.4255	0.5436	0.1790
0.2125	82.5714	9000	1.4117	0.5331	0.1805
0.1927	83.4885	9100	1.3948	0.5366	0.1775
0.1803	84.4055	9200	1.3804	0.5505	0.1767
0.1902	85.3226	9300	1.3633	0.5436	0.1767
0.1713	86.2396	9400	1.4131	0.5401	0.1797
0.1683	87.1567	9500	1.3884	0.5401	0.1782
0.2077	88.0737	9600	1.3943	0.5331	0.1744
0.1976	88.9954	9700	1.3902	0.5366	0.1736
0.1835	89.9124	9800	1.4138	0.5505	0.1782
0.1686	90.8295	9900	1.4114	0.5436	0.1797
0.1761	91.7465	10000	1.4197	0.5575	0.1782
0.166	92.6636	10100	1.4012	0.5436	0.1744
0.1665	93.5806	10200	1.4099	0.5540	0.1767
0.1887	94.4977	10300	1.4095	0.5540	0.1782
0.1999	95.4147	10400	1.3848	0.5505	0.1759
0.1542	96.3318	10500	1.3773	0.5505	0.1752
0.188	97.2488	10600	1.3887	0.5331	0.1729
0.1739	98.1659	10700	1.3844	0.5366	0.1744
0.187	99.0829	10800	1.3830	0.5401	0.1744
0.1796	100.0	10900	1.3823	0.5436	0.1752

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu128
• Datasets 3.0.0
• Tokenizers 0.22.1