w2v2-lmk_19

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.0836
• Wer: 0.4216
• Cer: 0.1455

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
9.9313	0.9009	100	4.5019	1.0	1.0
3.2354	1.8018	200	2.9738	1.0	1.0
3.0444	2.7027	300	2.8885	1.0	1.0
2.9623	3.6036	400	2.8218	1.0	1.0
2.7147	4.5045	500	2.3713	1.0	0.9772
2.1329	5.4054	600	1.4856	0.9408	0.4775
1.8169	6.3063	700	1.1966	0.9233	0.4288
1.5707	7.2072	800	1.0041	0.6829	0.2620
1.4409	8.1081	900	0.8599	0.6028	0.2216
1.3522	9.0090	1000	0.8020	0.5714	0.1980
1.2304	9.9099	1100	0.7654	0.5366	0.1835
1.1243	10.8108	1200	0.7025	0.5087	0.1744
1.0629	11.7117	1300	0.6877	0.4669	0.1607
1.0488	12.6126	1400	0.6935	0.5192	0.1668
0.9656	13.5135	1500	0.6731	0.5331	0.1676
0.8964	14.4144	1600	0.6878	0.4599	0.1607
0.9137	15.3153	1700	0.6571	0.4460	0.1447
0.8268	16.2162	1800	0.7097	0.4460	0.1592
0.8127	17.1171	1900	0.6337	0.4390	0.1554
0.8318	18.0180	2000	0.6534	0.4495	0.1508
0.7663	18.9189	2100	0.6960	0.4530	0.1554
0.7293	19.8198	2200	0.6238	0.4564	0.1439
0.6942	20.7207	2300	0.6828	0.4425	0.1546
0.6548	21.6216	2400	0.7838	0.4808	0.1668
0.66	22.5225	2500	0.7039	0.4669	0.1584
0.5825	23.4234	2600	0.6381	0.4251	0.1386
0.645	24.3243	2700	0.6896	0.4425	0.1478
0.5921	25.2252	2800	0.7104	0.4181	0.1447
0.6106	26.1261	2900	0.7366	0.3868	0.1455
0.5953	27.0270	3000	0.7506	0.4077	0.1478
0.5834	27.9279	3100	0.7805	0.4216	0.1538
0.5517	28.8288	3200	0.7860	0.4286	0.1538
0.5539	29.7297	3300	0.8348	0.4216	0.1523
0.5358	30.6306	3400	0.7150	0.4495	0.1478
0.4931	31.5315	3500	0.7843	0.4739	0.1637
0.4488	32.4324	3600	0.7595	0.4530	0.1531
0.4684	33.3333	3700	0.8238	0.4739	0.1607
0.4525	34.2342	3800	0.7702	0.4355	0.1508
0.5005	35.1351	3900	0.8459	0.4425	0.1569
0.4589	36.0360	4000	0.7738	0.4286	0.1508
0.39	36.9369	4100	0.7548	0.4460	0.1676
0.3869	37.8378	4200	0.8074	0.4460	0.1516
0.4349	38.7387	4300	0.8406	0.4530	0.1577
0.4009	39.6396	4400	0.8026	0.3937	0.1493
0.3972	40.5405	4500	0.8456	0.4251	0.1584
0.3603	41.4414	4600	0.8148	0.4425	0.1599
0.4276	42.3423	4700	0.8618	0.4181	0.1516
0.358	43.2432	4800	0.8645	0.4530	0.1561
0.3723	44.1441	4900	0.8812	0.4286	0.1500
0.3798	45.0450	5000	0.8375	0.4286	0.1516
0.3423	45.9459	5100	0.8725	0.4355	0.1554
0.3529	46.8468	5200	0.8748	0.4181	0.1554
0.338	47.7477	5300	0.9033	0.4251	0.1554
0.3651	48.6486	5400	0.9047	0.4564	0.1569
0.3449	49.5495	5500	0.9212	0.4495	0.1660
0.3091	50.4505	5600	0.9280	0.4599	0.1653
0.3111	51.3514	5700	0.9505	0.4355	0.1592
0.3118	52.2523	5800	1.0096	0.4460	0.1706
0.2764	53.1532	5900	0.9291	0.4321	0.1584
0.3091	54.0541	6000	1.0081	0.4251	0.1584
0.2998	54.9550	6100	1.0030	0.4495	0.1592
0.2501	55.8559	6200	0.9901	0.4495	0.1577
0.2556	56.7568	6300	1.0038	0.4530	0.1592
0.2937	57.6577	6400	0.9850	0.4774	0.1630
0.2871	58.5586	6500	0.9966	0.4599	0.1615
0.2677	59.4595	6600	0.9827	0.4530	0.1569
0.261	60.3604	6700	0.9686	0.4251	0.1523
0.2645	61.2613	6800	0.9436	0.4077	0.1394
0.2863	62.1622	6900	0.9346	0.4390	0.1500
0.2667	63.0631	7000	0.9870	0.4495	0.1500
0.2185	63.9640	7100	1.0608	0.4530	0.1531
0.2448	64.8649	7200	1.0084	0.4077	0.1455
0.2397	65.7658	7300	0.9756	0.4251	0.1485
0.2231	66.6667	7400	1.0335	0.4286	0.1516
0.2372	67.5676	7500	0.9935	0.3902	0.1447
0.212	68.4685	7600	1.0240	0.4146	0.1432
0.2146	69.3694	7700	1.0775	0.4181	0.1470
0.2303	70.2703	7800	1.0020	0.4286	0.1516
0.2125	71.1712	7900	1.0393	0.3902	0.1447
0.2159	72.0721	8000	1.0347	0.3937	0.1424
0.2129	72.9730	8100	1.0236	0.4286	0.1493
0.2276	73.8739	8200	1.0194	0.4251	0.1516
0.2142	74.7748	8300	1.0487	0.4181	0.1470
0.1972	75.6757	8400	1.0542	0.4321	0.1508
0.2143	76.5766	8500	1.0339	0.4286	0.1493
0.1977	77.4775	8600	1.0695	0.4355	0.1523
0.1807	78.3784	8700	1.0720	0.4077	0.1569
0.2	79.2793	8800	1.0740	0.4077	0.1531
0.2179	80.1802	8900	1.0339	0.3868	0.1478
0.1866	81.0811	9000	1.0598	0.3693	0.1432
0.1944	81.9820	9100	1.0794	0.4077	0.1516
0.1838	82.8829	9200	1.0925	0.4146	0.1546
0.1835	83.7838	9300	1.0811	0.4042	0.1493
0.1913	84.6847	9400	1.0939	0.3937	0.1470
0.1707	85.5856	9500	1.0858	0.3868	0.1455
0.1827	86.4865	9600	1.0677	0.3798	0.1432
0.1934	87.3874	9700	1.0625	0.3868	0.1409
0.1953	88.2883	9800	1.0556	0.3937	0.1478
0.1843	89.1892	9900	1.0856	0.3937	0.1439
0.1835	90.0901	10000	1.0863	0.4077	0.1462
0.2053	90.9910	10100	1.0907	0.4077	0.1493
0.1812	91.8919	10200	1.0642	0.4077	0.1432
0.161	92.7928	10300	1.0856	0.4077	0.1432
0.1411	93.6937	10400	1.0985	0.4007	0.1470
0.1759	94.5946	10500	1.0939	0.3972	0.1432
0.1607	95.4955	10600	1.0919	0.4042	0.1432
0.1689	96.3964	10700	1.0874	0.4216	0.1455
0.1811	97.2973	10800	1.0832	0.4146	0.1424
0.1796	98.1982	10900	1.0837	0.4181	0.1432
0.158	99.0991	11000	1.0836	0.4181	0.1455
0.1744	100.0	11100	1.0836	0.4216	0.1455

Framework versions

• Transformers 4.57.3
• Pytorch 2.6.0+cu124
• Datasets 3.2.0
• Tokenizers 0.22.1