apac_5sents_XLS-R_2_1e-6_10000

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 235.4551
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
283.4719	5.54	100	275.8542	1.1562
284.9797	11.11	200	275.2194	1.1317
281.7698	16.65	300	272.7092	1.0714
277.0381	22.22	400	262.7592	1.0022
255.1038	27.76	500	235.9033	1.0
210.0999	33.32	600	187.6442	1.0
165.675	38.86	700	147.7660	1.0
136.7341	44.43	800	119.9158	1.0
115.6304	49.97	900	103.3855	1.0
102.5195	55.54	1000	92.6710	1.0
92.8874	61.11	1100	85.5879	1.0
85.7572	66.65	1200	80.7715	1.0
82.2298	72.22	1300	77.2967	1.0
78.2917	77.76	1400	74.5786	1.0
76.1737	83.32	1500	72.3749	1.0
73.4032	88.86	1600	70.5145	1.0
72.0308	94.43	1700	68.8751	1.0
70.2158	99.97	1800	67.3867	1.0
69.0366	105.54	1900	66.0221	1.0
67.505	111.11	2000	64.7385	1.0
65.9735	116.65	2100	63.5326	1.0
64.9547	122.22	2200	62.3629	1.0
63.5495	127.76	2300	61.2392	1.0
62.5682	133.32	2400	60.1489	1.0
61.3491	138.86	2500	59.0903	1.0
60.4633	144.43	2600	58.0601	1.0
59.1225	149.97	2700	57.0598	1.0
58.4608	155.54	2800	56.0791	1.0
57.3886	161.11	2900	55.1206	1.0
56.0671	166.65	3000	54.1854	1.0
55.4618	172.22	3100	53.2703	1.0
54.3707	177.76	3200	52.3758	1.0
53.5913	183.32	3300	51.4962	1.0
52.3333	188.86	3400	50.6445	1.0
51.9007	194.43	3500	49.8066	1.0
50.7232	199.97	3600	48.9866	1.0
50.1234	205.54	3700	48.1893	1.0
49.3482	211.11	3800	47.4028	1.0
48.2667	216.65	3900	46.6400	1.0
47.6979	222.22	4000	45.8918	1.0
46.8316	227.76	4100	45.1604	1.0
46.2709	233.32	4200	44.4558	1.0
45.2221	238.86	4300	43.7601	1.0
44.7131	244.43	4400	43.0846	1.0
43.9629	249.97	4500	42.4227	1.0
43.4376	255.54	4600	41.7802	1.0
42.8012	261.11	4700	41.1533	1.0
41.963	266.65	4800	40.5411	1.0
41.5048	272.22	4900	39.9454	1.0
40.7667	277.76	5000	39.3722	1.0
40.3653	283.32	5100	38.8072	1.0
39.4755	288.86	5200	38.2610	1.0
39.2028	294.43	5300	37.7295	1.0
38.5202	299.97	5400	37.2128	1.0
38.1717	305.54	5500	36.7152	1.0
37.6775	311.11	5600	36.2277	1.0
36.9204	316.65	5700	35.7579	1.0
36.6353	322.22	5800	35.2993	1.0
36.0619	327.76	5900	34.8580	1.0
35.7494	333.32	6000	34.4316	1.0
35.0993	338.86	6100	34.0134	1.0
34.903	344.43	6200	33.6148	1.0
34.3435	349.97	6300	33.2321	1.0
34.1451	355.54	6400	32.8573	1.0
33.663	361.11	6500	32.4956	1.0
33.1929	366.65	6600	32.1511	1.0
32.9852	372.22	6700	31.8129	1.0
32.5427	377.76	6800	31.4935	1.0
32.3938	383.32	6900	31.1799	1.0
31.777	388.86	7000	30.8769	1.0
31.7616	394.43	7100	30.5909	1.0
31.2618	399.97	7200	30.3215	1.0
31.2144	405.54	7300	30.0532	1.0
30.8797	411.11	7400	29.8014	1.0
30.4686	416.65	7500	29.5594	1.0
30.3715	422.22	7600	29.3304	1.0
30.0413	427.76	7700	29.1044	1.0
29.9067	433.32	7800	28.9034	1.0
29.5911	438.86	7900	28.7019	1.0
29.4913	444.43	8000	28.5096	1.0
29.2069	449.97	8100	28.3284	1.0
29.1644	455.54	8200	28.1615	1.0
29.012	461.11	8300	28.0040	1.0
28.665	466.65	8400	27.8538	1.0
28.6858	472.22	8500	27.7132	1.0
28.4118	477.76	8600	27.5848	1.0
28.3825	483.32	8700	27.4693	1.0
28.1234	488.86	8800	27.3570	1.0
28.1963	494.43	8900	27.2488	1.0
27.9488	499.97	9000	27.1602	1.0
28.0135	505.54	9100	27.0756	1.0
27.894	511.11	9200	26.9989	1.0
27.6633	516.65	9300	26.9334	1.0
27.768	522.22	9400	26.8775	1.0
27.6398	527.76	9500	26.8242	1.0
27.6333	533.32	9600	26.7874	1.0
27.5064	538.86	9700	26.7575	1.0
27.601	544.43	9800	26.7354	1.0
27.4418	549.97	9900	26.7223	1.0
27.6197	555.54	10000	26.7163	1.0

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.1+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3