5sents_XLS-R_2_e-4

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3629
• Wer: 0.2063

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 400
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
70.1025	99.89	200	23.5652	1.0
17.8988	199.89	400	10.4265	1.0
7.3246	299.89	600	4.0969	1.0
3.5815	399.89	800	2.4899	1.0
1.4553	499.89	1000	1.3636	0.7354
0.3355	599.89	1200	1.4502	0.3651
0.1232	699.89	1400	0.8715	0.3280
0.0615	799.89	1600	0.9018	0.3968
0.0372	899.89	1800	1.7271	0.4339
0.0247	999.89	2000	0.6459	0.2751
0.0166	1099.89	2200	0.4516	0.2540
0.0216	1199.89	2400	0.6955	0.2487
0.0093	1299.89	2600	1.1281	0.2646
0.0084	1399.89	2800	0.6150	0.1376
0.0076	1499.89	3000	1.1476	0.2646
0.0125	1599.89	3200	1.0682	0.2487
0.0096	1699.89	3400	0.8676	0.2487
0.0121	1799.89	3600	2.8241	0.2963
0.0107	1899.89	3800	0.3758	0.2381
0.0107	1999.89	4000	0.8708	0.2381
0.0051	2099.89	4200	0.8423	0.2804
0.0081	2199.89	4400	0.9489	0.2698
0.0044	2299.89	4600	0.8984	0.2857
0.0026	2399.89	4800	0.5836	0.2328
0.0169	2499.89	5000	0.9432	0.2434
0.0055	2599.89	5200	0.4225	0.2381
0.0033	2699.89	5400	1.1866	0.1693
0.0019	2799.89	5600	0.6218	0.1746
0.002	2899.89	5800	0.3831	0.1799
0.0026	2999.89	6000	0.6229	0.1323

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.1+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3