apac_5sents_XLS-R_5e-5

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6693
• Wer: 0.6607

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 20000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
98.1859	11.11	200	75.6610	1.0
66.0186	22.22	400	52.7194	1.0
40.6772	33.32	600	26.2206	1.0
16.4279	44.43	800	8.3596	1.0
6.1402	55.54	1000	5.3091	1.0
4.733	66.65	1200	4.5618	1.0
3.4737	77.76	1400	3.4971	1.0
1.6412	88.86	1600	2.7199	0.9219
0.8524	99.97	1800	2.0043	0.9241
0.6227	111.11	2000	2.1371	0.9330
0.4834	122.22	2200	1.6094	0.9196
0.3944	133.32	2400	2.2378	0.9531
0.2975	144.43	2600	1.7241	0.8862
0.194	155.54	2800	1.8496	0.7679
0.1211	166.65	3000	1.4915	0.6808
0.0866	177.76	3200	1.9463	0.6429
0.0696	188.86	3400	2.0191	0.5737
0.0563	199.97	3600	2.4305	0.6562
0.0461	211.11	3800	1.6485	0.6138
0.0413	222.22	4000	2.2569	0.6272

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.1+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3