torgo_xlsr_finetune_F01_keep_all

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4829
• Wer: 0.2307

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.5238	0.54	1000	3.3622	1.0
2.0516	1.08	2000	1.8955	0.8696
1.0289	1.62	3000	1.6729	0.7099
0.7811	2.15	4000	1.4110	0.5810
0.6504	2.69	5000	1.3730	0.4746
0.5201	3.23	6000	1.3490	0.4285
0.4961	3.77	7000	1.4873	0.4758
0.4362	4.31	8000	1.4745	0.4083
0.4124	4.85	9000	1.5057	0.3881
0.3923	5.39	10000	1.2302	0.3473
0.364	5.92	11000	1.4894	0.3351
0.3295	6.46	12000	1.4680	0.3412
0.3246	7.0	13000	1.3344	0.3206
0.2864	7.54	14000	1.4118	0.3050
0.2897	8.08	15000	1.4808	0.2993
0.2376	8.62	16000	1.7672	0.3145
0.2539	9.15	17000	1.5791	0.3035
0.2424	9.69	18000	1.3562	0.2867
0.2266	10.23	19000	1.5603	0.2753
0.213	10.77	20000	1.3815	0.2673
0.2296	11.31	21000	1.1711	0.2482
0.2094	11.85	22000	1.2037	0.2490
0.1901	12.39	23000	1.3386	0.2451
0.1849	12.92	24000	1.5142	0.2482
0.2062	13.46	25000	1.4039	0.2409
0.1654	14.0	26000	1.4757	0.2398
0.1926	14.54	27000	1.3155	0.2276
0.153	15.08	28000	1.6762	0.2551
0.1513	15.62	29000	1.4876	0.2505
0.1491	16.16	30000	1.3197	0.2280
0.1429	16.69	31000	1.4292	0.2371
0.1438	17.23	32000	1.3700	0.2310
0.1335	17.77	33000	1.4660	0.2265
0.1386	18.31	34000	1.5169	0.2291
0.1201	18.85	35000	1.5295	0.2291
0.1238	19.39	36000	1.4762	0.2287
0.1192	19.92	37000	1.4829	0.2307

Framework versions

• Transformers 4.26.1
• Pytorch 2.2.0
• Datasets 2.16.1
• Tokenizers 0.13.3