macarious/torgo_xlsr_finetune_M04_old

torgo_xlsr_finetune_M04

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6944
• Wer: 0.2951

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.5657	0.75	1000	3.3394	1.0
1.6728	1.5	2000	2.0240	0.8222
0.8728	2.25	3000	1.6539	0.6160
0.6675	3.0	4000	1.6387	0.5754
0.5636	3.75	5000	1.6527	0.5277
0.4598	4.5	6000	1.6385	0.4130
0.3835	5.25	7000	1.5931	0.4311
0.3881	6.0	8000	1.1019	0.3776
0.3276	6.75	9000	1.4476	0.4317
0.3099	7.5	10000	1.5607	0.3885
0.2973	8.25	11000	1.5421	0.3338
0.2961	9.0	12000	1.6224	0.3802
0.2702	9.74	13000	1.9051	0.3563
0.2437	10.49	14000	1.8258	0.3789
0.2226	11.24	15000	1.5751	0.3351
0.2059	11.99	16000	1.5211	0.3260
0.2244	12.74	17000	1.3021	0.3196
0.1904	13.49	18000	1.8146	0.3557
0.2026	14.24	19000	1.6795	0.3305
0.1696	14.99	20000	1.4434	0.2990
0.1753	15.74	21000	1.5751	0.2957
0.1648	16.49	22000	1.7432	0.3048
0.1364	17.24	23000	1.8501	0.3267
0.1499	17.99	24000	1.6999	0.3048
0.1452	18.74	25000	1.7274	0.2970
0.1367	19.49	26000	1.6944	0.2951

Framework versions

• Transformers 4.26.1
• Pytorch 2.1.2
• Datasets 2.16.1
• Tokenizers 0.13.3