Vigec-V5

This model is a fine-tuned version of VietAI/vit5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3694
• Bleu: 77.0736
• Gen Len: 10.0475

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.195	0.01	500	0.9492	43.0845	7.2405
0.978	0.01	1000	0.7804	61.0671	9.7255
0.8418	0.02	1500	0.6798	64.3811	9.9025
0.8148	0.03	2000	0.6046	66.1944	10.043
0.7622	0.04	2500	0.5513	68.2851	10.1215
0.7199	0.04	3000	0.5146	69.7161	10.0795
0.7898	0.05	3500	0.4869	71.1868	10.079
0.6921	0.06	4000	0.4648	72.4203	10.0345
0.6827	0.07	4500	0.4490	73.2133	10.039
0.6102	0.07	5000	0.4355	73.6841	10.078
0.5805	0.08	5500	0.4176	74.2559	10.059
0.6806	0.09	6000	0.4081	74.7389	10.0655
0.6544	0.09	6500	0.3958	75.2603	10.025
0.6244	0.1	7000	0.3904	75.9306	10.0565
0.7212	0.11	7500	0.3822	76.3268	10.0505
0.5446	0.12	8000	0.3785	76.5306	10.0505
0.5574	0.12	8500	0.3741	76.7101	10.0545
0.6265	0.13	9000	0.3721	76.8858	10.043
0.5379	0.14	9500	0.3695	77.001	10.051
0.6164	0.14	10000	0.3694	77.0736	10.0475

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2