vietnamese_mt5_summary_model

This model is a fine-tuned version of huggingface-course/mt5-finetuned-amazon-en-es on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4447
• Rouge1: 28.6205
• Rouge2: 12.1892
• Rougel: 22.6626
• Rougelsum: 22.9635

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.7781	1.0	625	1.8391	28.5024	11.2717	22.108	22.4361
2.0622	2.0	1250	1.7576	28.0245	10.6112	21.7353	22.0685
1.8636	3.0	1875	1.6179	27.353	10.6238	21.4686	21.7512
1.7408	4.0	2500	1.6143	28.0928	11.2857	22.06	22.3629
1.6492	5.0	3125	1.5411	27.8209	10.9184	21.6819	21.9773
1.5448	6.0	3750	1.4802	28.0433	11.4232	22.0696	22.373
1.4454	7.0	4375	1.4621	27.8552	11.1708	21.8958	22.1949
1.3636	8.0	5000	1.4522	28.3264	11.7945	22.3563	22.6524
1.2978	9.0	5625	1.4347	28.444	11.9388	22.4279	22.7344
1.2445	10.0	6250	1.4447	28.6205	12.1892	22.6626	22.9635

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1