mt5-small-finetuned-amazon-es

This model is a fine-tuned version of google/mt5-small on the amazon_reviews_multi dataset. It achieves the following results on the evaluation set:

• Loss: 2.8877
• Rouge1: 12.5959
• Rouge2: 6.541
• Rougel: 12.4351
• Rougelsum: 12.2872

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
10.7598	1.0	439	3.3647	6.8015	1.522	6.7566	6.6935
4.6043	2.0	878	3.0154	13.0236	4.258	12.8286	12.8619
4.0593	3.0	1317	2.9726	12.6059	5.7546	12.2192	12.2128
3.82	4.0	1756	2.9247	14.4714	5.7779	14.1695	14.1091
3.6676	5.0	2195	2.9124	13.3845	6.7018	13.2512	13.1154
3.5678	6.0	2634	2.8913	12.9049	6.2959	12.7556	12.6255
3.4971	7.0	3073	2.8911	12.7622	6.5549	12.631	12.4987
3.4737	8.0	3512	2.8877	12.5959	6.541	12.4351	12.2872

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cpu
• Datasets 2.14.5
• Tokenizers 0.13.3