mt5-small-finetuned-amazon-zh-full

This model is a fine-tuned version of google/mt5-small on the amazon_reviews_multi dataset. It achieves the following results on the evaluation set:

• Loss: 3.2111
• Rouge1: 10.2419
• Rouge2: 4.7619
• Rougel: 10.2419
• Rougelsum: 10.2419

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
11.4787	1.0	229	4.4488	3.1349	1.3605	3.1349	3.1349
4.8938	2.0	458	3.7642	8.2011	2.381	8.2011	8.2011
4.0429	3.0	687	3.6643	10.2419	4.7619	10.2419	10.2419
3.7038	4.0	916	3.5566	10.2419	4.7619	10.2419	10.2419
3.5159	5.0	1145	3.4467	10.2419	4.7619	10.2419	10.2419
3.3715	6.0	1374	3.4368	10.2419	4.7619	10.2419	10.2419
3.2854	7.0	1603	3.3713	10.2419	4.7619	10.2419	10.2419
3.1206	8.0	1832	3.3467	10.2419	4.7619	10.2419	10.2419
3.0899	9.0	2061	3.2809	10.2419	4.7619	10.2419	10.2419
3.0251	10.0	2290	3.2420	10.2419	4.7619	10.2419	10.2419
2.9903	11.0	2519	3.2251	10.2419	4.7619	10.2419	10.2419
2.9599	12.0	2748	3.2408	10.2419	4.7619	10.2419	10.2419
2.9247	13.0	2977	3.2150	10.2419	4.7619	10.2419	10.2419
2.9131	14.0	3206	3.2121	10.2419	4.7619	10.2419	10.2419
2.8942	15.0	3435	3.2111	10.2419	4.7619	10.2419	10.2419

Framework versions

• Transformers 4.27.4
• Pytorch 1.13.0
• Datasets 2.1.0
• Tokenizers 0.13.2