xsum-t5-small

This model is a fine-tuned version of t5-small on the xsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.4789
• Rouge1: 28.3309
• Rouge2: 7.7568
• Rougel: 22.2948
• Rougelsum: 22.2942
• Gen Len: 18.824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.9158	0.16	2000	2.5725	26.6629	6.6436	20.8032	20.7995	18.7886
2.7868	0.31	4000	2.5286	27.3979	7.1077	21.4451	21.4487	18.8045
2.756	0.47	6000	2.5058	27.8049	7.4383	21.8465	21.8479	18.8179
2.7388	0.63	8000	2.4903	28.1541	7.6412	22.1566	22.1572	18.8265
2.7208	0.78	10000	2.4819	28.2559	7.6877	22.2086	22.2118	18.8268
2.7175	0.94	12000	2.4789	28.3309	7.7568	22.2948	22.2942	18.824

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3