t5-small-samsum

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7108
• Rouge1: 42.8796
• Rouge2: 19.1218
• Rougel: 35.393
• Rougelsum: 39.3635
• Gen Len: 16.8901

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.0185	1.0	1842	1.7918	40.569	17.0622	33.4617	37.1907	16.8938
1.8881	2.0	3684	1.7479	41.9209	18.5938	34.8969	38.5288	16.6435
1.8222	3.0	5526	1.7269	42.2611	19.1114	35.3077	39.0834	17.0696
1.8011	4.0	7368	1.7136	42.8138	19.2426	35.6329	39.4298	16.9158
1.7812	5.0	9210	1.7108	42.8796	19.1218	35.393	39.3635	16.8901

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0