flan-t5-base-xsum

This model is a fine-tuned version of google/flan-t5-base on the xsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.0798
• Rouge1: 32.3503
• Rouge2: 10.8909
• Rougel: 25.9346
• Rougelsum: 25.9216
• Gen Len: 18.8494

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adafactor
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.335	1.0	1417	2.0823	31.3453	10.2077	25.0051	25.008	18.8259
1.8642	2.0	2834	2.0798	32.3503	10.8909	25.9346	25.9216	18.8494
1.5208	3.0	4251	2.1272	32.6743	11.3394	26.3776	26.3724	18.8435
1.2628	4.0	5668	2.2110	32.695	11.3273	26.3215	26.322	18.8306
1.0649	5.0	7085	2.3143	32.5287	11.3662	26.274	26.2741	18.8345

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.10.0
• Tokenizers 0.13.2