flan-t5-base-samsum

This model is a fine-tuned version of google/flan-t5-base on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.3733
• Rouge1: 47.0613
• Rouge2: 23.3862
• Rougel: 39.7982
• Rougelsum: 43.3754
• Gen Len: 17.2308

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	461	1.3902	46.6525	23.0352	39.1875	42.9396	17.2601
1.4553	2.0	922	1.3787	47.2118	23.4378	39.5672	43.4137	17.1355
1.3763	3.0	1383	1.3737	47.2951	23.5914	39.7571	43.457	17.1966
1.3412	4.0	1844	1.3750	47.2382	23.697	39.9147	43.5955	17.3443
1.3045	5.0	2305	1.3733	47.0613	23.3862	39.7982	43.3754	17.2308

Framework versions

• Transformers 4.42.4
• Pytorch 1.13.1+cu117
• Datasets 2.20.0
• Tokenizers 0.19.1