T5-model-1-feedback-0510

This model is a fine-tuned version of theojolliffe/T5-model-1-feedback-1109 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2334
• Rouge1: 91.6115
• Rouge2: 86.7084
• Rougel: 91.0616
• Rougelsum: 91.1197
• Gen Len: 14.7895

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.4381	1.0	542	0.2731	90.324	84.2616	89.0178	89.1459	14.5614
0.2999	2.0	1084	0.2374	91.6458	86.2909	90.8275	90.8257	14.7719
0.273	3.0	1626	0.2382	91.4445	86.8218	91.1231	91.0886	14.8947
0.2248	4.0	2168	0.2334	91.6115	86.7084	91.0616	91.1197	14.7895

Framework versions

• Transformers 4.22.2
• Pytorch 1.12.1+cu113
• Datasets 2.5.2
• Tokenizers 0.12.1