T5-model-1-feedback-2010-e4

This model is a fine-tuned version of theojolliffe/T5-model-1-feedback-1109 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2075
• Rouge1: 92.2165
• Rouge2: 86.2314
• Rougel: 91.5975
• Rougelsum: 91.509
• Gen Len: 15.2586

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.2922	1.0	1646	0.2603	91.6366	84.8657	90.8246	90.9026	15.092
0.2264	2.0	3292	0.2311	92.5522	86.8008	91.9435	91.88	15.2586
0.187	3.0	4938	0.2085	91.982	86.0585	91.3852	91.3091	15.3161
0.1879	4.0	6584	0.2075	92.2165	86.2314	91.5975	91.509	15.2586

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1