t5-small-kaikouraKP

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7249
• Rouge1: 42.6897
• Rouge2: 32.7199
• Rougel: 42.3348
• Rougelsum: 42.2663
• Gen Len: 7.3490

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.1273	1.0	6027	1.7432	41.184	30.8391	40.7699	40.7216	7.4856
0.9409	2.0	12054	1.7249	42.6897	32.7199	42.3348	42.2663	7.3490
0.8509	3.0	18081	1.7455	42.3295	31.679	41.9898	41.9304	6.6437
0.8181	4.0	24108	1.7340	42.5976	32.262	42.3525	42.2605	7.1271
0.7525	5.0	30135	1.7586	43.2207	32.8503	42.8558	42.8051	7.0032
0.7207	6.0	36162	1.7574	42.561	32.0232	42.2216	42.1765	6.7649
0.7126	7.0	42189	1.7606	42.8581	32.3955	42.4638	42.437	7.0624
0.6731	8.0	48216	1.7660	43.1632	32.8011	42.8033	42.7771	7.0314

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2