constrative_keyphrases

This model is a fine-tuned version of Salesforce/codet5p-220m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5247
• Rouge1: 33.8012
• Rouge2: 12.5322
• Rougel: 29.7596
• Rougelsum: 29.8083
• Gen Len: 13.4644

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 192
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
41.5752	1.0	1250	0.9700	28.8691	7.8102	24.8577	24.9034	13.4083
14.9865	2.0	2500	0.5539	31.9887	11.1384	28.0777	28.1287	13.4395
13.8288	3.0	3750	0.5372	32.7235	11.7772	28.8147	28.8678	13.3851
13.1334	4.0	5000	0.5297	33.4074	12.2326	29.3845	29.4305	13.5744
12.7533	5.0	6250	0.5270	33.4695	12.249	29.4447	29.4954	13.1345
12.5021	6.0	7500	0.5255	33.6736	12.4754	29.6942	29.74	13.3991
12.2308	7.0	8750	0.5240	33.7476	12.5094	29.7432	29.7943	13.5127
12.2174	8.0	10000	0.5247	33.8012	12.5322	29.7596	29.8083	13.4644

Framework versions

• Transformers 4.53.2
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2