pargraphs_titlesV1.2

This model is a fine-tuned version of AhmedTaha012/pargraphs_titlesV1.1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2078
• Rouge1: 85.6494
• Rouge2: 78.7423
• Rougel: 85.6468
• Rougelsum: 85.6405
• Gen Len: 55044224.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 10
• total_train_batch_size: 40
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.2417	0.98	50	0.2071	85.4314	78.3381	85.4768	85.5142	55044224.0
0.2245	1.99	101	0.2075	85.5919	78.5428	85.6216	85.6173	55044224.0
0.2182	2.99	152	0.2076	85.8144	78.9212	85.8405	85.8498	55044224.0
0.2122	4.0	203	0.2078	85.7866	78.9183	85.7687	85.7753	55044224.0
0.2078	5.0	254	0.2075	85.6336	78.7705	85.6598	85.6483	55044224.0
0.2027	5.91	300	0.2078	85.6494	78.7423	85.6468	85.6405	55044224.0

Framework versions

• Transformers 4.36.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0