BMU_Finetuned_GPT2_model_version_2_MedQUAD

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 6.5607

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
3.389	1.0	2461	2.8390
2.6842	2.0	4922	2.6354
2.1162	3.0	7383	2.5356
1.528	4.0	9844	2.5343
1.5564	5.0	12305	2.6331
1.3195	6.0	14766	2.7461
1.1524	7.0	17227	2.8844
1.0166	8.0	19688	3.0437
0.7149	9.0	22149	3.2543
0.5399	10.0	24610	3.5052
0.5829	11.0	27071	3.6106
0.4897	12.0	29532	3.6989
0.4379	13.0	31993	3.8391
0.4071	14.0	34454	3.9292
0.3563	15.0	36915	4.0696
0.327	16.0	39376	4.1101
0.3359	17.0	41837	4.1817
0.2941	18.0	44298	4.2695
0.263	19.0	46759	4.2825
0.1996	20.0	49220	4.4129
0.2273	21.0	51681	4.3808
0.2038	22.0	54142	4.4962
0.1763	23.0	56603	4.6193
0.1995	24.0	59064	4.6855
0.1842	25.0	61525	4.6419
0.1655	26.0	63986	4.7630
0.15	27.0	66447	4.8052
0.139	28.0	68908	4.8711
0.1421	29.0	71369	4.9689
0.143	30.0	73830	4.9929
0.1496	31.0	76291	4.9971
0.1171	32.0	78752	5.0850
0.1128	33.0	81213	5.1968
0.1188	34.0	83674	5.2766
0.1214	35.0	86135	5.3399
0.1206	36.0	88596	5.3933
0.0977	37.0	91057	5.4530
0.104	38.0	93518	5.6013
0.1041	39.0	95979	5.6687
0.0852	40.0	98440	5.7046
0.0898	41.0	100901	5.9083
0.079	42.0	103362	5.9054
0.0848	43.0	105823	6.0329
0.0866	44.0	108284	6.0944
0.0875	45.0	110745	6.2469
0.0748	46.0	113206	6.2711
0.0797	47.0	115667	6.3853
0.0702	48.0	118128	6.4990
0.0733	49.0	120589	6.5186
0.0814	50.0	123050	6.5607

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.3.1
• Tokenizers 0.21.0