Se124M10KInfDelimiter

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5435

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3814	1.0	225	0.9747
0.2215	2.0	450	0.7053
0.1876	3.0	675	0.6494
0.1711	4.0	900	0.6211
0.1655	5.0	1125	0.6091
0.1611	6.0	1350	0.6025
0.1577	7.0	1575	0.5935
0.1553	8.0	1800	0.5883
0.154	9.0	2025	0.5816
0.1507	10.0	2250	0.5820
0.152	11.0	2475	0.5749
0.1487	12.0	2700	0.5773
0.1493	13.0	2925	0.5708
0.1457	14.0	3150	0.5685
0.1459	15.0	3375	0.5670
0.1442	16.0	3600	0.5670
0.1468	17.0	3825	0.5643
0.1444	18.0	4050	0.5608
0.1424	19.0	4275	0.5586
0.1439	20.0	4500	0.5606
0.1446	21.0	4725	0.5572
0.1428	22.0	4950	0.5575
0.1422	23.0	5175	0.5554
0.14	24.0	5400	0.5542
0.1395	25.0	5625	0.5545
0.1418	26.0	5850	0.5535
0.1393	27.0	6075	0.5504
0.1417	28.0	6300	0.5514
0.1419	29.0	6525	0.5516
0.1392	30.0	6750	0.5501
0.1403	31.0	6975	0.5492
0.1403	32.0	7200	0.5484
0.1414	33.0	7425	0.5476
0.1411	34.0	7650	0.5491
0.1379	35.0	7875	0.5467
0.1376	36.0	8100	0.5468
0.1393	37.0	8325	0.5454
0.1376	38.0	8550	0.5454
0.1388	39.0	8775	0.5459
0.1377	40.0	9000	0.5452
0.1409	41.0	9225	0.5447
0.1402	42.0	9450	0.5442
0.1401	43.0	9675	0.5445
0.1381	44.0	9900	0.5441
0.1371	45.0	10125	0.5444
0.1379	46.0	10350	0.5440
0.1369	47.0	10575	0.5437
0.1387	48.0	10800	0.5437
0.1379	49.0	11025	0.5438
0.1364	50.0	11250	0.5435

Framework versions

• PEFT 0.15.1
• Transformers 4.51.3
• Pytorch 2.6.0+cu118
• Datasets 3.5.0
• Tokenizers 0.21.1