Se124M10KInfSimple

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5204

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3959	1.0	237	0.9383
0.2182	2.0	474	0.6835
0.1772	3.0	711	0.6222
0.1676	4.0	948	0.5980
0.1609	5.0	1185	0.5840
0.1519	6.0	1422	0.5740
0.1535	7.0	1659	0.5659
0.1492	8.0	1896	0.5587
0.1456	9.0	2133	0.5575
0.1427	10.0	2370	0.5528
0.1442	11.0	2607	0.5499
0.1403	12.0	2844	0.5466
0.1425	13.0	3081	0.5449
0.1403	14.0	3318	0.5414
0.1405	15.0	3555	0.5399
0.1387	16.0	3792	0.5383
0.1377	17.0	4029	0.5368
0.1379	18.0	4266	0.5376
0.1353	19.0	4503	0.5349
0.1378	20.0	4740	0.5313
0.1372	21.0	4977	0.5320
0.135	22.0	5214	0.5286
0.1361	23.0	5451	0.5282
0.1357	24.0	5688	0.5287
0.1372	25.0	5925	0.5269
0.1343	26.0	6162	0.5271
0.1321	27.0	6399	0.5255
0.1341	28.0	6636	0.5240
0.1346	29.0	6873	0.5235
0.1327	30.0	7110	0.5239
0.1335	31.0	7347	0.5230
0.1332	32.0	7584	0.5228
0.1317	33.0	7821	0.5234
0.1331	34.0	8058	0.5220
0.1327	35.0	8295	0.5213
0.1338	36.0	8532	0.5204
0.1306	37.0	8769	0.5206
0.1317	38.0	9006	0.5208

Framework versions

• PEFT 0.15.1
• Transformers 4.51.3
• Pytorch 2.6.0+cu118
• Datasets 3.5.0
• Tokenizers 0.21.1