lyric-gpt

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.4268

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
3.4133	1.0	92	3.1858
2.8751	2.0	184	3.2040
2.9241	3.0	276	3.2442
2.8796	4.0	368	3.2893
2.5508	5.0	460	3.3988
2.3505	6.0	552	3.4852
2.1709	7.0	644	3.5973
1.9833	8.0	736	3.7801
1.9428	9.0	828	3.8925
1.6357	10.0	920	4.0321
1.6304	11.0	1012	4.1989
1.5399	12.0	1104	4.3302
1.3642	13.0	1196	4.4324
1.2646	14.0	1288	4.5765
1.0898	15.0	1380	4.7299
1.0398	16.0	1472	4.7749
0.945	17.0	1564	4.8901
0.8175	18.0	1656	4.9615
0.8893	19.0	1748	5.0045
0.8573	20.0	1840	5.1104
0.7946	21.0	1932	5.1732
0.713	22.0	2024	5.2327
0.7842	23.0	2116	5.2922
0.6976	24.0	2208	5.2967
0.6625	25.0	2300	5.3163
0.6503	26.0	2392	5.3695
0.5927	27.0	2484	5.3891
0.6025	28.0	2576	5.4029
0.6163	29.0	2668	5.4203
0.5178	30.0	2760	5.4268

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0