chess_gpt

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4539

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 96
• eval_batch_size: 96
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.9) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 200
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
4.8573	0.0942	500	4.6881
4.0115	0.1884	1000	3.8051
3.6439	0.2826	1500	3.4294
3.4261	0.3769	2000	3.2262
3.2929	0.4711	2500	3.0936
3.1784	0.5653	3000	2.9891
3.1042	0.6595	3500	2.9109
3.0348	0.7537	4000	2.8448
2.9826	0.8479	4500	2.7931
2.9282	0.9422	5000	2.7484
2.8859	1.0364	5500	2.7090
2.8502	1.1306	6000	2.6747
2.8225	1.2248	6500	2.6463
2.7970	1.3190	7000	2.6216
2.7716	1.4132	7500	2.5987
2.7500	1.5074	8000	2.5777
2.7315	1.6017	8500	2.5614
2.7069	1.6959	9000	2.5424
2.6970	1.7901	9500	2.5273
2.6773	1.8843	10000	2.5158
2.6660	1.9785	10500	2.5039
2.6524	2.0727	11000	2.4941
2.6415	2.1669	11500	2.4847
2.6324	2.2612	12000	2.4775
2.6318	2.3554	12500	2.4720
2.6227	2.4496	13000	2.4655
2.6203	2.5438	13500	2.4612
2.6169	2.6380	14000	2.4581
2.6174	2.7322	14500	2.4562
2.6103	2.8265	15000	2.4544
2.6106	2.9207	15500	2.4540
2.6079	3.0	15921	2.4539

Framework versions

• Transformers 5.9.0
• Pytorch 2.12.0+cu130
• Datasets 4.8.5
• Tokenizers 0.22.2