ChessQween3-base

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9453

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 2
• total_train_batch_size: 512
• total_eval_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
3.7947	0.0256	2000	3.7973
3.2355	0.0512	4000	3.3104
3.1324	0.0768	6000	3.0861
3.0801	0.1025	8000	2.9574
2.9370	0.1281	10000	2.8806
2.8139	0.1537	12000	2.7924
2.7011	0.1793	14000	2.7364
2.5453	0.2049	16000	2.6827
2.6728	0.2305	18000	2.6320
2.5903	0.2561	20000	2.5825
2.5910	0.2817	22000	2.5478
2.4977	0.3074	24000	2.5110
2.5118	0.3330	26000	2.4747
2.4336	0.3586	28000	2.4338
2.4289	0.3842	30000	2.3929
2.3808	0.4098	32000	2.3621
2.3663	0.4354	34000	2.3340
2.3412	0.4610	36000	2.3026
2.3884	0.4866	38000	2.2735
2.3094	0.5123	40000	2.2408
2.3330	0.5379	42000	2.2155
2.1700	0.5635	44000	2.1895
2.1146	0.5891	46000	2.1646
2.1595	0.6147	48000	2.1398
2.0966	0.6403	50000	2.1140
2.2037	0.6659	52000	2.0877
1.9864	0.6915	54000	2.0719
2.0506	0.7172	56000	2.0551
1.9694	0.7428	58000	2.0328
2.1410	0.7684	60000	2.0160
2.0000	0.7940	62000	1.9990
1.9383	0.8196	64000	1.9859
2.0483	0.8452	66000	1.9706
1.9896	0.8708	68000	1.9616
1.9138	0.8965	70000	1.9552
2.0671	0.9221	72000	1.9515
1.9800	0.9477	74000	1.9467
1.9927	0.9733	76000	1.9455
1.9911	0.9989	78000	1.9453

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2