train_math_qa_1754507503

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.6573
• Num Input Tokens Seen: 38732208

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6543	0.5	3357	0.7342	1938656
0.7324	1.0	6714	0.7043	3870688
0.6434	1.5	10071	0.6895	5801696
0.781	2.0	13428	0.6818	7741288
0.7155	2.5	16785	0.6756	9678632
0.6397	3.0	20142	0.6714	11611120
0.7104	3.5	23499	0.6678	13550032
0.6319	4.0	26856	0.6657	15489040
0.6331	4.5	30213	0.6644	17420720
0.648	5.0	33570	0.6603	19360624
0.6333	5.5	36927	0.6591	21295472
0.6909	6.0	40284	0.6595	23230504
0.7496	6.5	43641	0.6592	25166536
0.7005	7.0	46998	0.6587	27107328
0.525	7.5	50355	0.6593	29044256
0.5706	8.0	53712	0.6573	30985288
0.6458	8.5	57069	0.6577	32925544
0.6022	9.0	60426	0.6583	34856680
0.6268	9.5	63783	0.6581	36790952
0.7933	10.0	67140	0.6578	38732208

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1