train_math_qa_101112_1760638062

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.7990
• Num Input Tokens Seen: 77914328

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7903	1.0	6714	0.8025	3894384
0.818	2.0	13428	0.8035	7788792
0.8281	3.0	20142	0.8016	11683344
0.8043	4.0	26856	0.8015	15578064
0.7775	5.0	33570	0.8022	19479304
0.8063	6.0	40284	0.8024	23378352
0.8281	7.0	46998	0.8015	27274568
0.8305	8.0	53712	0.8033	31172664
0.8037	9.0	60426	0.8033	35068368
0.8123	10.0	67140	0.8025	38966392
0.8186	11.0	73854	0.8020	42861936
0.7842	12.0	80568	0.7999	46756048
0.7932	13.0	87282	0.7990	50652416
0.8127	14.0	93996	0.8016	54546936
0.8205	15.0	100710	0.8013	58442960
0.8368	16.0	107424	0.8020	62338944
0.8048	17.0	114138	0.8010	66231336
0.7987	18.0	120852	0.8009	70127040
0.8071	19.0	127566	0.8017	74021072
0.8294	20.0	134280	0.8014	77914328

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4