train_math_qa_456_1760637836

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.7729
• Num Input Tokens Seen: 77891968

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8144	1.0	6714	0.8044	3900904
0.8048	2.0	13428	0.8056	7795688
0.7852	3.0	20142	0.7398	11690736
0.6744	4.0	26856	0.6758	15583992
0.6056	5.0	33570	0.6689	19477680
0.6001	6.0	40284	0.6656	23372072
0.571	7.0	46998	0.6610	27267240
0.5573	8.0	53712	0.6567	31161216
0.5417	9.0	60426	0.6567	35058040
0.5279	10.0	67140	0.6782	38955336
0.491	11.0	73854	0.6672	42849552
0.4705	12.0	80568	0.6986	46744544
0.4584	13.0	87282	0.7088	50638504
0.2365	14.0	93996	0.7527	54532704
0.5157	15.0	100710	0.7999	58424776
0.2978	16.0	107424	0.8175	62319120
0.4732	17.0	114138	0.9005	66209648
0.2986	18.0	120852	0.9290	70104328
0.38	19.0	127566	0.9597	73997656
0.1678	20.0	134280	0.9709	77891968

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4