train_math_qa_789_1760637953

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.6617
• Num Input Tokens Seen: 77933776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7488	1.0	6714	0.7173	3898224
0.7462	2.0	13428	0.6924	7796616
0.6355	3.0	20142	0.6810	11688128
0.7012	4.0	26856	0.6748	15585640
0.6854	5.0	33570	0.6704	19481256
0.5453	6.0	40284	0.6689	23379928
0.6057	7.0	46998	0.6661	27274992
0.4848	8.0	53712	0.6656	31169464
0.5169	9.0	60426	0.6635	35061680
0.4915	10.0	67140	0.6654	38957336
0.8056	11.0	73854	0.6617	42854488
0.5765	12.0	80568	0.6670	46754376
0.6986	13.0	87282	0.6658	50647376
0.5473	14.0	93996	0.6650	54543272
0.5868	15.0	100710	0.6653	58447368
0.4529	16.0	107424	0.6674	62343120
0.6316	17.0	114138	0.6685	66240072
0.6377	18.0	120852	0.6683	70140040
0.5319	19.0	127566	0.6683	74035080
0.6213	20.0	134280	0.6690	77933776

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4