train_math_qa_42_1760637609

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.6714
• Num Input Tokens Seen: 77902976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6933	1.0	6714	0.7318	3894552
0.7755	2.0	13428	0.7054	7790784
0.7469	3.0	20142	0.6915	11684296
0.7618	4.0	26856	0.6807	15578848
0.7027	5.0	33570	0.6760	19476576
0.5926	6.0	40284	0.6747	23368392
0.7775	7.0	46998	0.6742	27263880
0.5752	8.0	53712	0.6715	31154960
0.6372	9.0	60426	0.6733	35053760
0.8978	10.0	67140	0.6714	38947216
0.704	11.0	73854	0.6760	42844400
0.5709	12.0	80568	0.6810	46741816
0.5736	13.0	87282	0.6778	50638456
0.6408	14.0	93996	0.6757	54533112
0.5423	15.0	100710	0.6810	58429624
0.6276	16.0	107424	0.6797	62323952
0.6063	17.0	114138	0.6825	66220752
0.6836	18.0	120852	0.6827	70113640
0.5667	19.0	127566	0.6828	74009160
0.4553	20.0	134280	0.6824	77902976

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4