train_math_qa_42_1760637606

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.7766
• Num Input Tokens Seen: 77902976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7892	1.0	6714	0.8037	3894552
0.7597	2.0	13428	0.6988	7790784
0.7599	3.0	20142	0.6781	11684296
0.7323	4.0	26856	0.6678	15578848
0.7219	5.0	33570	0.6676	19476576
0.5766	6.0	40284	0.6635	23368392
0.8267	7.0	46998	0.6693	27263880
0.6	8.0	53712	0.6668	31154960
0.5149	9.0	60426	0.6709	35053760
0.9571	10.0	67140	0.6944	38947216
0.5364	11.0	73854	0.6989	42844400
0.3756	12.0	80568	0.7338	46741816
0.3974	13.0	87282	0.7748	50638456
0.435	14.0	93996	0.8380	54533112
0.222	15.0	100710	0.9106	58429624
0.2255	16.0	107424	0.9085	62323952
0.4261	17.0	114138	1.0018	66220752
0.4205	18.0	120852	1.0642	70113640
0.0826	19.0	127566	1.1141	74009160
0.2447	20.0	134280	1.1284	77902976

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4