train_math_qa_456_1760637837

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.6337
• Num Input Tokens Seen: 77891968

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8699	1.0	6714	0.6626	3900904
0.6772	2.0	13428	0.6337	7795688
0.3656	3.0	20142	0.6616	11690736
0.4412	4.0	26856	0.7169	15583992
0.2704	5.0	33570	0.8352	19477680
0.4171	6.0	40284	1.0136	23372072
0.4404	7.0	46998	1.0930	27267240
0.1196	8.0	53712	1.2952	31161216
0.348	9.0	60426	1.4367	35058040
0.0459	10.0	67140	1.7610	38955336
0.1753	11.0	73854	1.7154	42849552
0.0863	12.0	80568	1.7316	46744544
0.1252	13.0	87282	2.0660	50638504
0.0001	14.0	93996	2.1376	54532704
0.0	15.0	100710	2.6822	58424776
0.0	16.0	107424	2.6007	62319120
0.0	17.0	114138	2.9885	66209648
0.0	18.0	120852	3.5015	70104328
0.0	19.0	127566	3.6769	73997656
0.0	20.0	134280	3.6991	77891968

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4