train_math_qa_456_1760637835

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.8014
• Num Input Tokens Seen: 77891968

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8169	1.0	6714	0.8234	3900904
0.8129	2.0	13428	0.8164	7795688
0.8112	3.0	20142	0.8132	11690736
0.8093	4.0	26856	0.8175	15583992
0.8112	5.0	33570	0.8224	19477680
0.8074	6.0	40284	0.8172	23372072
0.8208	7.0	46998	0.8184	27267240
0.8109	8.0	53712	0.8154	31161216
0.8223	9.0	60426	0.8197	35058040
0.7375	10.0	67140	0.8352	38955336
0.8125	11.0	73854	0.8076	42849552
0.8019	12.0	80568	0.8033	46744544
0.814	13.0	87282	0.8069	50638504
0.8091	14.0	93996	0.8173	54532704
0.8117	15.0	100710	0.8018	58424776
0.801	16.0	107424	0.8020	62319120
0.8217	17.0	114138	0.8018	66209648
0.8155	18.0	120852	0.8014	70104328
0.785	19.0	127566	0.8017	73997656
0.8233	20.0	134280	0.8015	77891968

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4