train_gsm8k_123_1760637708

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the gsm8k dataset. It achieves the following results on the evaluation set:

• Loss: 0.5359
• Num Input Tokens Seen: 30837792

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4825	2.0	2990	0.5423	3073904
0.4576	4.0	5980	0.5077	6160576
0.5076	6.0	8970	0.4979	9246832
0.3616	8.0	11960	0.4952	12330800
0.4245	10.0	14950	0.4962	15414800
0.3441	12.0	17940	0.5092	18492000
0.3093	14.0	20930	0.5210	21579648
0.3145	16.0	23920	0.5284	24669056
0.3427	18.0	26910	0.5340	27752896
0.3472	20.0	29900	0.5359	30837792

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4