train_gsm8k_123_1760637712

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the gsm8k dataset. It achieves the following results on the evaluation set:

• Loss: 1.3631
• Num Input Tokens Seen: 34679720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.4787	1.0	1682	1.3741	1738584
1.2875	2.0	3364	1.3661	3477544
1.5066	3.0	5046	1.3638	5217560
1.4441	4.0	6728	1.3635	6947896
1.397	5.0	8410	1.3632	8680664
1.5074	6.0	10092	1.3634	10414768
1.3287	7.0	11774	1.3638	12148592
1.2467	8.0	13456	1.3635	13883456
1.4459	9.0	15138	1.3639	15613432
1.345	10.0	16820	1.3634	17345296
1.241	11.0	18502	1.3638	19082288
1.207	12.0	20184	1.3637	20812896
1.3701	13.0	21866	1.3634	22544592
1.2795	14.0	23548	1.3631	24280408
1.3116	15.0	25230	1.3636	26007520
1.4023	16.0	26912	1.3632	27741272
1.4232	17.0	28594	1.3633	29470344
1.2621	18.0	30276	1.3633	31206528
1.4401	19.0	31958	1.3633	32940696
1.5504	20.0	33640	1.3633	34679720

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4