train_svamp_42_1760637542

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0478
• Num Input Tokens Seen: 1433520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1167	1.0	158	0.0971	71568
0.0618	2.0	316	0.0714	143232
0.0498	3.0	474	0.0771	214912
0.0981	4.0	632	0.0580	286448
0.021	5.0	790	0.0575	358176
0.0334	6.0	948	0.0576	429728
0.0169	7.0	1106	0.0478	501504
0.0271	8.0	1264	0.0654	573120
0.0045	9.0	1422	0.0664	644944
0.013	10.0	1580	0.0617	716448
0.0002	11.0	1738	0.0896	788256
0.0001	12.0	1896	0.0903	859808
0.0005	13.0	2054	0.0926	931472
0.0003	14.0	2212	0.0921	1003376
0.0011	15.0	2370	0.0920	1075088
0.0002	16.0	2528	0.0923	1146608
0.0005	17.0	2686	0.0915	1218368
0.0007	18.0	2844	0.0920	1290144
0.0004	19.0	3002	0.0921	1361984
0.0003	20.0	3160	0.0926	1433520

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4