train_svamp_1757340225

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0874
• Num Input Tokens Seen: 704688

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.2101	0.5	79	2.0056	35424
1.1862	1.0	158	1.1302	70560
0.457	1.5	237	0.2695	105728
0.1313	2.0	316	0.1418	140912
0.1163	2.5	395	0.1159	176272
0.0736	3.0	474	0.1094	211360
0.0637	3.5	553	0.1043	246912
0.0416	4.0	632	0.0983	281968
0.1427	4.5	711	0.0975	317392
0.1333	5.0	790	0.0933	352128
0.0277	5.5	869	0.0912	387744
0.0551	6.0	948	0.0900	422800
0.0849	6.5	1027	0.0905	457968
0.0619	7.0	1106	0.0888	493104
0.0406	7.5	1185	0.0887	528304
0.0368	8.0	1264	0.0880	563936
0.0494	8.5	1343	0.0892	598976
0.031	9.0	1422	0.0880	634144
0.0562	9.5	1501	0.0874	669632
0.0145	10.0	1580	0.0879	704688

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1