train_svamp_1757340224

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0529
• Num Input Tokens Seen: 704688

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1935	0.5	79	0.1355	35424
0.0734	1.0	158	0.0760	70560
0.1667	1.5	237	0.0702	105728
0.0488	2.0	316	0.0673	140912
0.0528	2.5	395	0.0529	176272
0.0324	3.0	474	0.0610	211360
0.0043	3.5	553	0.0644	246912
0.004	4.0	632	0.0600	281968
0.0148	4.5	711	0.0767	317392
0.0004	5.0	790	0.0821	352128
0.0001	5.5	869	0.0799	387744
0.0	6.0	948	0.0822	422800
0.0	6.5	1027	0.0815	457968
0.0001	7.0	1106	0.0889	493104
0.0001	7.5	1185	0.0849	528304
0.0	8.0	1264	0.0827	563936
0.0	8.5	1343	0.0837	598976
0.0001	9.0	1422	0.0838	634144
0.0001	9.5	1501	0.0857	669632
0.0	10.0	1580	0.0833	704688

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1