train_svamp_1755615499

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.1893
• Num Input Tokens Seen: 705184

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7697	0.5	79	0.6681	35776
0.5968	1.0	158	0.5173	70672
0.1124	1.5	237	0.1794	105904
0.132	2.0	316	0.1370	141328
0.1259	2.5	395	0.1006	176752
0.0482	3.0	474	0.0846	211808
0.0378	3.5	553	0.1207	247104
0.0761	4.0	632	0.0935	282048
0.0108	4.5	711	0.1449	317248
0.0208	5.0	790	0.1160	352592
0.0152	5.5	869	0.1450	388176
0.0132	6.0	948	0.1488	423184
0.0151	6.5	1027	0.1474	458640
0.0004	7.0	1106	0.1693	493440
0.0006	7.5	1185	0.1817	528768
0.0001	8.0	1264	0.1838	563872
0.0	8.5	1343	0.1869	599232
0.0002	9.0	1422	0.1876	634544
0.0004	9.5	1501	0.1893	670064
0.0001	10.0	1580	0.1893	705184

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1