train_svamp_1757340177

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.1559
• Num Input Tokens Seen: 704336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.3451	0.5	79	2.3240	35680
1.8301	1.0	158	1.8187	70512
1.4461	1.5	237	1.4272	105904
1.0701	2.0	316	1.0491	140960
0.7459	2.5	395	0.7375	176096
0.5271	3.0	474	0.5291	211424
0.3739	3.5	553	0.3848	246784
0.397	4.0	632	0.2966	281968
0.2722	4.5	711	0.2431	317232
0.1673	5.0	790	0.2115	352368
0.2107	5.5	869	0.1906	387824
0.1631	6.0	948	0.1795	422704
0.1419	6.5	1027	0.1703	457744
0.1859	7.0	1106	0.1647	493200
0.1359	7.5	1185	0.1607	528304
0.2939	8.0	1264	0.1594	563520
0.1592	8.5	1343	0.1569	599072
0.1264	9.0	1422	0.1560	634176
0.1811	9.5	1501	0.1563	669440
0.1813	10.0	1580	0.1559	704336

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1