train_svamp_456_1760637771

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.5475
• Num Input Tokens Seen: 1274400

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6899	2.0	280	0.6427	127584
0.5563	4.0	560	0.5719	254624
0.4469	6.0	840	0.5160	381632
0.2577	8.0	1120	0.3559	508672
0.2412	10.0	1400	0.3456	636256
0.1586	12.0	1680	0.4186	764032
0.0672	14.0	1960	0.4333	891392
0.0119	16.0	2240	0.5066	1019136
0.0222	18.0	2520	0.5436	1146496
0.0265	20.0	2800	0.5475	1274400

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4