train_svamp_123_1760637656

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.4480
• Num Input Tokens Seen: 1274432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5564	2.0	280	0.6053	127584
0.4667	4.0	560	0.5357	254816
0.3403	6.0	840	0.3506	382304
0.3517	8.0	1120	0.3066	509696
0.1972	10.0	1400	0.2995	637504
0.0951	12.0	1680	0.3259	765120
0.0939	14.0	1960	0.3630	892320
0.0831	16.0	2240	0.4137	1019456
0.0606	18.0	2520	0.4408	1147104
0.0861	20.0	2800	0.4480	1274432

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4