train_svamp_789_1760637890

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 2.3867
• Num Input Tokens Seen: 1430656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.48	1.0	158	2.4252	71280
2.4184	2.0	316	2.4125	143040
2.4017	3.0	474	2.4047	214736
2.3823	4.0	632	2.3957	286128
2.3858	5.0	790	2.3922	357600
2.2849	6.0	948	2.3877	428976
2.3676	7.0	1106	2.3900	500400
2.2794	8.0	1264	2.3879	572048
2.2344	9.0	1422	2.3916	643792
2.2348	10.0	1580	2.3898	715344
2.3841	11.0	1738	2.3881	786864
2.2463	12.0	1896	2.3896	858288
2.3311	13.0	2054	2.3867	929616
2.442	14.0	2212	2.3892	1001040
2.4143	15.0	2370	2.3873	1072880
2.2956	16.0	2528	2.3872	1144416
2.2352	17.0	2686	2.3907	1215856
2.3394	18.0	2844	2.3906	1287504
2.3437	19.0	3002	2.3875	1359168
2.4724	20.0	3160	2.3875	1430656

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4