train_svamp_123_1760637660

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 2.2963
• Num Input Tokens Seen: 1433472

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.4426	1.0	158	2.3351	71776
2.2659	2.0	316	2.3266	143552
2.3506	3.0	474	2.3136	215312
2.346	4.0	632	2.3081	286736
2.3631	5.0	790	2.3011	358416
2.4453	6.0	948	2.2989	430224
2.4361	7.0	1106	2.2981	501728
2.303	8.0	1264	2.2972	573328
2.3642	9.0	1422	2.2972	645088
2.3351	10.0	1580	2.2982	716976
2.2508	11.0	1738	2.2963	788752
2.3046	12.0	1896	2.2975	860352
2.1803	13.0	2054	2.2973	931872
2.332	14.0	2212	2.2981	1003600
2.3383	15.0	2370	2.2965	1075312
2.3539	16.0	2528	2.2987	1146848
2.3903	17.0	2686	2.2986	1218416
2.2562	18.0	2844	2.2963	1290080
2.3088	19.0	3002	2.2979	1361712
2.4604	20.0	3160	2.2979	1433472

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4