train_svamp_123_1760637658

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.1352
• Num Input Tokens Seen: 1433472

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0797	1.0	158	0.1315	71776
0.1211	2.0	316	0.0894	143552
0.0574	3.0	474	0.0797	215312
0.1061	4.0	632	0.0831	286736
0.0631	5.0	790	0.0842	358416
0.0762	6.0	948	0.0838	430224
0.0379	7.0	1106	0.0779	501728
0.0173	8.0	1264	0.0929	573328
0.0799	9.0	1422	0.0985	645088
0.0022	10.0	1580	0.0820	716976
0.0139	11.0	1738	0.1130	788752
0.0348	12.0	1896	0.1042	860352
0.0109	13.0	2054	0.1135	931872
0.005	14.0	2212	0.1268	1003600
0.001	15.0	2370	0.1292	1075312
0.0003	16.0	2528	0.1413	1146848
0.0004	17.0	2686	0.1489	1218416
0.0002	18.0	2844	0.1524	1290080
0.0002	19.0	3002	0.1530	1361712
0.0005	20.0	3160	0.1530	1433472

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4