train_svamp_42_1760637544

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the svamp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0492
• Num Input Tokens Seen: 1433520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0743	1.0	158	0.0966	71568
0.0313	2.0	316	0.0611	143232
0.0246	3.0	474	0.0492	214912
0.0696	4.0	632	0.0504	286448
0.0025	5.0	790	0.0699	358176
0.0283	6.0	948	0.0964	429728
0.0001	7.0	1106	0.0980	501504
0.018	8.0	1264	0.1090	573120
0.0002	9.0	1422	0.1303	644944
0.0001	10.0	1580	0.1172	716448
0.0	11.0	1738	0.1227	788256
0.0	12.0	1896	0.1193	859808
0.0	13.0	2054	0.1224	931472
0.0	14.0	2212	0.1244	1003376
0.0	15.0	2370	0.1243	1075088
0.0	16.0	2528	0.1255	1146608
0.0	17.0	2686	0.1254	1218368
0.0	18.0	2844	0.1281	1290144
0.0	19.0	3002	0.1265	1361984
0.0	20.0	3160	0.1270	1433520

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4