train_piqa_789_1767887387

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1069
• Num Input Tokens Seen: 44134648

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1486	1.0	3626	0.1394	2200752
0.0613	2.0	7252	0.1197	4414368
0.1116	3.0	10878	0.1118	6619552
0.108	4.0	14504	0.1083	8822272
0.1196	5.0	18130	0.1073	11026904
0.0885	6.0	21756	0.1071	13231496
0.1056	7.0	25382	0.1084	15445728
0.1449	8.0	29008	0.1069	17655032
0.1288	9.0	32634	0.1102	19867456
0.0222	10.0	36260	0.1099	22063544
0.0338	11.0	39886	0.1099	24265960
0.0283	12.0	43512	0.1102	26475880
0.0425	13.0	47138	0.1141	28688312
0.1532	14.0	50764	0.1149	30896248
0.0248	15.0	54390	0.1158	33103912
0.0319	16.0	58016	0.1162	35300696
0.0222	17.0	61642	0.1170	37507928
0.074	18.0	65268	0.1175	39713176
0.05	19.0	68894	0.1175	41919472
0.0823	20.0	72520	0.1178	44134648

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1