train_piqa_789_1767834298

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2312
• Num Input Tokens Seen: 44134648

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2236	1.0	3626	0.2341	2200752
0.2361	2.0	7252	0.2321	4414368
0.2299	3.0	10878	0.2316	6619552
0.2345	4.0	14504	0.2323	8822272
0.2356	5.0	18130	0.2316	11026904
0.2353	6.0	21756	0.2315	13231496
0.2375	7.0	25382	0.2315	15445728
0.232	8.0	29008	0.2317	17655032
0.2314	9.0	32634	0.2313	19867456
0.2345	10.0	36260	0.2315	22063544
0.2282	11.0	39886	0.2313	24265960
0.2329	12.0	43512	0.2314	26475880
0.2324	13.0	47138	0.2314	28688312
0.2287	14.0	50764	0.2313	30896248
0.2319	15.0	54390	0.2314	33103912
0.2366	16.0	58016	0.2314	35300696
0.2287	17.0	61642	0.2313	37507928
0.2319	18.0	65268	0.2314	39713176
0.2313	19.0	68894	0.2314	41919472
0.2345	20.0	72520	0.2312	44134648

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1