train_piqa_456_1765416484

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2372
• Num Input Tokens Seen: 44177928

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2329	1.0	3626	0.2308	2208216
0.2323	2.0	7252	0.2315	4420664
0.2325	3.0	10878	0.2314	6629696
0.2256	4.0	14504	0.2318	8840800
0.2694	5.0	18130	0.2368	11045752
0.2314	6.0	21756	0.2312	13254840
0.2382	7.0	25382	0.2311	15458512
0.2334	8.0	29008	0.2316	17666816
0.2298	9.0	32634	0.2312	19878664
0.2318	10.0	36260	0.2313	22082280
0.2236	11.0	39886	0.2312	24300584
0.2244	12.0	43512	0.2309	26515920
0.2226	13.0	47138	0.2317	28721912
0.2244	14.0	50764	0.2323	30927016
0.2297	15.0	54390	0.2321	33135160
0.2198	16.0	58016	0.2316	35347688
0.2357	17.0	61642	0.2317	37560560
0.2296	18.0	65268	0.2323	39771536
0.2169	19.0	68894	0.2325	41974792
0.2213	20.0	72520	0.2319	44177928

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1