train_piqa_1755694503

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2351
• Num Input Tokens Seen: 18129912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2319	0.5001	3626	0.2322	911792
0.2339	1.0001	7252	0.2315	1811112
0.2378	1.5002	10878	0.2313	2718104
0.2198	2.0003	14504	0.2325	3622376
0.2318	2.5003	18130	0.2345	4531480
0.2319	3.0004	21756	0.2325	5438312
0.2404	3.5005	25382	0.2320	6349480
0.2334	4.0006	29008	0.2319	7254040
0.2337	4.5006	32634	0.2319	8163208
0.2292	5.0007	36260	0.2316	9067320
0.2338	5.5008	39886	0.2320	9976536
0.2317	6.0008	43512	0.2324	10880152
0.2424	6.5009	47138	0.2331	11784504
0.2293	7.0010	50764	0.2330	12692344
0.2427	7.5010	54390	0.2327	13598136
0.2253	8.0011	58016	0.2332	14505048
0.2478	8.5012	61642	0.2346	15408488
0.2234	9.0012	65268	0.2341	16318072
0.2275	9.5013	68894	0.2354	17232632

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1