train_piqa_456_1765470774

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1050
• Num Input Tokens Seen: 44177928

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2199	1.0	3626	0.1220	2208216
0.0212	2.0	7252	0.1101	4420664
0.0436	3.0	10878	0.1050	6629696
0.031	4.0	14504	0.1099	8840800
0.0961	5.0	18130	0.1149	11045752
0.0173	6.0	21756	0.1326	13254840
0.0605	7.0	25382	0.1522	15458512
0.1066	8.0	29008	0.1928	17666816
0.0781	9.0	32634	0.2091	19878664
0.0136	10.0	36260	0.2598	22082280
0.0001	11.0	39886	0.2980	24300584
0.0001	12.0	43512	0.3218	26515920
0.0	13.0	47138	0.3798	28721912
0.0	14.0	50764	0.3934	30927016
0.0	15.0	54390	0.4581	33135160
0.0	16.0	58016	0.4922	35347688
0.0	17.0	61642	0.5229	37560560
0.0	18.0	65268	0.5355	39771536
0.0	19.0	68894	0.5416	41974792
0.0	20.0	72520	0.5437	44177928

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1