train_piqa_42_1760844501

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0882
• Num Input Tokens Seen: 44231432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0933	1.0	3626	0.0934	2217296
0.2199	2.0	7252	0.0882	4434064
0.0808	3.0	10878	0.1756	6651920
0.14	4.0	14504	0.2068	8863368
0.0	5.0	18130	0.2433	11072168
0.0001	6.0	21756	0.2589	13277800
0.0	7.0	25382	0.2707	15488632
0.0	8.0	29008	0.2486	17701536
0.0	9.0	32634	0.2677	19904744
0.0	10.0	36260	0.3191	22114288
0.0	11.0	39886	0.3345	24319944
0.0	12.0	43512	0.3868	26531960
0.0	13.0	47138	0.3402	28750368
0.0	14.0	50764	0.4202	30958840
0.0	15.0	54390	0.4130	33171360
0.0	16.0	58016	0.4285	35383136
0.0	17.0	61642	0.3974	37593824
0.0	18.0	65268	0.4219	39808768
0.0	19.0	68894	0.4347	42019120
0.0	20.0	72520	0.4397	44231432

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1