train_piqa_42_1760861208

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1064
• Num Input Tokens Seen: 44231432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1755	1.0	3626	0.1233	2217296
0.2852	2.0	7252	0.1064	4434064
0.0084	3.0	10878	0.1064	6651920
0.017	4.0	14504	0.1143	8863368
0.121	5.0	18130	0.1214	11072168
0.113	6.0	21756	0.1349	13277800
0.0045	7.0	25382	0.1557	15488632
0.0606	8.0	29008	0.1701	17701536
0.0643	9.0	32634	0.1885	19904744
0.0006	10.0	36260	0.2418	22114288
0.0003	11.0	39886	0.2680	24319944
0.0003	12.0	43512	0.3001	26531960
0.0001	13.0	47138	0.3507	28750368
0.0	14.0	50764	0.3914	30958840
0.0	15.0	54390	0.4106	33171360
0.0	16.0	58016	0.4499	35383136
0.0	17.0	61642	0.4668	37593824
0.0	18.0	65268	0.4852	39808768
0.0	19.0	68894	0.4853	42019120
0.0	20.0	72520	0.4872	44231432

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1