train_piqa_1754507483

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1045
• Num Input Tokens Seen: 22103448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3099	0.5	1813	0.1600	1118368
0.1034	1.0	3626	0.1211	2216600
0.1183	1.5	5439	0.1136	3320792
0.064	2.0	7252	0.1102	4419000
0.1255	2.5	9065	0.1081	5525176
0.0724	3.0	10878	0.1070	6628280
0.0725	3.5	12691	0.1093	7736376
0.14	4.0	14504	0.1053	8844408
0.0916	4.5	16317	0.1046	9951832
0.1239	5.0	18130	0.1055	11048200
0.0888	5.5	19943	0.1049	12157032
0.0845	6.0	21756	0.1052	13257624
0.0719	6.5	23569	0.1046	14360952
0.0986	7.0	25382	0.1052	15468632
0.0909	7.5	27195	0.1045	16574840
0.0711	8.0	29008	0.1050	17678024
0.1367	8.5	30821	0.1047	18780040
0.111	9.0	32634	0.1049	19894712
0.0628	9.5	34447	0.1049	21014840
0.0366	10.0	36260	0.1047	22103448

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1