train_piqa_1754507484

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0865
• Num Input Tokens Seen: 22103448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1198	0.5	1813	0.1015	1118368
0.0609	1.0	3626	0.0865	2216600
0.0172	1.5	5439	0.1115	3320792
0.1459	2.0	7252	0.1028	4419000
0.0007	2.5	9065	0.1501	5525176
0.0752	3.0	10878	0.1502	6628280
0.0	3.5	12691	0.2286	7736376
0.0453	4.0	14504	0.2164	8844408
0.0	4.5	16317	0.2093	9951832
0.0001	5.0	18130	0.1889	11048200
0.0	5.5	19943	0.2587	12157032
0.0	6.0	21756	0.2350	13257624
0.0	6.5	23569	0.3193	14360952
0.0	7.0	25382	0.3669	15468632
0.0	7.5	27195	0.3197	16574840
0.0	8.0	29008	0.3509	17678024
0.0	8.5	30821	0.3696	18780040
0.0	9.0	32634	0.3886	19894712
0.0	9.5	34447	0.3959	21014840
0.0	10.0	36260	0.3994	22103448

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1