train_openbookqa_1754507500

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1619
• Num Input Tokens Seen: 4204168

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1781	0.5	558	0.2757	210048
0.1268	1.0	1116	0.1619	420520
0.1931	1.5	1674	0.2260	630888
0.0602	2.0	2232	0.1949	841024
0.0003	2.5	2790	0.2673	1051168
0.0024	3.0	3348	0.2718	1261304
0.0	3.5	3906	0.4003	1472152
0.0001	4.0	4464	0.3326	1682016
0.0804	4.5	5022	0.2713	1892160
0.0	5.0	5580	0.2744	2102920
0.0	5.5	6138	0.3787	2311976
0.0005	6.0	6696	0.3015	2523672
0.0	6.5	7254	0.3745	2732440
0.0	7.0	7812	0.3472	2943688
0.0	7.5	8370	0.3506	3153640
0.0	8.0	8928	0.3728	3363864
0.0	8.5	9486	0.3764	3574616
0.0	9.0	10044	0.3847	3783840
0.0	9.5	10602	0.3884	3994976
0.0	10.0	11160	0.3919	4204168

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1