train_openbookqa_123_1760637686

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.5707
• Num Input Tokens Seen: 8496984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2485	1.0	1116	0.5831	424840
0.3823	2.0	2232	0.5799	849600
0.3906	3.0	3348	0.5737	1274392
0.3624	4.0	4464	0.5742	1699872
0.8706	5.0	5580	0.5730	2125480
0.8463	6.0	6696	0.5712	2551008
0.3586	7.0	7812	0.5719	2975440
0.418	8.0	8928	0.5732	3400064
0.4148	9.0	10044	0.5723	3825032
2.5106	10.0	11160	0.5723	4249648
0.2326	11.0	12276	0.5740	4673640
0.911	12.0	13392	0.5723	5097992
0.7709	13.0	14508	0.5741	5522784
0.8735	14.0	15624	0.5768	5948008
1.0873	15.0	16740	0.5707	6372656
0.3543	16.0	17856	0.5748	6797720
0.4845	17.0	18972	0.5738	7222896
0.394	18.0	20088	0.5711	7646872
0.6278	19.0	21204	0.5708	8071488
0.3486	20.0	22320	0.5708	8496984

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4