train_openbookqa_1754507499

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2537
• Num Input Tokens Seen: 4204168

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1575	0.5	558	0.5180	210048
0.1755	1.0	1116	0.3476	420520
0.2585	1.5	1674	0.3197	630888
0.2442	2.0	2232	0.3002	841024
0.2445	2.5	2790	0.2924	1051168
0.2615	3.0	3348	0.2820	1261304
0.077	3.5	3906	0.2786	1472152
0.1693	4.0	4464	0.2682	1682016
0.2201	4.5	5022	0.2648	1892160
0.3082	5.0	5580	0.2638	2102920
0.1839	5.5	6138	0.2633	2311976
0.4629	6.0	6696	0.2598	2523672
0.2638	6.5	7254	0.2586	2732440
0.0373	7.0	7812	0.2553	2943688
0.3764	7.5	8370	0.2552	3153640
0.2395	8.0	8928	0.2559	3363864
0.3905	8.5	9486	0.2537	3574616
0.1212	9.0	10044	0.2545	3783840
0.0915	9.5	10602	0.2549	3994976
0.9298	10.0	11160	0.2543	4204168

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1