train_openbookqa_42_1760637571

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2254
• Num Input Tokens Seen: 8500696

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2553	1.0	1116	0.3986	425624
0.1874	2.0	2232	0.3038	851112
0.4131	3.0	3348	0.2763	1276656
0.1017	4.0	4464	0.2629	1701464
0.4336	5.0	5580	0.2531	2126656
0.0814	6.0	6696	0.2469	2551992
0.2496	7.0	7812	0.2390	2977616
0.2507	8.0	8928	0.2361	3402128
0.2781	9.0	10044	0.2342	3826952
0.0919	10.0	11160	0.2290	4252632
0.1037	11.0	12276	0.2277	4677504
0.3048	12.0	13392	0.2282	5103104
0.1069	13.0	14508	0.2276	5527832
0.1087	14.0	15624	0.2254	5952336
0.096	15.0	16740	0.2279	6376760
0.0953	16.0	17856	0.2277	6801440
0.076	17.0	18972	0.2286	7226000
0.1993	18.0	20088	0.2285	7651232
0.1037	19.0	21204	0.2286	8076208
0.133	20.0	22320	0.2291	8500696

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4