train_openbookqa_42_1760637569

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1736
• Num Input Tokens Seen: 8500696

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2864	1.0	1116	0.1760	425624
0.0324	2.0	2232	0.1736	851112
0.0707	3.0	3348	0.2102	1276656
0.0002	4.0	4464	0.3024	1701464
0.0944	5.0	5580	0.3936	2126656
0.0	6.0	6696	0.5339	2551992
0.0	7.0	7812	0.5140	2977616
0.0	8.0	8928	0.4917	3402128
0.0	9.0	10044	0.5389	3826952
0.0	10.0	11160	0.4841	4252632
0.0	11.0	12276	0.5025	4677504
0.0	12.0	13392	0.5820	5103104
0.0	13.0	14508	0.5510	5527832
0.0	14.0	15624	0.5208	5952336
0.0	15.0	16740	0.5739	6376760
0.0	16.0	17856	0.5887	6801440
0.0	17.0	18972	0.6108	7226000
0.0	18.0	20088	0.6208	7651232
0.0	19.0	21204	0.6247	8076208
0.0	20.0	22320	0.6253	8500696

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4