train_openbookqa_42_1760637566

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 2.3222
• Num Input Tokens Seen: 7559336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7155	2.0	1984	0.6977	756424
0.5348	4.0	3968	0.6991	1513448
0.5806	6.0	5952	0.5779	2269376
0.6205	8.0	7936	0.5596	3025880
0.2731	10.0	9920	0.6090	3782584
0.2472	12.0	11904	0.8592	4538312
0.627	14.0	13888	1.2366	5293704
0.1046	16.0	15872	1.8956	6048648
0.0123	18.0	17856	2.2486	6804272
0.0003	20.0	19840	2.3222	7559336

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4