train_openbookqa_789_1760637911

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the openbookqa dataset. It achieves the following results on the evaluation set:

• Loss: 1.9605
• Num Input Tokens Seen: 7554272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6864	2.0	1984	0.6934	755384
0.7524	4.0	3968	0.6763	1509968
0.5909	6.0	5952	0.6351	2265440
0.5798	8.0	7936	0.6822	3022176
0.5903	10.0	9920	0.6504	3777856
0.1202	12.0	11904	0.8376	4532416
0.1914	14.0	13888	1.0906	5289080
0.1217	16.0	15872	1.6510	6044432
0.0012	18.0	17856	1.9140	6799432
0.076	20.0	19840	1.9605	7554272

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4