train_boolq_789_1767707803

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 1.0177
• Num Input Tokens Seen: 38037912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3073	2.0	3772	0.3834	3795792
0.3476	4.0	7544	0.3342	7605072
0.3299	6.0	11316	0.3315	11401504
0.2648	8.0	15088	0.3914	15210032
0.0804	10.0	18860	0.4778	19014376
0.0194	12.0	22632	0.5860	22825200
0.1961	14.0	26404	0.7469	26630224
0.0047	16.0	30176	0.9148	30435912
0.0005	18.0	33948	1.0029	34236192
0.0002	20.0	37720	1.0177	38037912

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1