train_boolq_1753093536

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3103
• Num Input Tokens Seen: 18160480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.764	0.5	2121	0.5289	910896
0.3504	1.0	4242	0.3571	1820384
0.2991	1.5	6363	0.3375	2725088
0.2253	2.0	8484	0.3315	3635024
0.2148	2.5	10605	0.3272	4551376
0.3091	3.0	12726	0.3289	5454288
0.3015	3.5	14847	0.3238	6365376
0.2697	4.0	16968	0.3227	7267408
0.4527	4.5	19089	0.3166	8180384
0.4352	5.0	21210	0.3146	9086272
0.521	5.5	23331	0.3152	9998416
0.3222	6.0	25452	0.3127	10904352
0.228	6.5	27573	0.3123	11814320
0.2217	7.0	29694	0.3124	12718592
0.2598	7.5	31815	0.3106	13626928
0.2679	8.0	33936	0.3109	14536416
0.2485	8.5	36057	0.3104	15443568
0.2729	9.0	38178	0.3103	16348016
0.3942	9.5	40299	0.3105	17255920
0.2717	10.0	42420	0.3113	18160480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1