train_boolq_789_1767713899

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3288
• Num Input Tokens Seen: 42723072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2747	1.0	2121	0.3419	2132416
0.3646	2.0	4242	0.3325	4255296
0.2544	3.0	6363	0.3364	6393664
0.3019	4.0	8484	0.3325	8532736
0.298	5.0	10605	0.3318	10659936
0.3511	6.0	12726	0.3318	12794688
0.3646	7.0	14847	0.3338	14935648
0.308	8.0	16968	0.3288	17068640
0.3323	9.0	19089	0.3325	19208672
0.3159	10.0	21210	0.3301	21357280
0.3209	11.0	23331	0.3302	23492192
0.3833	12.0	25452	0.3308	25634336
0.3364	13.0	27573	0.3293	27764736
0.2361	14.0	29694	0.3318	29902592
0.2786	15.0	31815	0.3310	32041216
0.3644	16.0	33936	0.3318	34178752
0.3495	17.0	36057	0.3346	36309792
0.2959	18.0	38178	0.3348	38446560
0.2948	19.0	40299	0.3361	40588096
0.2811	20.0	42420	0.3363	42723072

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1