train_boolq_789_1767795626

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1832
• Num Input Tokens Seen: 42723072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0033	1.0	2121	0.2513	2132416
0.2354	2.0	4242	0.1832	4255296
0.0044	3.0	6363	0.2300	6393664
0.007	4.0	8484	0.1940	8532736
0.002	5.0	10605	0.3950	10659936
0.0009	6.0	12726	0.3612	12794688
0.0	7.0	14847	0.4341	14935648
0.0	8.0	16968	0.5985	17068640
0.3282	9.0	19089	0.3978	19208672
0.0	10.0	21210	0.3888	21357280
0.0001	11.0	23331	0.5911	23492192
0.0145	12.0	25452	0.5184	25634336
0.0	13.0	27573	0.6214	27764736
0.0	14.0	29694	0.6458	29902592
0.0	15.0	31815	0.6960	32041216
0.0	16.0	33936	0.7695	34178752
0.0	17.0	36057	0.7907	36309792
0.0	18.0	38178	0.8310	38446560
0.0	19.0	40299	0.8462	40588096
0.0	20.0	42420	0.8515	42723072

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1