train_boolq_789_1767807361

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1909
• Num Input Tokens Seen: 42723072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0605	1.0	2121	0.2567	2132416
0.161	2.0	4242	0.2044	4255296
0.0096	3.0	6363	0.1974	6393664
0.0198	4.0	8484	0.1933	8532736
0.1818	5.0	10605	0.1909	10659936
0.0724	6.0	12726	0.2224	12794688
0.1391	7.0	14847	0.2592	14935648
0.0278	8.0	16968	0.2785	17068640
0.003	9.0	19089	0.3106	19208672
0.1081	10.0	21210	0.3240	21357280
0.0016	11.0	23331	0.3694	23492192
0.1869	12.0	25452	0.3855	25634336
0.0008	13.0	27573	0.4114	27764736
0.0008	14.0	29694	0.4370	29902592
0.0004	15.0	31815	0.4963	32041216
0.0007	16.0	33936	0.4973	34178752
0.0833	17.0	36057	0.5294	36309792
0.001	18.0	38178	0.5357	38446560
0.0002	19.0	40299	0.5411	40588096
0.0008	20.0	42420	0.5343	42723072

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1