train_boolq_789_1767722785

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 1.2950
• Num Input Tokens Seen: 42723072

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2684	1.0	2121	0.3589	2132416
0.3582	2.0	4242	0.3311	4255296
0.2544	3.0	6363	0.3360	6393664
0.3032	4.0	8484	0.3310	8532736
0.2762	5.0	10605	0.3300	10659936
0.3371	6.0	12726	0.3300	12794688
0.3554	7.0	14847	0.3328	14935648
0.3358	8.0	16968	0.3261	17068640
0.3756	9.0	19089	0.3230	19208672
0.2488	10.0	21210	0.3243	21357280
0.2666	11.0	23331	0.3207	23492192
0.3283	12.0	25452	0.3208	25634336
0.3128	13.0	27573	0.3246	27764736
0.2072	14.0	29694	0.3297	29902592
0.1784	15.0	31815	0.3346	32041216
0.2937	16.0	33936	0.3359	34178752
0.2507	17.0	36057	0.3443	36309792
0.1936	18.0	38178	0.3469	38446560
0.1981	19.0	40299	0.3518	40588096
0.2253	20.0	42420	0.3555	42723072

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1