train_boolq_1754587244

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1978
• Num Input Tokens Seen: 21342336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8159	0.5002	1061	0.2517	1069568
0.3046	1.0005	2122	0.2182	2133248
0.2353	1.5007	3183	0.2334	3194016
0.2127	2.0009	4244	0.2192	4266592
0.1608	2.5012	5305	0.2054	5343200
0.1139	3.0014	6366	0.1978	6407840
0.3431	3.5017	7427	0.2064	7476544
0.192	4.0019	8488	0.2092	8540672
0.1091	4.5021	9549	0.2167	9613088
0.0205	5.0024	10610	0.2308	10682528
0.2371	5.5026	11671	0.2199	11757792
0.1841	6.0028	12732	0.2255	12820704
0.3061	6.5031	13793	0.2306	13892416
0.4188	7.0033	14854	0.2270	14957120
0.4564	7.5035	15915	0.2281	16020704
0.2621	8.0038	16976	0.2298	17090912
0.5396	8.5040	18037	0.2308	18157184
0.1557	9.0042	19098	0.2296	19222688
0.3085	9.5045	20159	0.2298	20290176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1