train_qnli_1752755841

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.1534
• Num Input Tokens Seen: 199744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
9.948	0.5217	12	6.3386	10688
4.2188	1.0435	24	1.9789	20672
0.4292	1.5652	36	0.5822	31488
0.2289	2.0870	48	0.1556	41984
0.2864	2.6087	60	0.1566	52544
0.1964	3.1304	72	0.1736	62272
0.1653	3.6522	84	0.2433	73088
1.0462	4.1739	96	0.4191	83104
0.3356	4.6957	108	0.1534	93792
0.1437	5.2174	120	0.3287	103936
0.1812	5.7391	132	0.2281	114752
0.1592	6.2609	144	0.2266	125344
0.1603	6.7826	156	0.2513	135648
0.155	7.3043	168	0.2568	145856
0.1559	7.8261	180	0.1678	156608
0.1908	8.3478	192	0.1675	166720
0.155	8.8696	204	0.1932	177536
0.2123	9.3913	216	0.1931	187744
0.1304	9.9130	228	0.1905	198304

Framework versions

• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1