train_qnli_789_1760637973

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.2892
• Num Input Tokens Seen: 183958192

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0748	2.0	41898	0.0853	18391504
0.0287	4.0	83796	0.0510	36788368
0.0043	6.0	125694	0.0567	55184496
0.0468	8.0	167592	0.0687	73580144
0.0003	10.0	209490	0.1025	91981232
0.0	12.0	251388	0.1557	110382704
0.0	14.0	293286	0.2166	128787184
0.0005	16.0	335184	0.2327	147182272
0.0	18.0	377082	0.2755	165571984
0.0	20.0	418980	0.2892	183958192

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4