train_qnli_42_1773148413

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.0500
• Num Input Tokens Seen: 56574368

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0721	0.25	2946	0.0879	2823680
0.0474	0.5	5892	0.0703	5652800
0.0521	0.75	8838	0.0601	8482944
0.011	1.0	11784	0.0566	11312256
0.0548	1.25	14730	0.0539	14142784
0.0895	1.5	17676	0.0536	16969472
0.0372	1.75	20622	0.0530	19782400
0.0373	2.0	23568	0.0562	22629440
0.0137	2.25	26514	0.0528	25460032
0.0393	2.5	29460	0.0542	28284608
0.0368	2.75	32406	0.0520	31130432
0.0543	3.0	35352	0.0500	33947392
0.0425	3.25	38298	0.0530	36783040
0.026	3.5	41244	0.0523	39604544
0.0296	3.75	44190	0.0543	42421440
0.0454	4.0	47136	0.0523	45265344
0.0207	4.25	50082	0.0529	48098944
0.0446	4.5	53028	0.0530	50906176
0.0696	4.75	55974	0.0530	53746240
0.0665	5.0	58920	0.0528	56574368

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4