train_qnli_42_1760637633

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.0397
• Num Input Tokens Seen: 207226464

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1008	1.0	23567	0.0450	10362048
0.0489	2.0	47134	0.0397	20723232
0.0179	3.0	70701	0.0435	31087712
0.0002	4.0	94268	0.0629	41448512
0.0397	5.0	117835	0.0667	51808576
0.0	6.0	141402	0.0826	62164384
0.1	7.0	164969	0.1081	72528320
0.0005	8.0	188536	0.1119	82892896
0.0	9.0	212103	0.1115	93260448
0.0001	10.0	235670	0.1124	103622208
0.0	11.0	259237	0.0998	113983328
0.0	12.0	282804	0.1150	124345088
0.0	13.0	306371	0.1465	134702688
0.0	14.0	329938	0.1426	145065792
0.0	15.0	353505	0.1291	155429344
0.0	16.0	377072	0.1774	165793952
0.0	17.0	400639	0.1959	176154208
0.0	18.0	424206	0.2133	186506176
0.0	19.0	447773	0.2188	196865248
0.0	20.0	471340	0.2194	207226464

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4