train_mrpc_42_1760637565

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1204
• Num Input Tokens Seen: 6769320

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1585	1.0	826	0.1989	337344
0.186	2.0	1652	0.1650	675368
0.1018	3.0	2478	0.1571	1014008
0.2093	4.0	3304	0.1725	1353224
0.1468	5.0	4130	0.1322	1692320
0.0819	6.0	4956	0.1338	2030488
0.0564	7.0	5782	0.1339	2368032
0.2202	8.0	6608	0.1236	2706552
0.0741	9.0	7434	0.1210	3044984
0.1178	10.0	8260	0.1241	3384288
0.0656	11.0	9086	0.1273	3722552
0.0982	12.0	9912	0.1204	4061216
0.0647	13.0	10738	0.1278	4399064
0.0968	14.0	11564	0.1204	4737648
0.1723	15.0	12390	0.1262	5075144
0.0816	16.0	13216	0.1204	5414032
0.1208	17.0	14042	0.1249	5752528
0.1787	18.0	14868	0.1257	6091472
0.0937	19.0	15694	0.1245	6430144
0.0821	20.0	16520	0.1246	6769320

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4