train_mrpc_101112_1760638018

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.0728
• Num Input Tokens Seen: 6767120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1946	1.0	826	0.1936	339144
0.1581	2.0	1652	0.2190	677000
0.1508	3.0	2478	0.1798	1015816
0.1825	4.0	3304	0.1686	1354072
0.1276	5.0	4130	0.1500	1692608
0.1212	6.0	4956	0.1180	2030416
0.1017	7.0	5782	0.1093	2369136
0.0644	8.0	6608	0.0782	2707920
0.0664	9.0	7434	0.0729	3046392
0.0447	10.0	8260	0.0728	3383592
0.0689	11.0	9086	0.0771	3722064
0.0335	12.0	9912	0.0910	4060512
0.0585	13.0	10738	0.1085	4398224
0.0011	14.0	11564	0.1211	4737184
0.0007	15.0	12390	0.1278	5075136
0.0009	16.0	13216	0.1258	5413384
0.0024	17.0	14042	0.1272	5751600
0.0049	18.0	14868	0.1256	6090384
0.0011	19.0	15694	0.1269	6429416
0.0005	20.0	16520	0.1259	6767120

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4