train_mrpc_123_1760637679

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.2301
• Num Input Tokens Seen: 6774288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1553	1.0	826	0.2400	339568
0.1436	2.0	1652	0.2378	678688
0.1718	3.0	2478	0.2332	1017368
0.2108	4.0	3304	0.2317	1356744
0.2006	5.0	4130	0.2325	1694912
0.2218	6.0	4956	0.2318	2033992
0.2116	7.0	5782	0.2315	2372464
0.1124	8.0	6608	0.2301	2710624
0.2539	9.0	7434	0.2319	3049392
0.203	10.0	8260	0.2304	3388032
0.2767	11.0	9086	0.2302	3727312
0.2047	12.0	9912	0.2312	4065504
0.3393	13.0	10738	0.2303	4404624
0.2593	14.0	11564	0.2308	4743080
0.1668	15.0	12390	0.2313	5082240
0.2622	16.0	13216	0.2308	5420840
0.2252	17.0	14042	0.2308	5759384
0.2081	18.0	14868	0.2317	6097784
0.304	19.0	15694	0.2310	6435544
0.3309	20.0	16520	0.2313	6774288

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4