train_mrpc_1756729600

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.4432
• Num Input Tokens Seen: 3186272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1943	0.5003	826	0.1882	159824
0.1788	1.0006	1652	0.1734	318992
0.1887	1.5009	2478	0.1938	478672
0.0994	2.0012	3304	0.1400	637952
0.288	2.5015	4130	0.1896	795872
0.0249	3.0018	4956	0.1457	956728
0.1076	3.5021	5782	0.1425	1116152
0.3561	4.0024	6608	0.1409	1275352
0.0437	4.5027	7434	0.1326	1434648
0.202	5.0030	8260	0.1252	1593600
0.1119	5.5033	9086	0.1426	1752592
0.1407	6.0036	9912	0.1635	1912280
0.0006	6.5039	10738	0.2185	2072056
0.124	7.0042	11564	0.2175	2231152
0.0001	7.5045	12390	0.3413	2390560
0.0001	8.0048	13216	0.3109	2550320
0.0	8.5051	14042	0.4262	2709536
0.0	9.0055	14868	0.4333	2869056
0.0	9.5058	15694	0.4443	3028464

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1