train_mrpc_1753094152

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mrpc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1376
• Num Input Tokens Seen: 3388032

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1648	0.5	413	0.1758	170336
0.1132	1.0	826	0.1789	339568
0.144	1.5	1239	0.1783	509456
0.0424	2.0	1652	0.1412	678688
0.1393	2.5	2065	0.1389	847104
0.0702	3.0	2478	0.1444	1017368
0.1314	3.5	2891	0.1376	1187704
0.1547	4.0	3304	0.1381	1356744
0.1357	4.5	3717	0.1436	1526088
0.069	5.0	4130	0.1472	1694912
0.2009	5.5	4543	0.1578	1864000
0.043	6.0	4956	0.1472	2033992
0.0217	6.5	5369	0.1563	2203208
0.0205	7.0	5782	0.1504	2372464
0.2827	7.5	6195	0.1562	2540880
0.1001	8.0	6608	0.1588	2710624
0.0458	8.5	7021	0.1609	2880128
0.1856	9.0	7434	0.1609	3049392
0.0522	9.5	7847	0.1604	3218352
0.0796	10.0	8260	0.1604	3388032

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1