train_stsb_1753094148

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.4726
• Num Input Tokens Seen: 4364240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4873	0.5	647	0.5906	217472
0.3791	1.0	1294	0.5400	435488
0.4086	1.5	1941	0.4958	652480
0.3812	2.0	2588	0.4796	871200
0.3718	2.5	3235	0.4809	1089120
0.4055	3.0	3882	0.4726	1307968
0.4955	3.5	4529	0.4881	1529024
0.2927	4.0	5176	0.5424	1745568
0.3406	4.5	5823	0.5415	1965984
0.4391	5.0	6470	0.5299	2182352
0.2673	5.5	7117	0.6492	2399760
0.2281	6.0	7764	0.6053	2619888
0.2492	6.5	8411	0.7269	2837808
0.2122	7.0	9058	0.7272	3057216
0.1311	7.5	9705	0.8538	3275904
0.0777	8.0	10352	0.9175	3493600
0.1843	8.5	10999	1.0837	3712320
0.1225	9.0	11646	1.1024	3928704
0.1639	9.5	12293	1.1604	4147200
0.1883	10.0	12940	1.1662	4364240

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1