train_stsb_1754502818

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.5281
• Num Input Tokens Seen: 4364240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6971	0.5	647	0.8380	217472
0.6463	1.0	1294	0.7214	435488
0.5077	1.5	1941	0.6835	652480
0.4527	2.0	2588	0.6478	871200
0.4397	2.5	3235	0.5840	1089120
0.5232	3.0	3882	0.5805	1307968
0.4755	3.5	4529	0.5764	1529024
0.3385	4.0	5176	0.5770	1745568
0.4848	4.5	5823	0.5474	1965984
0.673	5.0	6470	0.5423	2182352
0.3992	5.5	7117	0.5356	2399760
0.561	6.0	7764	0.5338	2619888
0.542	6.5	8411	0.5333	2837808
0.4066	7.0	9058	0.5314	3057216
0.4068	7.5	9705	0.5295	3275904
0.4087	8.0	10352	0.5286	3493600
0.4841	8.5	10999	0.5286	3712320
0.4318	9.0	11646	0.5287	3928704
0.4052	9.5	12293	0.5282	4147200
0.4315	10.0	12940	0.5281	4364240

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1