train_stsb_123_1760637696

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 2.3309
• Num Input Tokens Seen: 8725024

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5796	1.0	1294	0.7036	435488
0.405	2.0	2588	0.6050	871200
0.4826	3.0	3882	0.5194	1307968
0.355	4.0	5176	0.5213	1745568
0.6005	5.0	6470	0.4893	2182352
0.5369	6.0	7764	0.4895	2619888
0.3825	7.0	9058	0.4864	3057216
0.3519	8.0	10352	0.4998	3493600
0.3994	9.0	11646	0.4883	3928704
0.4038	10.0	12940	0.4858	4364240
0.4779	11.0	14234	0.4757	4800144
0.3692	12.0	15528	0.4697	5234320
0.3688	13.0	16822	0.4744	5670720
0.404	14.0	18116	0.4865	6108240
0.2171	15.0	19410	0.4986	6543632
0.3107	16.0	20704	0.5108	6978816
0.2954	17.0	21998	0.5228	7415824
0.3044	18.0	23292	0.5332	7851088
0.255	19.0	24586	0.5385	8287536
0.2648	20.0	25880	0.5399	8725024

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4