train_stsb_42_1760637581

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 7.6559
• Num Input Tokens Seen: 8733312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6248	1.0	1294	0.6578	435696
0.4618	2.0	2588	0.5365	872288
0.4272	3.0	3882	0.4837	1309872
0.4921	4.0	5176	0.4801	1747344
0.419	5.0	6470	0.4569	2184032
0.4155	6.0	7764	0.4539	2622912
0.7048	7.0	9058	0.4605	3059648
0.4442	8.0	10352	0.4353	3496896
0.3902	9.0	11646	0.4319	3934000
0.3408	10.0	12940	0.4330	4369680
0.5267	11.0	14234	0.4380	4807440
0.3039	12.0	15528	0.4353	5243776
0.3718	13.0	16822	0.4518	5681232
0.3094	14.0	18116	0.4547	6118000
0.2657	15.0	19410	0.4738	6554032
0.2457	16.0	20704	0.4883	6989408
0.3097	17.0	21998	0.5141	7425264
0.2693	18.0	23292	0.5249	7861712
0.2302	19.0	24586	0.5358	8297664
0.2992	20.0	25880	0.5355	8733312

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4