train_stsb_456_1760637809

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.6750
• Num Input Tokens Seen: 7768912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6763	2.0	2300	0.7221	775040
0.64	4.0	4600	0.6318	1550584
0.527	6.0	6900	0.6111	2327584
0.5278	8.0	9200	0.5686	3105920
0.4213	10.0	11500	0.5354	3882840
0.3775	12.0	13800	0.5529	4661160
0.3438	14.0	16100	0.5783	5437984
0.2458	16.0	18400	0.6256	6215584
0.2988	18.0	20700	0.6620	6991000
0.3062	20.0	23000	0.6750	7768912

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4