train_sst2_42_1760637619

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0529
• Num Input Tokens Seen: 67768656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0821	1.0	15154	0.0582	3383904
0.0287	2.0	30308	0.0607	6774480
0.0186	3.0	45462	0.0540	10163152
0.1246	4.0	60616	0.0548	13552544
0.1157	5.0	75770	0.0559	16941120
0.0868	6.0	90924	0.0529	20331520
0.0091	7.0	106078	0.0571	23721840
0.0196	8.0	121232	0.0531	27111168
0.1583	9.0	136386	0.0556	30498912
0.0875	10.0	151540	0.0558	33886240
0.0289	11.0	166694	0.0551	37277536
0.0117	12.0	181848	0.0564	40664576
0.0429	13.0	197002	0.0587	44053792
0.0602	14.0	212156	0.0566	47441984
0.0387	15.0	227310	0.0597	50830480
0.007	16.0	242464	0.0581	54215696
0.0078	17.0	257618	0.0583	57603296
0.014	18.0	272772	0.0583	60987568
0.0139	19.0	287926	0.0583	64378192
0.0131	20.0	303080	0.0583	67768656

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4