train_sst2_42_1760637623

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0584
• Num Input Tokens Seen: 67768656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1373	1.0	15154	0.0828	3383904
0.0428	2.0	30308	0.0698	6774480
0.0208	3.0	45462	0.0647	10163152
0.1631	4.0	60616	0.0627	13552544
0.1034	5.0	75770	0.0609	16941120
0.0733	6.0	90924	0.0596	20331520
0.0071	7.0	106078	0.0584	23721840
0.0768	8.0	121232	0.0588	27111168
0.1709	9.0	136386	0.0594	30498912
0.1147	10.0	151540	0.0600	33886240
0.0381	11.0	166694	0.0620	37277536
0.0218	12.0	181848	0.0597	40664576
0.1016	13.0	197002	0.0615	44053792
0.0983	14.0	212156	0.0599	47441984
0.0535	15.0	227310	0.0617	50830480
0.0101	16.0	242464	0.0611	54215696
0.0054	17.0	257618	0.0617	57603296
0.025	18.0	272772	0.0621	60987568
0.0577	19.0	287926	0.0619	64378192
0.0253	20.0	303080	0.0620	67768656

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4