train_sst2_1754652139

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2435
• Num Input Tokens Seen: 33869824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4139	0.5	7577	0.3520	1694048
0.3916	1.0	15154	0.3510	3385616
0.3683	1.5	22731	0.3444	5082864
0.352	2.0	30308	0.3483	6774096
0.3562	2.5	37885	0.3420	8467152
0.3185	3.0	45462	0.3288	10161824
0.3305	3.5	53039	0.3129	11856000
0.227	4.0	60616	0.2920	13549104
0.3576	4.5	68193	0.2824	15241168
0.1752	5.0	75770	0.2703	16935568
0.2731	5.5	83347	0.2657	18626160
0.3093	6.0	90924	0.2584	20320896
0.1685	6.5	98501	0.2534	22013696
0.3451	7.0	106078	0.2629	23709008
0.1974	7.5	113655	0.2515	25400400
0.1451	8.0	121232	0.2452	27099520
0.1085	8.5	128809	0.2437	28792480
0.3203	9.0	136386	0.2442	30484864
0.2115	9.5	143963	0.2450	32173664
0.078	10.0	151540	0.2435	33869824

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1