train_wsc_101112_1760637998

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3691
• Num Input Tokens Seen: 980224

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.963	1.0	125	0.6903	48944
0.3999	2.0	250	0.4549	98080
0.4239	3.0	375	0.4099	146624
0.4011	4.0	500	0.3954	196192
0.3613	5.0	625	0.3815	245216
0.3595	6.0	750	0.3821	294128
0.3601	7.0	875	0.3801	342416
0.354	8.0	1000	0.3756	391552
0.3793	9.0	1125	0.3731	440848
0.3715	10.0	1250	0.3757	488816
0.3838	11.0	1375	0.3722	537840
0.338	12.0	1500	0.3709	586624
0.355	13.0	1625	0.3754	635776
0.3246	14.0	1750	0.3691	684416
0.3851	15.0	1875	0.3697	733488
0.3526	16.0	2000	0.3726	782688
0.3682	17.0	2125	0.3707	831792
0.3381	18.0	2250	0.3718	881328
0.3646	19.0	2375	0.3739	930656
0.3393	20.0	2500	0.3714	980224

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4