train_wsc_456_1760637768

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3265
• Num Input Tokens Seen: 970208

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4051	1.0	125	0.3454	48240
0.4078	2.0	250	0.3265	96896
0.3366	3.0	375	0.3293	145184
0.3876	4.0	500	0.3863	194384
0.339	5.0	625	0.3380	242624
0.3411	6.0	750	0.3378	291216
0.3376	7.0	875	0.3363	339568
0.3585	8.0	1000	0.3406	388576
0.3462	9.0	1125	0.3384	436656
0.3438	10.0	1250	0.3589	485152
0.3392	11.0	1375	0.3560	533200
0.2629	12.0	1500	0.4291	581792
0.3291	13.0	1625	0.4854	630384
0.2467	14.0	1750	0.6170	678480
0.2967	15.0	1875	0.9682	727056
0.4055	16.0	2000	1.1292	775168
0.2787	17.0	2125	1.3442	824240
0.1277	18.0	2250	1.4829	872896
0.2009	19.0	2375	1.5894	921296
0.1005	20.0	2500	1.6008	970208

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4