train_wsc_456_1760637767

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3492
• Num Input Tokens Seen: 970208

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3801	1.0	125	0.7905	48240
0.3592	2.0	250	0.3415	96896
0.3334	3.0	375	0.3361	145184
0.3787	4.0	500	0.3648	194384
0.3524	5.0	625	0.3401	242624
0.3464	6.0	750	0.3441	291216
0.3474	7.0	875	0.3409	339568
0.3544	8.0	1000	0.3401	388576
0.351	9.0	1125	0.3529	436656
0.3541	10.0	1250	0.3509	485152
0.3524	11.0	1375	0.3458	533200
0.3496	12.0	1500	0.3470	581792
0.3444	13.0	1625	0.3509	630384
0.3373	14.0	1750	0.3423	678480
0.3509	15.0	1875	0.3484	727056
0.3573	16.0	2000	0.3462	775168
0.3451	17.0	2125	0.3479	824240
0.3433	18.0	2250	0.3458	872896
0.3485	19.0	2375	0.3455	921296
0.342	20.0	2500	0.3448	970208

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4