train_wsc_789_1760356703

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3528
• Num Input Tokens Seen: 487904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5009	0.504	63	0.3697	24608
0.3679	1.008	126	0.5641	49312
0.4566	1.512	189	0.3817	73792
0.4003	2.016	252	0.4326	98496
0.5035	2.52	315	0.3661	124160
0.3547	3.024	378	0.3624	147968
0.4199	3.528	441	0.3688	172320
0.3821	4.032	504	0.3703	197008
0.3555	4.536	567	0.3785	223216
0.4789	5.04	630	0.3675	246288
0.4248	5.5440	693	0.3732	271760
0.3811	6.048	756	0.3844	295040
0.3549	6.552	819	0.3595	320608
0.3491	7.056	882	0.3528	344512
0.499	7.5600	945	0.4010	369600
0.3551	8.064	1008	0.3601	393712
0.3473	8.568	1071	0.3530	418384
0.3591	9.072	1134	0.3538	442912
0.3496	9.576	1197	0.3549	468160

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1