train_wsc_789_1760347387

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3421
• Num Input Tokens Seen: 487904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
5.2854	0.504	63	4.6218	24608
1.0178	1.008	126	0.6724	49312
0.3533	1.512	189	0.3516	73792
0.3261	2.016	252	0.3760	98496
0.341	2.52	315	0.3426	124160
0.3579	3.024	378	0.3562	147968
0.3465	3.528	441	0.3441	172320
0.3559	4.032	504	0.3521	197008
0.3449	4.536	567	0.3457	223216
0.3509	5.04	630	0.3483	246288
0.3486	5.5440	693	0.3460	271760
0.3426	6.048	756	0.3445	295040
0.3439	6.552	819	0.3421	320608
0.3508	7.056	882	0.3492	344512
0.3529	7.5600	945	0.3454	369600
0.3535	8.064	1008	0.3459	393712
0.3439	8.568	1071	0.3495	418384
0.358	9.072	1134	0.3453	442912
0.3452	9.576	1197	0.3460	468160

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1