train_wsc_456_1760347113

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3279
• Num Input Tokens Seen: 485152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4437	0.504	63	0.3450	24704
0.3371	1.008	126	0.4380	48688
0.3562	1.512	189	0.3320	73456
0.3556	2.016	252	0.3279	97568
0.3794	2.52	315	0.3905	121888
0.3405	3.024	378	0.3348	146336
0.3531	3.528	441	0.3317	172480
0.3697	4.032	504	0.3633	196240
0.3461	4.536	567	0.3538	221136
0.3315	5.04	630	0.3363	244736
0.3456	5.5440	693	0.3388	268480
0.3495	6.048	756	0.3458	293424
0.3493	6.552	819	0.3447	317840
0.3513	7.056	882	0.3410	342384
0.3502	7.5600	945	0.3402	366288
0.339	8.064	1008	0.3421	391840
0.3422	8.568	1071	0.3410	416320
0.3513	9.072	1134	0.3432	440048
0.3577	9.576	1197	0.3439	464688

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1