train_wsc_101112_1760637992

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3991
• Num Input Tokens Seen: 873896

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.379	2.0	222	0.3636	87928
0.3914	4.0	444	0.3694	175872
0.3438	6.0	666	0.3487	263400
0.4019	8.0	888	0.3586	350384
0.3813	10.0	1110	0.3519	438024
0.3562	12.0	1332	0.3595	524920
0.3181	14.0	1554	0.3852	612192
0.3071	16.0	1776	0.3937	699536
0.349	18.0	1998	0.4000	786984
0.3458	20.0	2220	0.3991	873896

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4