train_wsc_123_1760637651

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3482
• Num Input Tokens Seen: 977568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.0256	1.0	125	0.4192	49376
0.3267	2.0	250	0.3620	98240
0.3649	3.0	375	0.3543	147648
0.3266	4.0	500	0.3515	197024
0.3521	5.0	625	0.3493	245472
0.3508	6.0	750	0.3482	293616
0.3087	7.0	875	0.3623	343040
0.3402	8.0	1000	0.3505	392080
0.3238	9.0	1125	0.3518	440848
0.3387	10.0	1250	0.3555	490000
0.3358	11.0	1375	0.3487	538944
0.3564	12.0	1500	0.3546	587536
0.3613	13.0	1625	0.3549	636208
0.3597	14.0	1750	0.3498	685120
0.3479	15.0	1875	0.3515	734352
0.3601	16.0	2000	0.3551	782368
0.3357	17.0	2125	0.3522	831888
0.3428	18.0	2250	0.3536	880112
0.3539	19.0	2375	0.3529	928992
0.334	20.0	2500	0.3523	977568

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4