train_wsc_1754652156

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 2.7778
• Num Input Tokens Seen: 490000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
2.4086	0.504	63	0.3988	25504
0.5346	1.008	126	0.5207	49696
0.3725	1.512	189	0.4400	74112
0.343	2.016	252	0.3816	99136
0.3615	2.52	315	0.3501	123904
0.5066	3.024	378	0.3483	148736
0.3459	3.528	441	0.3604	174432
0.3261	4.032	504	0.3559	198656
0.3529	4.536	567	0.3538	224032
0.3441	5.04	630	0.3520	247424
0.3572	5.5440	693	0.3479	271232
0.3479	6.048	756	0.3478	295728
0.3533	6.552	819	0.3526	320464
0.3186	7.056	882	0.3628	345856
0.3306	7.5600	945	0.3622	371040
0.3524	8.064	1008	0.3506	395216
0.3447	8.568	1071	0.3489	419184
0.3639	9.072	1134	0.3548	444560
0.3419	9.576	1197	0.3464	469104

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1