train_winogrande_123_1760637727

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.2497
• Num Input Tokens Seen: 34137248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2302	2.0	16160	0.2323	3413104
0.1752	4.0	32320	0.2234	6828224
0.1669	6.0	48480	0.1961	10241120
0.2531	8.0	64640	0.0697	13653680
0.0551	10.0	80800	0.0728	17067040
0.0946	12.0	96960	0.0891	20481008
0.0	14.0	113120	0.1453	23896656
0.0	16.0	129280	0.2087	27309216
0.0	18.0	145440	0.2456	30722384
0.0	20.0	161600	0.2497	34137248

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4