train_winogrande_1754652172

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.2314
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2524	0.5	4545	0.2435	1541600
0.2366	1.0	9090	0.2343	3081600
0.2297	1.5	13635	0.2327	4623680
0.223	2.0	18180	0.2345	6165104
0.2372	2.5	22725	0.2324	7706064
0.2331	3.0	27270	0.2317	9248016
0.2266	3.5	31815	0.2336	10789584
0.2318	4.0	36360	0.2314	12330800
0.2337	4.5	40905	0.2322	13871920
0.2306	5.0	45450	0.2322	15413776
0.2369	5.5	49995	0.2319	16954320
0.2286	6.0	54540	0.2316	18496992
0.2328	6.5	59085	0.2314	20039264
0.2328	7.0	63630	0.2319	21579792
0.2335	7.5	68175	0.2314	23122160
0.2347	8.0	72720	0.2314	24664400
0.2325	8.5	77265	0.2316	26207280
0.2316	9.0	81810	0.2314	27747856
0.2314	9.5	86355	0.2314	29287888
0.2336	10.0	90900	0.2315	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1