train_winogrande_42_1760637613

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 9.0314
• Num Input Tokens Seen: 38397712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2316	1.0	9090	0.2342	1918960
0.2206	2.0	18180	0.2526	3839712
0.2561	3.0	27270	0.2372	5759216
0.2161	4.0	36360	0.2406	7678944
0.2338	5.0	45450	0.2366	9598112
0.233	6.0	54540	0.2320	11518608
0.2265	7.0	63630	0.2352	13438816
0.2307	8.0	72720	0.2338	15359200
0.2298	9.0	81810	0.2323	17280320
0.2307	10.0	90900	0.2318	19200384
0.2329	11.0	99990	0.2317	21120032
0.2288	12.0	109080	0.2313	23039856
0.2295	13.0	118170	0.2316	24959536
0.2371	14.0	127260	0.2315	26879696
0.2282	15.0	136350	0.2325	28798160
0.2296	16.0	145440	0.2315	30718896
0.2362	17.0	154530	0.2315	32638160
0.2265	18.0	163620	0.2315	34558000
0.2327	19.0	172710	0.2316	36477680
0.2316	20.0	181800	0.2315	38397712

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4