train_winogrande_42_1760637612

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.2311
• Num Input Tokens Seen: 38397712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.232	1.0	9090	0.2313	1918960
0.2346	2.0	18180	0.2313	3839712
0.2314	3.0	27270	0.2316	5759216
0.2314	4.0	36360	0.2317	7678944
0.2304	5.0	45450	0.2316	9598112
0.2308	6.0	54540	0.2312	11518608
0.233	7.0	63630	0.2311	13438816
0.2299	8.0	72720	0.2312	15359200
0.2314	9.0	81810	0.2312	17280320
0.2308	10.0	90900	0.2313	19200384
0.2352	11.0	99990	0.2321	21120032
0.2314	12.0	109080	0.2314	23039856
0.2324	13.0	118170	0.2313	24959536
0.2314	14.0	127260	0.2314	26879696
0.2324	15.0	136350	0.2313	28798160
0.2298	16.0	145440	0.2315	30718896
0.2303	17.0	154530	0.2313	32638160
0.2329	18.0	163620	0.2315	34558000
0.2314	19.0	172710	0.2314	36477680
0.2288	20.0	181800	0.2313	38397712

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4