train_winogrande_123_1760637729

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 4.1014
• Num Input Tokens Seen: 38394016

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2318	1.0	9090	0.2313	1918144
0.2289	2.0	18180	0.2326	3838192
0.2314	3.0	27270	0.2314	5757648
0.2293	4.0	36360	0.2313	7676976
0.2245	5.0	45450	0.2309	9596496
0.2293	6.0	54540	0.2304	11516256
0.2365	7.0	63630	0.2294	13435600
0.2491	8.0	72720	0.2264	15356752
0.2306	9.0	81810	0.2227	17276752
0.2301	10.0	90900	0.2196	19196064
0.1945	11.0	99990	0.2167	21115472
0.1619	12.0	109080	0.2156	23035440
0.1863	13.0	118170	0.2189	24955600
0.1902	14.0	127260	0.2130	26875344
0.2043	15.0	136350	0.2127	28795600
0.2152	16.0	145440	0.2145	30715008
0.2223	17.0	154530	0.2153	32634912
0.2099	18.0	163620	0.2174	34554080
0.184	19.0	172710	0.2180	36472448
0.1635	20.0	181800	0.2182	38394016

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4