train_copa_789_1760637874

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2340
• Num Input Tokens Seen: 563744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.245	1.0	90	0.2327	28192
0.2443	2.0	180	0.2295	56192
0.2401	3.0	270	0.2313	84192
0.2337	4.0	360	0.2311	112544
0.2353	5.0	450	0.2336	140960
0.2335	6.0	540	0.2307	169216
0.2306	7.0	630	0.2328	197248
0.2366	8.0	720	0.2327	225440
0.2396	9.0	810	0.2341	253632
0.228	10.0	900	0.2329	281984
0.2447	11.0	990	0.2357	310176
0.229	12.0	1080	0.2378	338400
0.2283	13.0	1170	0.2320	366688
0.2299	14.0	1260	0.2350	394752
0.2301	15.0	1350	0.2347	422912
0.2232	16.0	1440	0.2348	451008
0.2318	17.0	1530	0.2435	479104
0.2203	18.0	1620	0.2461	507392
0.2287	19.0	1710	0.2632	535584
0.1829	20.0	1800	0.2552	563744

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4