train_copa_42_1760637530

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2680
• Num Input Tokens Seen: 564096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2402	1.0	90	0.2291	28256
0.3408	2.0	180	0.2556	56480
0.2317	3.0	270	0.2304	84736
0.2337	4.0	360	0.2300	113024
0.2289	5.0	450	0.2351	141440
0.2351	6.0	540	0.2339	169600
0.234	7.0	630	0.2294	197792
0.2259	8.0	720	0.2292	225984
0.2316	9.0	810	0.2337	254112
0.2333	10.0	900	0.2569	282368
0.2305	11.0	990	0.2349	310560
0.2303	12.0	1080	0.2336	338784
0.2296	13.0	1170	0.2321	366944
0.2277	14.0	1260	0.2375	395104
0.2347	15.0	1350	0.2337	423360
0.234	16.0	1440	0.2339	451424
0.2286	17.0	1530	0.2401	479744
0.2242	18.0	1620	0.2419	507872
0.2176	19.0	1710	0.2452	535968
0.2169	20.0	1800	0.2442	564096

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4