train_copa_1757340253

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 2.4562
• Num Input Tokens Seen: 281984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.411	0.5	45	0.3011	14240
0.2838	1.0	90	0.2367	28192
0.2526	1.5	135	0.2344	42080
0.2443	2.0	180	0.2417	56192
0.2322	2.5	225	0.2302	70048
0.2431	3.0	270	0.2324	84192
0.2331	3.5	315	0.2341	98304
0.2335	4.0	360	0.2316	112544
0.2376	4.5	405	0.2335	126784
0.2371	5.0	450	0.2323	140960
0.2308	5.5	495	0.2329	155200
0.2303	6.0	540	0.2314	169216
0.2276	6.5	585	0.2329	183232
0.2262	7.0	630	0.2323	197248
0.2449	7.5	675	0.2311	211424
0.2253	8.0	720	0.2308	225440
0.2314	8.5	765	0.2292	239392
0.2314	9.0	810	0.2329	253632
0.2303	9.5	855	0.2335	267680
0.2304	10.0	900	0.2313	281984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1