train_copa_1757340254

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0287
• Num Input Tokens Seen: 281984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1204	0.5	45	0.0726	14240
0.3342	1.0	90	0.0615	28192
0.0623	1.5	135	0.0627	42080
0.0032	2.0	180	0.0287	56192
0.0001	2.5	225	0.0371	70048
0.0012	3.0	270	0.0343	84192
0.0002	3.5	315	0.0305	98304
0.0	4.0	360	0.0306	112544
0.0	4.5	405	0.0308	126784
0.0	5.0	450	0.0322	140960
0.0	5.5	495	0.0331	155200
0.0	6.0	540	0.0347	169216
0.0	6.5	585	0.0347	183232
0.0	7.0	630	0.0355	197248
0.0	7.5	675	0.0402	211424
0.0	8.0	720	0.0311	225440
0.0	8.5	765	0.0339	239392
0.0	9.0	810	0.0341	253632
0.0	9.5	855	0.0374	267680
0.0	10.0	900	0.0409	281984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1