train_copa_1757340255

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0593
• Num Input Tokens Seen: 281984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2081	0.5	45	0.1157	14240
0.2591	1.0	90	0.0701	28192
0.0367	1.5	135	0.0683	42080
0.1331	2.0	180	0.0597	56192
0.0047	2.5	225	0.0593	70048
0.0918	3.0	270	0.0596	84192
0.1091	3.5	315	0.0617	98304
0.0053	4.0	360	0.0622	112544
0.0101	4.5	405	0.0630	126784
0.0808	5.0	450	0.0620	140960
0.0104	5.5	495	0.0627	155200
0.0012	6.0	540	0.0637	169216
0.0056	6.5	585	0.0677	183232
0.0014	7.0	630	0.0702	197248
0.0042	7.5	675	0.0686	211424
0.0692	8.0	720	0.0670	225440
0.0005	8.5	765	0.0679	239392
0.0015	9.0	810	0.0698	253632
0.0069	9.5	855	0.0690	267680
0.003	10.0	900	0.0675	281984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1