train_copa_42_1760637531

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0320
• Num Input Tokens Seen: 564096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0823	1.0	90	0.0761	28256
0.0632	2.0	180	0.0425	56480
0.0001	3.0	270	0.0320	84736
0.0004	4.0	360	0.1331	113024
0.0	5.0	450	0.0919	141440
0.0	6.0	540	0.0916	169600
0.0	7.0	630	0.0936	197792
0.0	8.0	720	0.0946	225984
0.0	9.0	810	0.0954	254112
0.0	10.0	900	0.0975	282368
0.0	11.0	990	0.0974	310560
0.0	12.0	1080	0.0984	338784
0.0	13.0	1170	0.1005	366944
0.0	14.0	1260	0.1015	395104
0.0	15.0	1350	0.0984	423360
0.0	16.0	1440	0.1015	451424
0.0	17.0	1530	0.1004	479744
0.0	18.0	1620	0.1015	507872
0.0	19.0	1710	0.1015	535968
0.0	20.0	1800	0.1015	564096

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4