train_copa_42_1760637529

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2292
• Num Input Tokens Seen: 564096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.26	1.0	90	0.2362	28256
0.2246	2.0	180	0.2292	56480
0.2336	3.0	270	0.2309	84736
0.2347	4.0	360	0.2318	113024
0.2277	5.0	450	0.2366	141440
0.2348	6.0	540	0.2345	169600
0.2294	7.0	630	0.2314	197792
0.218	8.0	720	0.2308	225984
0.238	9.0	810	0.2327	254112
0.2218	10.0	900	0.2355	282368
0.2349	11.0	990	0.2300	310560
0.2315	12.0	1080	0.2369	338784
0.2297	13.0	1170	0.2312	366944
0.2276	14.0	1260	0.2317	395104
0.2402	15.0	1350	0.2313	423360
0.2338	16.0	1440	0.2334	451424
0.2339	17.0	1530	0.2329	479744
0.2307	18.0	1620	0.2334	507872
0.2307	19.0	1710	0.2349	535968
0.2288	20.0	1800	0.2324	564096

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4