train_copa_456_1760637760

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2301
• Num Input Tokens Seen: 562720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2398	1.0	90	0.2346	28096
0.233	2.0	180	0.2352	56352
0.2393	3.0	270	0.2382	84544
0.2245	4.0	360	0.2353	112800
0.2239	5.0	450	0.2354	140800
0.2347	6.0	540	0.2346	168736
0.2294	7.0	630	0.2393	196896
0.2305	8.0	720	0.2325	225056
0.2306	9.0	810	0.2331	253312
0.2336	10.0	900	0.2310	281408
0.2327	11.0	990	0.2301	309440
0.2334	12.0	1080	0.2312	337536
0.2296	13.0	1170	0.2348	365760
0.233	14.0	1260	0.2345	393856
0.23	15.0	1350	0.2377	421952
0.2255	16.0	1440	0.2366	450016
0.2256	17.0	1530	0.2370	478240
0.239	18.0	1620	0.2385	506368
0.2192	19.0	1710	0.2388	534528
0.2363	20.0	1800	0.2376	562720

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4