train_copa_456_1760637761

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.4119
• Num Input Tokens Seen: 562720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2658	1.0	90	0.2338	28096
0.2448	2.0	180	0.2335	56352
0.2379	3.0	270	0.2368	84544
0.223	4.0	360	0.2444	112800
0.2277	5.0	450	0.2391	140800
0.2325	6.0	540	0.2354	168736
0.2296	7.0	630	0.2359	196896
0.2306	8.0	720	0.2307	225056
0.2271	9.0	810	0.2339	253312
0.2352	10.0	900	0.2325	281408
0.2299	11.0	990	0.2367	309440
0.2327	12.0	1080	0.2337	337536
0.2405	13.0	1170	0.2351	365760
0.1816	14.0	1260	0.2497	393856
0.1983	15.0	1350	0.2575	421952
0.1913	16.0	1440	0.2836	450016
0.1652	17.0	1530	0.3111	478240
0.1664	18.0	1620	0.3957	506368
0.0979	19.0	1710	0.4245	534528
0.1026	20.0	1800	0.4507	562720

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4