train_conala_1754507516

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the conala dataset. It achieves the following results on the evaluation set:

• Loss: 0.5669
• Num Input Tokens Seen: 1524216

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6251	0.5	268	0.6428	75936
0.8223	1.0	536	0.5908	152672
0.5594	1.5	804	0.5868	229344
0.5917	2.0	1072	0.5669	305288
0.4615	2.5	1340	0.5915	382120
0.3834	3.0	1608	0.5813	457952
0.3483	3.5	1876	0.6248	534688
0.2366	4.0	2144	0.6150	610944
0.1721	4.5	2412	0.6774	687328
0.3979	5.0	2680	0.6894	762440
0.1644	5.5	2948	0.7816	839656
0.2128	6.0	3216	0.7775	914920
0.0629	6.5	3484	0.8908	992104
0.1303	7.0	3752	0.8886	1067520
0.1196	7.5	4020	1.0025	1142912
0.1486	8.0	4288	0.9892	1220200
0.1137	8.5	4556	1.0715	1295720
0.1189	9.0	4824	1.0935	1372560
0.064	9.5	5092	1.1125	1447376
0.0079	10.0	5360	1.1125	1524216

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1