train_codealpacapy_1754891836

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.4932
• Num Input Tokens Seen: 12472912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5525	0.5	954	0.5660	616992
0.642	1.0	1908	0.5418	1248304
0.5482	1.5	2862	0.5222	1877040
0.4587	2.0	3816	0.5130	2497016
0.6237	2.5	4770	0.5073	3129368
0.443	3.0	5724	0.5051	3742552
0.5001	3.5	6678	0.5016	4361944
0.4455	4.0	7632	0.4978	4985200
0.5341	4.5	8586	0.4971	5611760
0.5067	5.0	9540	0.4963	6233920
0.4896	5.5	10494	0.4949	6849184
0.5004	6.0	11448	0.4946	7478504
0.3888	6.5	12402	0.4947	8083560
0.4767	7.0	13356	0.4938	8722744
0.4332	7.5	14310	0.4934	9345976
0.7832	8.0	15264	0.4936	9977520
0.2855	8.5	16218	0.4937	10604656
0.5143	9.0	17172	0.4935	11225416
0.4405	9.5	18126	0.4935	11845704
0.672	10.0	19080	0.4932	12472912

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1