train_codealpacapy_456_1765199214

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.6115
• Num Input Tokens Seen: 22326528

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.39	2.0	3392	0.5126	2240840
0.4011	4.0	6784	0.4946	4476776
0.6057	6.0	10176	0.4917	6712376
0.3139	8.0	13568	0.5016	8942288
0.4498	10.0	16960	0.5173	11175920
0.4502	12.0	20352	0.5455	13399864
0.2905	14.0	23744	0.5702	15636912
0.2389	16.0	27136	0.5845	17867032
0.3118	18.0	30528	0.6043	20098848
0.3651	20.0	33920	0.6115	22326528

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1