train_codealpacapy_789_1767616167

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.6284
• Num Input Tokens Seen: 21981320

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6441	2.0	3392	0.5280	2199016
0.4641	4.0	6784	0.5045	4393784
0.4362	6.0	10176	0.5032	6595976
0.4142	8.0	13568	0.5134	8792592
0.2648	10.0	16960	0.5319	10994984
0.304	12.0	20352	0.5534	13192800
0.2875	14.0	23744	0.5814	15387192
0.4434	16.0	27136	0.6058	17580200
0.3317	18.0	30528	0.6231	19780824
0.2477	20.0	33920	0.6284	21981320

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1