train_apps_1754507523

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the apps dataset. It achieves the following results on the evaluation set:

• Loss: 0.6802
• Num Input Tokens Seen: 880041568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7405	0.5000	13189	0.7618	44223136
0.6817	1.0000	26378	0.7273	87957952
0.6867	1.5001	39567	0.7115	131814656
0.5979	2.0001	52756	0.7031	175975840
0.6202	2.5001	65945	0.6974	219881664
0.6493	3.0001	79134	0.6931	263949472
0.6901	3.5001	92323	0.6900	307925280
0.6226	4.0002	105512	0.6877	352048320
0.5947	4.5002	118701	0.6858	396106880
0.6446	5.0002	131890	0.6843	440014752
0.7361	5.5002	145079	0.6832	484066880
0.6901	6.0002	158268	0.6823	528105600
0.858	6.5002	171457	0.6816	572089824
0.6076	7.0003	184646	0.6811	616130592
0.7387	7.5003	197835	0.6807	660063168
0.6938	8.0003	211024	0.6804	704033600
0.7924	8.5003	224213	0.6803	747976128
0.6861	9.0003	237402	0.6802	792077152
0.5865	9.5004	250591	0.6802	836063392

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1