train_multirc_123_1764927932

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3157
• Num Input Tokens Seen: 264547520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3641	1.0	6130	0.3419	13255424
0.3802	2.0	12260	0.3391	26471216
0.3397	3.0	18390	0.3381	39694112
0.3309	4.0	24520	0.3236	52929744
0.329	5.0	30650	0.3203	66152480
0.31	6.0	36780	0.3372	79389648
0.3205	7.0	42910	0.3282	92621824
0.352	8.0	49040	0.3183	105830544
0.337	9.0	55170	0.3230	119047920
0.4224	10.0	61300	0.3175	132272272
0.3779	11.0	67430	0.3180	145487264
0.3303	12.0	73560	0.3168	158737232
0.2844	13.0	79690	0.3211	171979232
0.3063	14.0	85820	0.3171	185199728
0.3332	15.0	91950	0.3163	198426688
0.3605	16.0	98080	0.3157	211640976
0.3138	17.0	104210	0.3173	224870720
0.4002	18.0	110340	0.3172	238102672
0.2938	19.0	116470	0.3175	251320768
0.2706	20.0	122600	0.3172	264547520

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1