train_multirc_123_1765021423

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1400
• Num Input Tokens Seen: 264547520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1936	1.0	6130	0.1516	13255424
0.1205	2.0	12260	0.1400	26471216
0.0523	3.0	18390	0.1891	39694112
0.0009	4.0	24520	0.1859	52929744
0.0038	5.0	30650	0.2150	66152480
0.0008	6.0	36780	0.3181	79389648
0.008	7.0	42910	0.2110	92621824
0.0	8.0	49040	0.3400	105830544
0.0	9.0	55170	0.3483	119047920
0.0	10.0	61300	0.3544	132272272
0.0001	11.0	67430	0.3108	145487264
0.0001	12.0	73560	0.3321	158737232
0.0	13.0	79690	0.4225	171979232
0.0	14.0	85820	0.4302	185199728
0.0	15.0	91950	0.5138	198426688
0.0	16.0	98080	0.4468	211640976
0.0	17.0	104210	0.5200	224870720
0.0	18.0	110340	0.5392	238102672
0.0	19.0	116470	0.5471	251320768
0.0	20.0	122600	0.5459	264547520

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1