train_multirc_456_1767340224

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.9299
• Num Input Tokens Seen: 264580656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1019	1.0	6130	0.1492	13210560
0.0956	2.0	12260	0.1234	26427632
0.0815	3.0	18390	0.1242	39656608
0.0414	4.0	24520	0.1161	52911264
0.2137	5.0	30650	0.1174	66151456
0.1817	6.0	36780	0.1147	79368416
0.1124	7.0	42910	0.1164	92601264
0.0382	8.0	49040	0.1142	105825424
0.0232	9.0	55170	0.1228	119051808
0.1069	10.0	61300	0.1186	132282512
0.1778	11.0	67430	0.1314	145503424
0.0117	12.0	73560	0.1569	158718688
0.0519	13.0	79690	0.1616	171968272
0.009	14.0	85820	0.1869	185192912
0.0024	15.0	91950	0.2105	198406496
0.0432	16.0	98080	0.2395	211654512
0.0009	17.0	104210	0.2806	224875552
0.0018	18.0	110340	0.2952	238097088
0.0049	19.0	116470	0.3146	251336240
0.0007	20.0	122600	0.3175	264580656

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1