train_siqa_1754507487

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the siqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1941
• Num Input Tokens Seen: 29840264

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3784	0.5	3759	0.2872	1495072
0.1227	1.0	7518	0.2456	2984720
0.2894	1.5	11277	0.2292	4477104
0.2361	2.0	15036	0.2174	5970384
0.1743	2.5	18795	0.2126	7462384
0.1359	3.0	22554	0.2060	8954176
0.0448	3.5	26313	0.2038	10445088
0.2796	4.0	30072	0.2000	11937344
0.2362	4.5	33831	0.1983	13430048
0.2512	5.0	37590	0.1994	14920992
0.0642	5.5	41349	0.1975	16412032
0.0621	6.0	45108	0.1941	17904680
0.276	6.5	48867	0.1949	19397416
0.3155	7.0	52626	0.1958	20888856
0.2464	7.5	56385	0.1955	22381080
0.435	8.0	60144	0.1947	23872880
0.1351	8.5	63903	0.1947	25363344
0.2039	9.0	67662	0.1943	26855848
0.1998	9.5	71421	0.1945	28348712
0.2321	10.0	75180	0.1946	29840264

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1