train_siqa_1754507488

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the siqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2014
• Num Input Tokens Seen: 29840264

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1685	0.5	3759	0.2103	1495072
0.0643	1.0	7518	0.2110	2984720
0.1168	1.5	11277	0.2156	4477104
0.3262	2.0	15036	0.2014	5970384
0.1856	2.5	18795	0.2871	7462384
0.1235	3.0	22554	0.2563	8954176
0.0457	3.5	26313	0.4125	10445088
0.2364	4.0	30072	0.3816	11937344
0.0731	4.5	33831	0.4247	13430048
0.0	5.0	37590	0.4945	14920992
0.001	5.5	41349	0.4947	16412032
0.0001	6.0	45108	0.5627	17904680
0.0	6.5	48867	0.6525	19397416
0.0	7.0	52626	0.7320	20888856
0.0	7.5	56385	0.7387	22381080
0.0005	8.0	60144	0.9056	23872880
0.0	8.5	63903	1.0000	25363344
0.0	9.0	67662	0.9857	26855848
0.0	9.5	71421	1.0109	28348712
0.0	10.0	75180	1.0122	29840264

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1