train_siqa_1754507489

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the siqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2043
• Num Input Tokens Seen: 29840264

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3732	0.5	3759	0.2616	1495072
0.0959	1.0	7518	0.2394	2984720
0.2425	1.5	11277	0.2178	4477104
0.2264	2.0	15036	0.2132	5970384
0.1443	2.5	18795	0.2104	7462384
0.0769	3.0	22554	0.2043	8954176
0.0355	3.5	26313	0.2150	10445088
0.1974	4.0	30072	0.2121	11937344
0.2309	4.5	33831	0.2186	13430048
0.234	5.0	37590	0.2406	14920992
0.0825	5.5	41349	0.2476	16412032
0.0427	6.0	45108	0.2370	17904680
0.1816	6.5	48867	0.2530	19397416
0.3742	7.0	52626	0.2595	20888856
0.0675	7.5	56385	0.2672	22381080
0.514	8.0	60144	0.2603	23872880
0.0856	8.5	63903	0.2688	25363344
0.0116	9.0	67662	0.2671	26855848
0.2034	9.5	71421	0.2691	28348712
0.1225	10.0	75180	0.2691	29840264

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1