train_piqa_789_1767826483

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.3534
• Num Input Tokens Seen: 39173664

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2381	2.0	6446	0.2318	3920704
0.2346	4.0	12892	0.2317	7838880
0.236	6.0	19338	0.2317	11753184
0.2325	8.0	25784	0.2312	15669040
0.2395	10.0	32230	0.2321	19584560
0.2057	12.0	38676	0.2388	23497808
0.1891	14.0	45122	0.2593	27418928
0.1466	16.0	51568	0.2939	31336560
0.094	18.0	58014	0.3373	35253392
0.1638	20.0	64460	0.3534	39173664

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1