train_math_qa_123_1760637720

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 2.3716
• Num Input Tokens Seen: 69273824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8202	2.0	11936	0.8008	6927488
0.6989	4.0	23872	0.7470	13853552
0.8507	6.0	35808	0.7326	20783424
0.7086	8.0	47744	0.7419	27711856
0.4468	10.0	59680	0.7902	34634288
0.449	12.0	71616	0.8520	41566552
0.4718	14.0	83552	1.1926	48493504
0.4175	16.0	95488	1.6549	55418256
0.1038	18.0	107424	2.2074	62346360
0.1559	20.0	119360	2.3716	69273824

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4