train_record_456_1766854165

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3041
• Num Input Tokens Seen: 928892640

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3069	1.0	31242	0.4128	46454656
0.3459	2.0	62484	0.3382	92898208
0.1799	3.0	93726	0.3156	139330944
0.4149	4.0	124968	0.3095	185787424
0.3992	5.0	156210	0.3041	232232736
0.2352	6.0	187452	0.3095	278675168
0.2106	7.0	218694	0.3214	325124320
0.1436	8.0	249936	0.3222	371565312
0.3777	9.0	281178	0.3379	418010016
0.1611	10.0	312420	0.3476	464454880
0.1494	11.0	343662	0.3599	510906784
0.1012	12.0	374904	0.3680	557340128
0.1716	13.0	406146	0.3843	603790528
0.1444	14.0	437388	0.3924	650253184
0.181	15.0	468630	0.4001	696691296
0.0998	16.0	499872	0.4028	743122464
0.1245	17.0	531114	0.4113	789557088
0.209	18.0	562356	0.4151	835994816
0.2816	19.0	593598	0.4178	882444928
0.2542	20.0	624840	0.4183	928892640

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1