train_record_1754594729

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3460
• Num Input Tokens Seen: 464483424

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5711	0.5	15621	0.6434	23227520
0.4926	1.0	31242	0.5021	46454112
0.4776	1.5	46863	0.4279	69694624
0.2224	2.0	62484	0.3974	92908288
0.2756	2.5	78105	0.3910	116099296
0.315	3.0	93726	0.3748	139351808
0.3339	3.5	109347	0.3696	162566976
0.3642	4.0	124968	0.3616	185790304
0.2647	4.5	140589	0.3546	208997696
0.461	5.0	156210	0.3507	232243968
0.2471	5.5	171831	0.3520	255458112
0.389	6.0	187452	0.3460	278686752
0.3541	6.5	203073	0.3478	301925344
0.2941	7.0	218694	0.3463	325137568
0.4292	7.5	234315	0.3475	348361920
0.3843	8.0	249936	0.3474	371592704
0.2701	8.5	265557	0.3475	394838368
0.3541	9.0	281178	0.3473	418033696
0.5188	9.5	296799	0.3472	441282560
0.3273	10.0	312420	0.3472	464483424

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1