train_record_123_1764690864

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3035
• Num Input Tokens Seen: 928969984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5504	1.0	31242	0.4961	46454112
0.2245	2.0	62484	0.3945	92908288
0.2926	3.0	93726	0.3553	139351808
0.33	4.0	124968	0.3355	185790304
0.3204	5.0	156210	0.3241	232243968
0.2461	6.0	187452	0.3171	278686752
0.236	7.0	218694	0.3123	325137568
0.2352	8.0	249936	0.3103	371592704
0.258	9.0	281178	0.3076	418033696
0.3331	10.0	312420	0.3052	464483424
0.2565	11.0	343662	0.3039	510926720
0.2404	12.0	374904	0.3035	557369088
0.2283	13.0	406146	0.3041	603816992
0.214	14.0	437388	0.3038	650269248
0.2432	15.0	468630	0.3046	696727936
0.3179	16.0	499872	0.3048	743174112
0.2852	17.0	531114	0.3039	789614720
0.1657	18.0	562356	0.3044	836057280
0.1924	19.0	593598	0.3045	882504192
0.1652	20.0	624840	0.3042	928969984

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1