train_mnli_1752826674

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1169
• Num Input Tokens Seen: 347859920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3684	0.5	44179	0.3379	17403808
0.2878	1.0	88358	0.2393	34786008
0.1163	1.5	132537	0.1938	52165240
0.1347	2.0	176716	0.1690	69564424
0.1089	2.5	220895	0.1554	86951080
0.1385	3.0	265074	0.1447	104352808
0.1468	3.5	309253	0.1368	121746504
0.1962	4.0	353432	0.1336	139123792
0.1397	4.5	397611	0.1285	156526672
0.1049	5.0	441790	0.1246	173916408
0.0996	5.5	485969	0.1228	191309592
0.1188	6.0	530148	0.1214	208701328
0.0982	6.5	574327	0.1209	226098768
0.104	7.0	618506	0.1187	243493272
0.0989	7.5	662685	0.1184	260881240
0.1001	8.0	706864	0.1174	278276232
0.0852	8.5	751043	0.1172	295687496
0.1511	9.0	795222	0.1169	313062872
0.1153	9.5	839401	0.1169	330444056
0.0404	10.0	883580	0.1169	347859920

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1