train_wic_1756729606

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wic dataset. It achieves the following results on the evaluation set:

• Loss: 0.9198
• Num Input Tokens Seen: 4063904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3437	0.5002	1222	0.3579	202848
0.3506	1.0004	2444	0.3595	406568
0.3567	1.5006	3666	0.3403	609912
0.3306	2.0008	4888	0.3404	813272
0.2855	2.5010	6110	0.3362	1016632
0.3023	3.0012	7332	0.3397	1219720
0.3683	3.5014	8554	0.3276	1422696
0.2582	4.0016	9776	0.3698	1626184
0.403	4.5018	10998	0.3320	1828792
0.2438	5.0020	12220	0.3536	2032536
0.2303	5.5023	13442	0.3724	2236040
0.1731	6.0025	14664	0.3558	2439144
0.2849	6.5027	15886	0.5030	2642552
0.4576	7.0029	17108	0.4559	2845672
0.204	7.5031	18330	0.6020	3048792
0.2955	8.0033	19552	0.5823	3252416
0.0122	8.5035	20774	0.8709	3455872
0.1146	9.0037	21996	0.7875	3659120
0.3564	9.5039	23218	0.9142	3862176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1