train_wsc_789_1760372488

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 1.2339
• Num Input Tokens Seen: 1462816

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3507	1.504	188	0.3546	73440
0.382	3.008	376	0.3655	147296
0.3509	4.5120	564	0.3463	221744
0.3565	6.016	752	0.3463	293760
0.3248	7.52	940	0.3903	367808
0.353	9.024	1128	0.3551	440288
0.3466	10.528	1316	0.3552	512864
0.3383	12.032	1504	0.3610	587120
0.3473	13.536	1692	0.3624	660352
0.3373	15.04	1880	0.3700	733072
0.345	16.544	2068	0.3461	805824
0.3536	18.048	2256	0.4304	880160
0.332	19.552	2444	0.4619	955488
0.3151	21.056	2632	0.4579	1028544
0.2754	22.56	2820	0.6302	1101536
0.3333	24.064	3008	0.7934	1174032
0.2259	25.568	3196	0.8756	1247296
0.2669	27.072	3384	1.0698	1320640
0.2069	28.576	3572	1.2049	1394256

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1