3f98d0ad63aa848b5da7fd68ddc7ab2f

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the nyu-mll/glue [stsb] dataset. It achieves the following results on the evaluation set:

• Loss: 0.5509
• Data Size: 1.0
• Epoch Runtime: 6.4474
• Mse: 0.5511
• Mae: 0.5580
• R2: 0.7535

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Mse	Mae	R2
No log	0	0	7.8540	0	0.9826	7.8553	2.3714	-2.5139
No log	1	179	5.9440	0.0078	1.2683	5.9451	2.0266	-1.6595
No log	2	358	3.2602	0.0156	1.4827	3.2612	1.5167	-0.4589
No log	3	537	2.2253	0.0312	1.4509	2.2261	1.2630	0.0042
No log	4	716	1.1032	0.0625	1.8202	1.1035	0.8635	0.5063
No log	5	895	0.9936	0.125	1.8730	0.9939	0.8246	0.5554
0.1251	6	1074	0.8622	0.25	2.5181	0.8624	0.6974	0.6142
0.6656	7	1253	0.5972	0.5	3.7077	0.5976	0.6084	0.7327
0.4603	8.0	1432	0.5940	1.0	6.1998	0.5942	0.5839	0.7342
0.3075	9.0	1611	0.5813	1.0	6.0365	0.5817	0.5747	0.7398
0.2103	10.0	1790	0.6247	1.0	6.1508	0.6250	0.5952	0.7204
0.175	11.0	1969	0.6059	1.0	5.8653	0.6063	0.5958	0.7288
0.1258	12.0	2148	0.5752	1.0	5.9733	0.5755	0.5775	0.7426
0.1118	13.0	2327	0.5749	1.0	5.9488	0.5752	0.5836	0.7427
0.1072	14.0	2506	0.6250	1.0	6.3627	0.6253	0.5955	0.7203
0.0915	15.0	2685	0.5812	1.0	6.1292	0.5814	0.5819	0.7399
0.0861	16.0	2864	0.5729	1.0	6.1470	0.5733	0.5869	0.7435
0.0746	17.0	3043	0.5570	1.0	6.1972	0.5573	0.5651	0.7507
0.0622	18.0	3222	0.5680	1.0	6.2005	0.5683	0.5658	0.7458
0.0615	19.0	3401	0.5926	1.0	6.1791	0.5930	0.5899	0.7348
0.0586	20.0	3580	0.5546	1.0	6.4506	0.5549	0.5664	0.7518
0.0536	21.0	3759	0.5897	1.0	6.2795	0.5900	0.5813	0.7361
0.0459	22.0	3938	0.6049	1.0	6.5179	0.6052	0.6006	0.7293
0.0536	23.0	4117	0.6260	1.0	6.6828	0.6263	0.6095	0.7198
0.0513	24.0	4296	0.5492	1.0	6.8893	0.5496	0.5620	0.7542
0.0449	25.0	4475	0.6370	1.0	6.6974	0.6373	0.6078	0.7149
0.0445	26.0	4654	0.6491	1.0	6.4258	0.6493	0.6097	0.7095
0.0421	27.0	4833	0.5900	1.0	6.4171	0.5902	0.5879	0.7360
0.043	28.0	5012	0.5509	1.0	6.4474	0.5511	0.5580	0.7535

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1