9b66e4483f5cbff92cfb2c20d8e1a81f

This model is a fine-tuned version of distilbert/distilbert-base-cased-distilled-squad on the nyu-mll/glue [stsb] dataset. It achieves the following results on the evaluation set:

• Loss: 0.6967
• Data Size: 1.0
• Epoch Runtime: 5.9652
• Mse: 0.6969
• Mae: 0.6427
• R2: 0.6883

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Mse	Mae	R2
No log	0	0	7.3543	0	0.9942	7.3556	2.2808	-2.2904
No log	1	179	4.2252	0.0078	1.3244	4.2263	1.7023	-0.8906
No log	2	358	2.3292	0.0156	1.1961	2.3301	1.3202	-0.0423
No log	3	537	2.1836	0.0312	1.3239	2.1843	1.2308	0.0229
No log	4	716	0.9544	0.0625	1.5381	0.9546	0.7922	0.5730
No log	5	895	0.7618	0.125	1.7451	0.7621	0.7013	0.6591
0.1014	6	1074	0.6948	0.25	2.4084	0.6951	0.6461	0.6891
0.6512	7	1253	0.6423	0.5	3.6445	0.6426	0.6408	0.7125
0.4968	8.0	1432	0.8064	1.0	6.0629	0.8064	0.6965	0.6392
0.3322	9.0	1611	0.7923	1.0	5.9311	0.7924	0.6862	0.6455
0.2245	10.0	1790	0.6275	1.0	5.8613	0.6276	0.6081	0.7193
0.1754	11.0	1969	0.5531	1.0	5.9418	0.5534	0.5708	0.7524
0.1354	12.0	2148	0.6066	1.0	5.9438	0.6067	0.5873	0.7286
0.1233	13.0	2327	0.5548	1.0	6.0239	0.5551	0.5662	0.7517
0.1093	14.0	2506	0.5704	1.0	6.0559	0.5706	0.5739	0.7448
0.0962	15.0	2685	0.6967	1.0	5.9652	0.6969	0.6427	0.6883

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1