f0423f31a6211b5ecbf2df8d4434f9e6

This model is a fine-tuned version of google-bert/bert-large-uncased-whole-word-masking-finetuned-squad on the nyu-mll/glue [stsb] dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: constant
num_epochs: 50

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Mse	Mae	R2
No log	0	0	4.8834	0	1.6867	4.8846	1.8258	-1.1850
No log	1	179	2.4962	0.0078	2.1935	2.4969	1.3040	-0.1170
No log	2	358	2.3085	0.0156	2.4065	2.3091	1.2431	-0.0329
No log	3	537	1.0328	0.0312	2.9389	1.0326	0.8196	0.5381
No log	4	716	1.4205	0.0625	4.0007	1.4206	0.9986	0.3645
No log	5	895	1.0129	0.125	5.5330	1.0129	0.8154	0.5469
0.0642	6	1074	0.7299	0.25	8.3834	0.7301	0.6644	0.6734
0.5283	7	1253	0.5287	0.5	11.7109	0.5290	0.5638	0.7633
0.4959	8.0	1432	0.4980	1.0	20.8520	0.4982	0.5322	0.7771
0.3067	9.0	1611	0.4710	1.0	20.8597	0.4713	0.5261	0.7892
0.2461	10.0	1790	0.4967	1.0	19.7950	0.4969	0.5221	0.7777
0.2146	11.0	1969	0.4542	1.0	19.9428	0.4544	0.5057	0.7967
0.1765	12.0	2148	0.4712	1.0	19.9652	0.4715	0.5254	0.7891
0.1737	13.0	2327	0.4619	1.0	20.0585	0.4620	0.5043	0.7933
0.1581	14.0	2506	0.5046	1.0	20.9366	0.5048	0.5257	0.7742
0.148	15.0	2685	0.4691	1.0	19.9136	0.4693	0.5243	0.7901

Safetensors

Model size

0.3B params

Tensor type

F32

Base model

Finetuned

(42)

this model