distilbert_sa_GLUE_Experiment_data_aug_mrpc_256

This model is a fine-tuned version of distilbert-base-uncased on the GLUE MRPC dataset. It achieves the following results on the evaluation set:

• Loss: 0.0
• Accuracy: 1.0
• F1: 1.0
• Combined Score: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 10
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Combined Score
0.2052	1.0	980	0.0476	0.9853	0.9894	0.9873
0.0409	2.0	1960	0.0031	1.0	1.0	1.0
0.0211	3.0	2940	0.0006	1.0	1.0	1.0
0.0131	4.0	3920	0.0005	1.0	1.0	1.0
0.0078	5.0	4900	0.0001	1.0	1.0	1.0
0.0058	6.0	5880	0.0002	1.0	1.0	1.0
0.0041	7.0	6860	0.0000	1.0	1.0	1.0
0.0035	8.0	7840	0.0000	1.0	1.0	1.0
0.0029	9.0	8820	0.0000	1.0	1.0	1.0
0.0022	10.0	9800	0.0000	1.0	1.0	1.0
0.0021	11.0	10780	0.0000	1.0	1.0	1.0
0.0015	12.0	11760	0.0000	1.0	1.0	1.0
0.0017	13.0	12740	0.0000	1.0	1.0	1.0
0.0011	14.0	13720	0.0000	1.0	1.0	1.0
0.0013	15.0	14700	0.0000	1.0	1.0	1.0
0.0011	16.0	15680	0.0000	1.0	1.0	1.0
0.0009	17.0	16660	0.0000	1.0	1.0	1.0
0.0008	18.0	17640	0.0000	1.0	1.0	1.0
0.0008	19.0	18620	0.0000	1.0	1.0	1.0
0.0007	20.0	19600	0.0000	1.0	1.0	1.0
0.0006	21.0	20580	0.0000	1.0	1.0	1.0
0.0007	22.0	21560	0.0000	1.0	1.0	1.0
0.0005	23.0	22540	0.0000	1.0	1.0	1.0
0.0003	24.0	23520	0.0000	1.0	1.0	1.0
0.0003	25.0	24500	0.0000	1.0	1.0	1.0
0.0003	26.0	25480	0.0000	1.0	1.0	1.0
0.0003	27.0	26460	0.0000	1.0	1.0	1.0
0.0003	28.0	27440	0.0000	1.0	1.0	1.0
0.0002	29.0	28420	0.0000	1.0	1.0	1.0
0.0002	30.0	29400	0.0000	1.0	1.0	1.0
0.0002	31.0	30380	0.0000	1.0	1.0	1.0
0.0002	32.0	31360	0.0000	1.0	1.0	1.0
0.0001	33.0	32340	0.0000	1.0	1.0	1.0
0.0001	34.0	33320	0.0000	1.0	1.0	1.0
0.0002	35.0	34300	0.0000	1.0	1.0	1.0
0.0001	36.0	35280	0.0000	1.0	1.0	1.0
0.0	37.0	36260	0.0000	1.0	1.0	1.0
0.0	38.0	37240	0.0000	1.0	1.0	1.0
0.0	39.0	38220	0.0000	1.0	1.0	1.0
0.0001	40.0	39200	0.0000	1.0	1.0	1.0
0.0	41.0	40180	0.0	1.0	1.0	1.0
0.0	42.0	41160	0.0	1.0	1.0	1.0
0.0001	43.0	42140	0.0000	1.0	1.0	1.0
0.0	44.0	43120	0.0	1.0	1.0	1.0
0.0	45.0	44100	0.0	1.0	1.0	1.0
0.0	46.0	45080	0.0	1.0	1.0	1.0

Framework versions

• Transformers 4.26.0
• Pytorch 1.14.0a0+410ce96
• Datasets 2.9.0
• Tokenizers 0.13.2