distilbert_sa_GLUE_Experiment_data_aug_mrpc

This model is a fine-tuned version of distilbert-base-uncased on the GLUE MRPC dataset. It achieves the following results on the evaluation set:

• Loss: 0.0
• Accuracy: 1.0
• F1: 1.0
• Combined Score: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 10
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Combined Score
0.1488	1.0	980	0.0012	1.0	1.0	1.0
0.0183	2.0	1960	0.0002	1.0	1.0	1.0
0.0072	3.0	2940	0.0000	1.0	1.0	1.0
0.0044	4.0	3920	0.0000	1.0	1.0	1.0
0.0031	5.0	4900	0.0002	1.0	1.0	1.0
0.0026	6.0	5880	0.0000	1.0	1.0	1.0
0.002	7.0	6860	0.0000	1.0	1.0	1.0
0.0018	8.0	7840	0.0000	1.0	1.0	1.0
0.0015	9.0	8820	0.0077	0.9975	0.9982	0.9979
0.0015	10.0	9800	0.0000	1.0	1.0	1.0
0.0012	11.0	10780	0.0001	1.0	1.0	1.0
0.0011	12.0	11760	0.0000	1.0	1.0	1.0
0.0007	13.0	12740	0.0000	1.0	1.0	1.0
0.001	14.0	13720	0.0000	1.0	1.0	1.0
0.0005	15.0	14700	0.0000	1.0	1.0	1.0
0.0007	16.0	15680	0.0000	1.0	1.0	1.0
0.0006	17.0	16660	0.0000	1.0	1.0	1.0
0.0005	18.0	17640	0.0000	1.0	1.0	1.0
0.0003	19.0	18620	0.0000	1.0	1.0	1.0
0.0007	20.0	19600	0.0000	1.0	1.0	1.0
0.0002	21.0	20580	0.0000	1.0	1.0	1.0
0.0005	22.0	21560	0.0000	1.0	1.0	1.0
0.0002	23.0	22540	0.0000	1.0	1.0	1.0
0.0003	24.0	23520	0.0000	1.0	1.0	1.0
0.0004	25.0	24500	0.0000	1.0	1.0	1.0
0.0004	26.0	25480	0.0000	1.0	1.0	1.0
0.0003	27.0	26460	0.0000	1.0	1.0	1.0
0.0002	28.0	27440	0.0000	1.0	1.0	1.0
0.0002	29.0	28420	0.0000	1.0	1.0	1.0
0.0002	30.0	29400	0.0000	1.0	1.0	1.0
0.0001	31.0	30380	0.0000	1.0	1.0	1.0
0.0001	32.0	31360	0.0000	1.0	1.0	1.0
0.0001	33.0	32340	0.0000	1.0	1.0	1.0
0.0	34.0	33320	0.0	1.0	1.0	1.0
0.0001	35.0	34300	0.0000	1.0	1.0	1.0
0.0	36.0	35280	0.0	1.0	1.0	1.0
0.0	37.0	36260	0.0000	1.0	1.0	1.0
0.0	38.0	37240	0.0	1.0	1.0	1.0
0.0001	39.0	38220	0.0000	1.0	1.0	1.0

Framework versions

• Transformers 4.26.0
• Pytorch 1.14.0a0+410ce96
• Datasets 2.9.0
• Tokenizers 0.13.2