bert_sm_cv_summarized_4

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.9996
• Accuracy: 0.802
• Precision: 0.48
• Recall: 0.1846
• F1: 0.2667
• D-index: 1.4986

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 8000
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	D-index
No log	1.0	250	0.4713	0.812	0.5814	0.1282	0.2101	1.4926
0.5708	2.0	500	0.4584	0.811	0.5625	0.1385	0.2222	1.4948
0.5708	3.0	750	0.4557	0.813	0.5769	0.1538	0.2429	1.5029
0.4231	4.0	1000	0.4700	0.81	0.5316	0.2154	0.3066	1.5202
0.4231	5.0	1250	0.4979	0.812	0.5385	0.2513	0.3427	1.5353
0.3292	6.0	1500	0.5337	0.816	0.5647	0.2462	0.3429	1.5389
0.3292	7.0	1750	0.6282	0.797	0.4615	0.2462	0.3211	1.5131
0.2218	8.0	2000	0.7182	0.805	0.5	0.2513	0.3345	1.5257
0.2218	9.0	2250	0.8488	0.809	0.5208	0.2564	0.3436	1.5329
0.1478	10.0	2500	0.9830	0.809	0.5294	0.1846	0.2738	1.5082
0.1478	11.0	2750	1.0302	0.79	0.4419	0.2923	0.3519	1.5193
0.077	12.0	3000	1.0467	0.795	0.4658	0.3487	0.3988	1.5452
0.077	13.0	3250	1.2609	0.803	0.4931	0.3641	0.4189	1.5612
0.0328	14.0	3500	1.4127	0.806	0.5044	0.2923	0.3701	1.5411
0.0328	15.0	3750	1.6626	0.802	0.4835	0.2256	0.3077	1.5128
0.0189	16.0	4000	1.7062	0.81	0.5362	0.1897	0.2803	1.5113
0.0189	17.0	4250	1.9225	0.809	0.54	0.1385	0.2204	1.4921
0.0214	18.0	4500	1.8228	0.81	0.5269	0.2513	0.3403	1.5325
0.0214	19.0	4750	1.9544	0.789	0.4355	0.2769	0.3386	1.5127
0.0184	20.0	5000	1.9996	0.802	0.48	0.1846	0.2667	1.4986

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3