bert_sm_gen1_cv_4

This model is a fine-tuned version of wiorz/bert_sm_gen1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4140
• Accuracy: 0.82
• Precision: 0.5758
• Recall: 0.2923
• F1: 0.3878
• D-index: 1.5600

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 8000
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	D-index
0.962	1.0	1000	0.6663	0.821	0.5930	0.2615	0.3630	1.5509
0.5779	2.0	2000	0.6352	0.818	0.5481	0.3795	0.4485	1.5864
0.4922	3.0	3000	0.9985	0.819	0.6667	0.1436	0.2363	1.5076
0.2595	4.0	4000	1.3708	0.806	0.5062	0.2103	0.2971	1.5130
0.1417	5.0	5000	1.5550	0.811	0.5326	0.2513	0.3415	1.5339
0.1007	6.0	6000	1.8121	0.808	0.5185	0.2154	0.3043	1.5175
0.1046	7.0	7000	1.9016	0.818	0.5657	0.2872	0.3810	1.5556
0.1286	8.0	8000	1.8942	0.815	0.5714	0.2051	0.3019	1.5235
0.108	9.0	9000	1.9444	0.802	0.4895	0.3590	0.4142	1.5581
0.0547	10.0	10000	1.8634	0.802	0.4887	0.3333	0.3963	1.5495
0.0288	11.0	11000	2.0029	0.83	0.6761	0.2462	0.3609	1.5578
0.0185	12.0	12000	2.2107	0.803	0.4926	0.3436	0.4048	1.5543
0.0088	13.0	13000	2.1847	0.817	0.5517	0.3282	0.4116	1.5680
0.0018	14.0	14000	2.3947	0.808	0.5118	0.3333	0.4037	1.5576
0.0152	15.0	15000	2.3443	0.823	0.5957	0.2872	0.3875	1.5623
0.016	16.0	16000	2.3187	0.815	0.5385	0.3590	0.4308	1.5756
0.0	17.0	17000	2.3557	0.817	0.5536	0.3179	0.4039	1.5646
0.0001	18.0	18000	2.4107	0.816	0.5433	0.3538	0.4286	1.5752
0.0	19.0	19000	2.4105	0.82	0.5758	0.2923	0.3878	1.5600
0.0	20.0	20000	2.4140	0.82	0.5758	0.2923	0.3878	1.5600

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3