combined-datset

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3475
• F1: 0.7749
• Chronic Disease F1: 0.7811
• Chronic Disease Num: 6976
• Cancer F1: 0.7088
• Cancer Num: 2484
• Allergy F1: 0.7781
• Allergy Num: 633
• Treatment F1: 0.7877
• Treatment Num: 9122
• Other F1: 0
• Other Num: 0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Chronic Disease F1	Chronic Disease Num	Cancer F1	Cancer Num	Allergy F1	Allergy Num	Treatment F1	Treatment Num	Other F1	Other Num
1.0053	0.0995	100	0.6273	0.4797	0.4210	6976	0.2625	2484	0.0	633	0.5749	9122	0	0
0.6022	0.1990	200	0.5180	0.6196	0.6303	6976	0.5183	2484	0.0372	633	0.6585	9122	0	0
0.5077	0.2985	300	0.4462	0.6808	0.6956	6976	0.6209	2484	0.5888	633	0.6912	9122	0	0
0.4836	0.3980	400	0.4200	0.7126	0.7088	6976	0.6376	2484	0.6402	633	0.7407	9122	0	0
0.428	0.4975	500	0.4063	0.7250	0.7205	6976	0.6494	2484	0.6271	633	0.7576	9122	0	0
0.3963	0.5970	600	0.3869	0.7388	0.7385	6976	0.6712	2484	0.7252	633	0.7588	9122	0	0
0.3916	0.6965	700	0.3812	0.7366	0.7304	6976	0.6629	2484	0.7206	633	0.7640	9122	0	0
0.4063	0.7960	800	0.3745	0.7464	0.7471	6976	0.6891	2484	0.7397	633	0.7616	9122	0	0
0.3839	0.8955	900	0.3609	0.7558	0.7544	6976	0.6856	2484	0.7549	633	0.7761	9122	0	0
0.3683	0.9950	1000	0.3671	0.7547	0.7558	6976	0.6987	2484	0.7197	633	0.7710	9122	0	0
0.3097	1.0945	1100	0.3593	0.7630	0.7633	6976	0.6958	2484	0.7649	633	0.7805	9122	0	0
0.3035	1.1940	1200	0.3611	0.7661	0.7665	6976	0.7029	2484	0.7731	633	0.7821	9122	0	0
0.312	1.2935	1300	0.3656	0.7641	0.7662	6976	0.6910	2484	0.7742	633	0.7825	9122	0	0
0.3281	1.3930	1400	0.3576	0.7653	0.7716	6976	0.7001	2484	0.7621	633	0.7783	9122	0	0
0.3213	1.4925	1500	0.3488	0.7664	0.7718	6976	0.6979	2484	0.7715	633	0.7811	9122	0	0
0.3169	1.5920	1600	0.3521	0.7703	0.7732	6976	0.7074	2484	0.7715	633	0.7856	9122	0	0
0.3227	1.6915	1700	0.3503	0.7747	0.7762	6976	0.7117	2484	0.7886	633	0.7891	9122	0	0
0.2871	1.7910	1800	0.3569	0.7721	0.7758	6976	0.7046	2484	0.7697	633	0.7881	9122	0	0
0.2927	1.8905	1900	0.3526	0.7731	0.7778	6976	0.7074	2484	0.7688	633	0.7875	9122	0	0
0.294	1.9900	2000	0.3475	0.7749	0.7811	6976	0.7088	2484	0.7781	633	0.7877	9122	0	0

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1