clm-model

This model is a fine-tuned version of emilyalsentzer/Bio_ClinicalBERT on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	38	0.0088
No log	2.0	76	0.0007
No log	3.0	114	0.0003
No log	4.0	152	0.0013
No log	5.0	190	0.0000
No log	6.0	228	0.0002
No log	7.0	266	0.0100
No log	8.0	304	0.0000
No log	9.0	342	0.0000
No log	10.0	380	0.0000
No log	11.0	418	0.0000
No log	12.0	456	0.0000
No log	13.0	494	0.0000
0.0057	14.0	532	0.0007
0.0057	15.0	570	0.0000
0.0057	16.0	608	0.0000
0.0057	17.0	646	0.0000
0.0057	18.0	684	0.0000
0.0057	19.0	722	0.0000
0.0057	20.0	760	0.0000
0.0057	21.0	798	0.0000
0.0057	22.0	836	0.0000
0.0057	23.0	874	0.0000
0.0057	24.0	912	0.0000
0.0057	25.0	950	0.0000
0.0057	26.0	988	0.0000
0.0018	27.0	1026	0.0000
0.0018	28.0	1064	0.0000
0.0018	29.0	1102	0.0000
0.0018	30.0	1140	0.0000
0.0018	31.0	1178	0.0000
0.0018	32.0	1216	0.0000
0.0018	33.0	1254	0.0000
0.0018	34.0	1292	0.0000
0.0018	35.0	1330	0.0000
0.0018	36.0	1368	0.0000
0.0018	37.0	1406	0.0000
0.0018	38.0	1444	0.0000
0.0018	39.0	1482	0.0000
0.0005	40.0	1520	0.0000
0.0005	41.0	1558	0.0000
0.0005	42.0	1596	0.0000
0.0005	43.0	1634	0.0000
0.0005	44.0	1672	0.0000
0.0005	45.0	1710	0.0000
0.0005	46.0	1748	0.0000
0.0005	47.0	1786	0.0000
0.0005	48.0	1824	0.0000
0.0005	49.0	1862	0.0000
0.0005	50.0	1900	0.0000

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3