Update README.md

README.md

@@ -1,16 +1,25 @@
 ---
 license: apache-2.0
+language: "en"
+tags:
+- longformer
+- clinical
+- biomedical
 ---
 
-KEPTlongfomer pretrained using [contrastive learning](https://arxiv.org/pdf/2210.03304.pdf).
+<span style="font-size:larger;">**KEPTlongfomer**</span> is a medical knowledge enhanced version of Longformer that was further pre-trained using [contrastive learning](https://arxiv.org/pdf/2210.03304.pdf). 
+The model achieves SOTA performance on auto ICD coding on MIMIC-III as of 11/12/2022.
+A sister model for better performance is available [here](https://huggingface.co/whaleloops/KEPTlongformer-PMM3/).
 
+### Pre-training
+We initialized this model from [clinical longformer](https://huggingface.co/yikuan8/Clinical-Longformer).
 
-First, The model was first inited from [clinical longformer](https://huggingface.co/yikuan8/Clinical-Longformer).
-
-And then pretrained with Hierarchical Self-Alignment Pretrainumls (HSAP) using Knowledge Graph UMLS.
+And then pretrained with Hierarchical Self-Alignment Pretrain (HSAP) using Knowledge Graph UMLS.
 This includes (a) Hierarchy, (b) Synonym, (c) Abbreviation. For more info, see section 3.3 in [paper](https://arxiv.org/pdf/2210.03304.pdf).
+The learning rate was 5e-5, weight decay was 0.01, adam epsilon was 1e-5.
 
-See [here](https://github.com/whaleloops/KEPT/tree/rerank300) for how to use this on auto ICD coding.
+### Usage
+See our [github](https://github.com/whaleloops/KEPT/tree/rerank300) for how to use this with prompts on auto ICD coding.
 
 With the following result:
 | Metric  | Score |
@@ -42,5 +51,4 @@ With the following result:
 |acc_macro| =0.08639139221100567|
 
 
-A sister model is available [here](https://huggingface.co/whaleloops/KEPTlongformer-PMM3/).