Update architecture description and Cyclosporine A example

README.md

@@ -11,7 +11,7 @@ tags:
   - peptide-language-model
 pipeline_tag: fill-mask
 widget:
-  - text: "PEPTIDE1{A.C.D.E.F}$$$$"
+  - text: "PEPTIDE1{[Abu].[Sar].[meL].V.[meL].A.[dA].[meL].[meL].[meV].[Me_Bmt(E)]}$PEPTIDE1,PEPTIDE1,1:R1-11:R2$$$"
 ---
 
 # HELM-BERT
@@ -20,12 +20,12 @@ A language model for peptide representation learning using **HELM (Hierarchical
 
 ## Model Description
 
-HELM-BERT is a BERT-style encoder designed specifically for peptide sequences in HELM notation. It incorporates several architectural innovations:
+HELM-BERT is built upon the DeBERTa architecture, designed for peptide sequences in HELM notation:
 
-- **Disentangled Attention**: Separate content and position representations (DeBERTa-style)
-- **Enhanced Mask Decoder (EMD)**: Absolute position encoding for MLM pretraining
-- **Span Masking**: Contiguous token masking for improved contextual learning
-- **nGiE**: n-gram Induced Encoding layer for local pattern recognition
+- **Disentangled Attention**: Decomposes attention into content-content and content-position terms
+- **Enhanced Mask Decoder (EMD)**: Injects absolute position embeddings at the decoder stage
+- **Span Masking**: Contiguous token masking with geometric distribution
+- **nGiE**: n-gram Induced Encoding layer (1D convolution, kernel size 3)
 
 Please check the [official repository](https://github.com/clinfo/HELM-BERT) for more implementation details and updates.
 
@@ -48,7 +48,8 @@ from transformers import AutoModel, AutoTokenizer
 model = AutoModel.from_pretrained("Flansma/helm-bert", trust_remote_code=True)
 tokenizer = AutoTokenizer.from_pretrained("Flansma/helm-bert", trust_remote_code=True)
 
-inputs = tokenizer("PEPTIDE1{A.C.D.E.F}$$$$", return_tensors="pt")
+# Cyclosporine A
+inputs = tokenizer("PEPTIDE1{[Abu].[Sar].[meL].V.[meL].A.[dA].[meL].[meL].[meV].[Me_Bmt(E)]}$PEPTIDE1,PEPTIDE1,1:R1-11:R2$$$", return_tensors="pt")
 outputs = model(**inputs)
 embeddings = outputs.last_hidden_state
 ```