Update README.md

README.md

@@ -9,7 +9,7 @@ base_model:
 
 ## **Fine-Tuning ESM-1b with Multiple Sequence Alignment (MSA) for Phosphosites**
 
-This repository provides a fine-tuned version of ESM-1b, incorporating genomic information by leveraging long phosphosite sequences from [DARKIN dataset](https://openreview.net/pdf?id=a4x5tbYRYV) and Multiple Sequence Alignment (MSA) of those phosphosites. The goal is to enhance the model's understanding of phosphorylation by integrating sequence conservation patterns.
+This repository provides a fine-tuned version of ESM-1b with Masked Language Modeling(MLM) Objective, incorporating genomic information by leveraging long phosphosite sequences from [DARKIN dataset](https://openreview.net/pdf?id=a4x5tbYRYV) and Multiple Sequence Alignment (MSA) of those phosphosites. The goal is to enhance the model's understanding of phosphorylation by integrating sequence conservation patterns.
 
 ### Developed by:
 
@@ -26,19 +26,20 @@ To construct a robust dataset, we extracted 256 MSA sequences per phosphosite fr
   - 10% of the data was reserved for validation.
   - The remaining 90% was used for fine-tuning with the Masked Language Modeling (MLM) objective.
 3. Data Processing & Preprocessing
-  - Special attention was given to retaining phosphorylation residues within sequences.
+  - Special attention was given to conserving phosphorylation residues within sequences.
   - To optimize memory efficiency, sequence lengths were truncated to 128 amino acids.
 
 ### Evaluation
 
 Perplexity: 2.69 (decreased from 7.05)
 
-from transformers import AutoTokenizer, AutoModelForMaskedLM
-import torch
 
 ### Usage
 
 ```
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+import torch
+
 # Load the model and tokenizer
 model_name = "isikz/phosphosite_msa_finetuned_esm1b"
 tokenizer = AutoTokenizer.from_pretrained(model_name)