license: cc-by-nc-nd-4.0
FusOn-pLM: A Fusion Oncoprotein-Specific Language Model via Focused Probabilistic Masking
In this work, we introduce FusOn-pLM, a novel pLM that fine-tunes state-of-the-art ESM-2 embeddings on fusion oncoprotein sequences, those that drive a large portion of pediatric cancers but are heavily disordered and undruggable, via masked language modeling (MLM). We specifically introduce a novel MLM strategy, employing a binding-site probability predictor to focus masking on key amino acid residues, thereby generating more optimal fusion oncoprotein-aware embeddings. Our model improves performance on both fusion oncoprotein-specific benchmarks and disorder prediction tasks in comparison to baseline ESM-2 representations, as well as manually-constructed biophysical embeddings, motivating downstream usage of FusOn-pLM embeddings for therapeutic design tasks targeting these fusions.
How to Use FusOn-pLM
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("ChatterjeeLab/FusOn-pLM")
model = AutoModelForMaskedLM.from_pretrained("ChatterjeeLab/FusOn-pLM")