2ms
/

EXAONE-Path-CRCMSI-Predictor

PyTorch

lg-ai

EXAONEPath-1.5

pathology

Model card Files Files and versions

xet

Community

2ms commited on Jul 7, 2025

Commit

9522baa

1 Parent(s): e46a236

update README.md

Browse files

Files changed (1) hide show

README.md +68 -74

README.md CHANGED Viewed

@@ -7,88 +7,82 @@ tags:
 - EXAONEPath-1.5
 - pathology
 ---
-# EXAONE Path 1.5
 ## Introduction
-EXAONE Path 1.5 is a whole slide image level(WSI-level) classification framework designed for downstream tasks in pathology, such as cancer subtyping, molecular subtyping and mutation prediction. It builds upon our previous work, EXAONE Path v1.0, which focused on patch-wise feature extraction by dividing a WSI into patches and embedding each patch into a feature vector.
-In EXAONE Path 1.5, we extend this pipeline to take an entire WSI as input. Each patch is first processed using the pretrained EXAONE Path 1.0 encoder to extract patch-level features. These features are then aggregated using a ViT-based (Vision Transformer) aggregator module to produce a slide-level representation.
-This aggregated representation is subsequently passed through a linear classifier to perform downstream tasks such as molecular subtyping, tumor subtyping, and mutation prediction.
-To effectively train the aggregator, we adopt a two-stage learning process:
-Pretraining: We employ multimodal learning by aligning slide images with various mRNA gene expression profiles to learn semantically meaningful slide-level representations.
-Fine-tuning: The pretrained model is then adapted to specific downstream classification tasks.
-In this repository, we release the model trained for EGFR mutation prediction in lung adenocarcinoma (LUAD), enabling researchers to leverage our pipeline for similar molecular pathology applications.
-## Quickstart
-### 1. Hardware Requirements ###
-- NVIDIA GPU is required
-- Minimum 40GB GPU memory recommended
-- Tested on Ubuntu 22.04 with NVIDIA driver version 550.144.03
-Note: This implementation requires NVIDIA GPU and drivers. The provided environment setup specifically uses CUDA-enabled PyTorch, making NVIDIA GPU mandatory for running the model.
-### 2. Environment Setup
-```
 pip install -r requirements.txt
 ```
-### 3-a. Load the model & Inference
-#### Load model with HuggingFace
 ```python
 from models.exaonepath import EXAONEPathV1p5Downstream
 hf_token = "YOUR_HUGGING_FACE_ACCESS_TOKEN"
-model = EXAONEPathV1p5Downstream.from_pretrained("LGAI-EXAONE/EXAONE-Path-1.5", use_auth_token=hf_token)
-slide_path = './samples/wsis/1/1.svs'
-probs = model(slide_path)
-```
-#### Fast CLI Inference
-Before running the command below, make sure you update your Hugging Face token.
-Open `tokens.py` and replace the placeholder with your actual token:
-```python
-HF_TOKEN = "YOUR_HUGGING_FACE_ACCESS_TOKEN"
 ```
-Then, run inference with:
 ```bash
 python inference.py --svs_path ./samples/wsis/1/1.svs
 ```
-### 3-b. Fine-tuning with Pretrained Weights
-We provide example scripts and files to help you fine-tune the model on your own dataset.
-The provided script fine-tunes the model using pretrained weights stored in `./pretrained_weight.pth`.
-#### Extract Features from WSI Images
-To train the model using WSI images and their corresponding labels,
-you must first extract patch-level features from each WSI using our provided feature extractor.
-```bash
-python feature_extract.py --input_dir ./samples/wsis/ --output_dir ./samples/feats/
-```
-This will generate .pt feature files in the output_dir.
-#### Fine-tuning
-```bash
-bash tuning_script.sh
-```
-Inside tuning_script.sh, you can modify the following variables to match your dataset:
-```bash
-FEAT_PATH=./samples/feats
-LABEL_PATH=./samples/label/label.csv
-LABEL_DICT="{'n':0, 'y':1}"
-SPLIT_PATH=./samples/splits
-```
-Change these paths to point to your own feature, label, and split files to start training.
-## Model Performance Comparison
-| Metric: AUC                      | Titan(Conch v1.5+iBot, image+text) | PRISM (virchow+pe receiver, Image+text) | CHIEF (CTransPath + CLAM, Image+text, clam+wsi contrastive) | Prov-GigaPath (GigaPath+LongNet, Image-only, mask precision manner) | UNI2-h + CLAM (Image-only) | EXAONE Path 1.5(image+gene expression) |
-|--------------------------|----------------------------------|-----------------------------------------|--------------------------------------------------------------|------------------------------------------------------------------------|-----------------------------|------------------|
-| **TMB (cutoff 10)**                | 0.74                             | 0.73                                    | 0.70                                                         | 0.69                                                                   | 0.71                        | 0.71             |
-| **LUAD-EGFR-mut**        | 0.76                             | 0.80                                    | 0.73                                                         | 0.73                                                                   | 0.79                        | 0.81             |
-| **LUAD-KRAS-mut**        | 0.61                             | 0.65                                    | 0.61                                                         | 0.66                                                                   | 0.60                        | 0.63             |
-| **LUAD-Gene-overexp[1]**    | 0.75                             | 0.68                                    | 0.71                                                         | 0.71                                                                   | 0.74                        | 0.72             |
-| **CRC-MSS/MSI**              | 0.89                             | 0.88                                    | 0.86                                                         | 0.90                                                                   | 0.90                        | 0.89             |
-| **BRCA-ER_PR_HER2**      | 0.82                             | 0.79                                    | 0.76                                                         | 0.79                                                                   | 0.81                        | 0.77             |
-| **Pan-cancer-Gene-mut[2]**  | 0.79                             | 0.77                                    | 0.73                                                         | 0.74                                                                   | 0.77                        | 0.76             |
-| **Avg. AUC**             | 0.77                             | 0.76                                    | 0.73                                                         | 0.74                                                                   | 0.77                        | 0.76             |
-[1]: **lung-gene-overexp**: total 11 genes were evaluated: LAG3, CLDN6, CD274, EGFR, ERBB2, ERBB3, CD276, VTCN1, TACSTD2, FOLR1, MET.
-[2]: **Pan-cancer-Gene-mut**: total 7 genes were evaluated: TP53, KRAS, ALK, PIK3CA, MET, EGFR, PTEN
-## License
-The model is licensed under [EXAONEPath AI Model License Agreement 1.0 - NC](./LICENSE)

 - EXAONEPath-1.5
 - pathology
 ---
+# EXAONEPath for CRCMSI – CRCMSI-centric Whole-Slide Image Classifier
+*A purpose-built upgrade of **EXAONE Path 1.5***
 ## Introduction
+**EXAONEPath for CRCMSI** is an **enhanced whole-slide image (WSI) classification framework** that retains the core architecture of EXAONE Path 1.5 while upgrading its internals for greater efficiency and richer multimodal integration.
+The pipeline still unfolds in two stages:
+1. **Patch-wise feature extraction** – Each WSI is tiled into 256 × 256 px patches, which are embedded into 768-dimensional vectors using the frozen **[EXAONE Path v1.0](https://huggingface.co/LGAI-EXAONE/EXAONEPath)** encoder.
+2. **Slide-level aggregation** – The patch embeddings are aggregated using a Vision Transformer, producing a unified slide-level representation that a lightweight classification head transforms into task-specific probabilities.
+---
+## Key Improvements
+- **[FlexAttention](https://pytorch.org/blog/flexattention/) + `torch.compile`**
+  *What changed:* Replaced vanilla multi‑head self‑attention with IO‑aware **FlexAttention** kernels and enabled `torch.compile` to fuse the forward/backward graph at runtime. The new kernel layout dramatically improves both memory efficiency and training-and-inference throughput.
+- **Coordinate‑aware Relative Bias**
+  *What changed:* Added an ALiBi‑style distance bias that is computed from the (x, y) patch coordinates themselves, allowing the ViT aggregator to reason about spatial proximity.
+- **Scalable Mixed‑Omics Encoder (Token‑mixing Transformer)**
+  *What changed:* Each omics modality is first tokenised into a fixed‑length set. **All modality‑specific tokens are concatenated into a single sequence and passed through a shared multi‑head self‑attention stack**, enabling direct information exchange across modalities in one shot. The aggregated omics representation is subsequently fused with image tokens via cross‑attention. This release uses **three modalities (RNA, CNV, DNA‑methylation)**, but the design is agnostic to modality count and scales linearly with token number.
+---
+## Quick Start
+### Requirements
+- NVIDIA GPU (≥ 40 GB)
+- CUDA 12.8
+- pytorch 2.7.0+cu128
+### Installation
+```bash
+git clone https://huggingface.co/LGAI-EXAONE/{MODEL_NAME}.git
+cd {MODEL_NAME}
 pip install -r requirements.txt
 ```
+### Quick Inference
 ```python
 from models.exaonepath import EXAONEPathV1p5Downstream
 hf_token = "YOUR_HUGGING_FACE_ACCESS_TOKEN"
+model = EXAONEPathV1p5Downstream.from_pretrained(
+    "LGAI-EXAONE/{MODEL_NAME}",
+    use_auth_token=hf_token
+)
+probs = model("./samples/wsis/1/1.svs")
+print(f"P(CRCMSI mutant) = {probs[1]:.3f}")
 ```
+#### Command‑line
 ```bash
 python inference.py --svs_path ./samples/wsis/1/1.svs
 ```
+### Model Performance Comparison
+| Metric (AUC) / Task                | Titan (Conch v1.5 + iBot, image-text) | PRISM (virchow + perceiver, image-text) | CHIEF (CTransPath + CLAM, image-text, WSI-contrastive) | Prov-GigaPath (GigaPath + LongNet, image-only, mask-prediction) | UNI2-h + CLAM (image-only) | EXAONEPath V1.5 | **{MODEL_NAME}** |
+|------------------------------------|---------------------------------------|-----------------------------------------|--------------------------------------------------------|-----------------------------------------------------------------|---------------------------|------------------------|------------------------|
+| **CRC-MSI**                    | 0.9370 | 0.9432 | 0.9273 | 0.9541 | <u>0.9808</u> | 0.9537 | **0.9844** |
+| LUAD-TMB (cutoff 10)           | 0.6901 | 0.6445 | 0.6501 | 0.6744 | 0.6686 | 0.6846 | 0.6842 |
+| LUAD-EGFR-mut                  | 0.8197 | 0.8152 | 0.7691 | 0.7623 | 0.8577 | 0.7607 | 0.8564 |
+| LUAD-KRAS-mut                  | 0.5405 | 0.6299 | 0.4676 | 0.5110 | 0.4690 | 0.5480 | 0.6038 |
+| BRCA-ER                        | 0.9343 | 0.8998 | 0.9115 | 0.9186 | 0.9454 | 0.9096 | 0.9278 |
+| BRCA-PR                        | 0.8804 | 0.8613 | 0.8470 | 0.8595 | 0.8770 | 0.8215 | 0.8430 |
+| BRCA-HER2                      | 0.8046 | 0.8154 | 0.7822 | 0.7891 | 0.8322 | 0.7811 | 0.8050 |
+| BRCA-TP53                      | 0.7879 | 0.8415 | 0.7879 | 0.7388 | 0.8080 | 0.6607 | 0.7656 |
+| BRCA-PIK3CA                    | 0.7577 | 0.8929 | 0.7015 | 0.7347 | 0.8571 | 0.7066 | 0.7908 |
+| RCC-PBRM1                      | 0.6383 | 0.5570 | 0.5129 | 0.5270 | 0.5011 | 0.4445 | 0.5780 |
+| RCC-BAP1                       | 0.7188 | 0.7690 | 0.7310 | 0.6970 | 0.7160 | 0.7337 | 0.7323 |
+| COAD-KRAS                      | 0.7642 | 0.7443 | 0.6989 | 0.8153 | 0.9432 | 0.6790 | 0.8693 |
+| COAD-TP53                      | 0.8889 | 0.8160 | 0.7014 | 0.7118 | 0.7830 | 0.8785 | 0.8715 |
+| <span style="color:red">**Average**</span> | 0.7817 | 0.7869 | 0.7299 | 0.7457 | <u>0.7876</u> | 0.7356 | <span style="color:red">**0.7932**</span> |