File size: 5,179 Bytes
03ae676 88fde16 37e3811 03ae676 88fde16 9522baa 5b6672a 88fde16 5b6672a 9522baa 03ae676 9522baa 03ae676 9522baa 03ae676 9522baa 03ae676 9522baa 03ae676 9522baa 599021c 88fde16 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 |
---
license: other
license_name: exaonepath
license_link: LICENSE
tags:
- lg-ai
- EXAONEPath-1.5
- pathology
---
<!--# EXAONE Path for CRCMSI – CRCMSI-centric Whole-Slide Image Classifier
*A purpose-built upgrade of **EXAONE Path 1.5***-->
## Introduction
<!--**EXAONE Path for CRCMSI** is an **enhanced whole-slide image (WSI) classification framework** that retains the core architecture of EXAONE Path 1.5 while upgrading its internals for greater efficiency and richer multimodal integration.-->
**EXAONE Path MSI** is an **enhanced whole-slide image (WSI) classification framework** that retains the core architecture of EXAONE Path while upgrading its internals for greater efficiency and richer multimodal integration.
The pipeline still unfolds in two stages:
1. **Patch-wise feature extraction** – Each WSI is tiled into 256 × 256 px patches, which are embedded into 768-dimensional vectors using the frozen **[EXAONE Path](https://huggingface.co/LGAI-EXAONE/EXAONEPath)** encoder.
2. **Slide-level aggregation** – The patch embeddings are aggregated using a Vision Transformer, producing a unified slide-level representation that a lightweight classification head transforms into task-specific probabilities.
---
## Key Improvements
- **[FlexAttention](https://pytorch.org/blog/flexattention/) + `torch.compile`**
*What changed:* Replaced vanilla multi‑head self‑attention with IO‑aware **FlexAttention** kernels and enabled `torch.compile` to fuse the forward/backward graph at runtime. The new kernel layout dramatically improves both memory efficiency and training-and-inference throughput.
- **Coordinate‑aware Relative Bias**
*What changed:* Added an ALiBi‑style distance bias that is computed from the (x, y) patch coordinates themselves, allowing the ViT aggregator to reason about spatial proximity.
- **Scalable Mixed‑Omics Encoder (Token‑mixing Transformer)**
*What changed:* Each omics modality is first tokenised into a fixed‑length set. **All modality‑specific tokens are concatenated into a single sequence and passed through a shared multi‑head self‑attention stack**, enabling direct information exchange across modalities in one shot. The aggregated omics representation is subsequently fused with image tokens via cross‑attention. This release uses **three modalities (RNA, CNV, DNA‑methylation)**, but the design is agnostic to modality count and scales linearly with token number.
---
## Quick Start
### Requirements
- NVIDIA GPU (≥ 40 GB)
- CUDA 12.8
- pytorch 2.7.0+cu128
### Installation
```bash
git clone https://huggingface.co/LGAI-EXAONE/{MODEL_NAME}.git
cd {MODEL_NAME}
pip install -r requirements.txt
```
### Quick Inference
```python
from models.exaonepath import EXAONEPathV1p5Downstream
hf_token = "YOUR_HUGGING_FACE_ACCESS_TOKEN"
model = EXAONEPathV1p5Downstream.from_pretrained(
"LGAI-EXAONE/{MODEL_NAME}",
use_auth_token=hf_token
)
probs = model("./samples/wsis/1/1.svs")
print(f"P(CRCMSI mutant) = {probs[1]:.3f}")
```
#### Command‑line
```bash
python inference.py --svs_path ./samples/wsis/1/1.svs
```
### Model Performance Comparison
| Metric (AUC) / Task | Titan (Conch v1.5 + iBot, image-text) | PRISM (virchow + perceiver, image-text) | CHIEF (CTransPath + CLAM, image-text, WSI-contrastive) | Prov-GigaPath (GigaPath + LongNet, image-only, mask-prediction) | UNI2-h + CLAM (image-only) | EXAONE Path 1.5 | EXAONE Path MSI |
|------------------------------------|---------------------------------------|-----------------------------------------|--------------------------------------------------------|-----------------------------------------------------------------|---------------------------|------------------------|------------------------|
| **CRC-MSI** | 0.9370 | 0.9432 | 0.9273 | 0.9541 | <u>0.9808</u> | 0.9537 |**0.9844** |
<!--| LUAD-TMB (cutoff 10) | 0.6901 | 0.6445 | 0.6501 | 0.6744 | 0.6686 | 0.6846 | |
| LUAD-EGFR-mut | 0.8197 | 0.8152 | 0.7691 | 0.7623 | 0.8577 | 0.7607 | |
| LUAD-KRAS-mut | 0.5405 | 0.6299 | 0.4676 | 0.5110 | 0.4690 | 0.5480 | |
| BRCA-ER | 0.9343 | 0.8998 | 0.9115 | 0.9186 | 0.9454 | 0.9096 | |
| BRCA-PR | 0.8804 | 0.8613 | 0.8470 | 0.8595 | 0.8770 | 0.8215 | |
| BRCA-HER2 | 0.8046 | 0.8154 | 0.7822 | 0.7891 | 0.8322 | 0.7811 | |
| BRCA-TP53 | 0.7879 | 0.8415 | 0.7879 | 0.7388 | 0.8080 | 0.6607 | |
| BRCA-PIK3CA | 0.7577 | 0.8929 | 0.7015 | 0.7347 | 0.8571 | 0.7066 | |
| RCC-PBRM1 | 0.6383 | 0.5570 | 0.5129 | 0.5270 | 0.5011 | 0.4445 | |
| RCC-BAP1 | 0.7188 | 0.7690 | 0.7310 | 0.6970 | 0.7160 | 0.7337 | |
| COAD-KRAS | 0.7642 | 0.7443 | 0.6989 | 0.8153 | 0.9432 | 0.6790 | |
| COAD-TP53 | 0.8889 | 0.8160 | 0.7014 | 0.7118 | 0.7830 | 0.8785 | |
| <span style="color:red">**Average**</span> | 0.7817 | 0.7869 | 0.7299 | 0.7457 | <u>0.7876</u> | <span style="color:red">**0.7932**</span> |--> |