| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: pytorch |
| base_model: Qwen/Qwen2.5-7B-Instruct |
| datasets: |
| - WinstonHu/LoC-Path-Data |
| tags: |
| - loc-path |
| - pathology |
| - whole-slide-imaging |
| - visual-question-answering |
| - lora |
| --- |
| |
| # LoC-Path |
|
|
| Pretrained LoC-Path weights for whole-slide pathology understanding from |
| pre-extracted CONCH-v1 features. This release uses a **128-latent resampler |
| (`L=128`)**, followed by routing to **96 visual latents (`M=96`)** for CARA. |
| The language backbone is **Qwen2.5-7B-Instruct**, with LoRA rank 128. |
|
|
| ## Released checkpoints |
|
|
| | File | Training checkpoint | Purpose | |
| |---|---|---| |
| | `checkpoints/mae.pth` | 100,000 iterations | Dedicated 128-latent MAE pretraining weights, including the reconstruction decoder | |
| | `checkpoints/stage1.pth` | 3,615 iterations | Complete visual weights after report alignment | |
| | `checkpoints/stage2.pth` | 20,480 iterations | Complete visual weights and Qwen LoRA for inference or initialization | |
|
|
| Each file is a PyTorch dictionary containing a tensor-only `state_dict`. |
| Checkpoints are consolidated in FP16 using the repository's ZeRO loader. Frozen |
| visual modules omitted by historical ZeRO saves are carried forward from their |
| MAE/Stage-1 sources into the complete exported state. A Stage-2 inference run |
| requires only `stage2.pth`, its matching config, the Qwen base model and features. |
|
|
| Qwen base weights are downloaded from |
| [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) by the |
| LoC-Path loader. CONCH weights are obtained separately when extracting new |
| features. This repository distributes the trained LoC-Path visual modules and |
| LoRA; original image/report assets and optimizer/scheduler states are absent. |
|
|
| The [model manifest](model_manifest.json) records source checkpoints, tensor |
| counts, sizes and SHA256 hashes. The [validation summary](validation.json) |
| records configuration/weight checks and execution tests. |
|
|
| ## Download |
|
|
| Install the LoC-Path training package using its README, then run: |
|
|
| ```bash |
| export LOCPATH_MODEL_ROOT=$PWD/pretrained_models/LoC-Path |
| uv run --locked python - <<'PY' |
| import os |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download( |
| repo_id="WinstonHu/LoC-Path", |
| repo_type="model", |
| revision="v1.0", |
| local_dir=os.environ["LOCPATH_MODEL_ROOT"], |
| ) |
| PY |
| ``` |
|
|
| For inference-only use, optionally add |
| `allow_patterns=["checkpoints/stage2.pth", "configs/locpath_stage2.py", "README.md", "model_manifest.json", "validation.json", "LICENSE"]` |
| to the download call. |
|
|
| ## Run inference |
|
|
| Download the prepared WSI-Bench/BCNB data from |
| [WinstonHu/LoC-Path-Data](https://huggingface.co/datasets/WinstonHu/LoC-Path-Data/tree/v1.0). |
| Run from the installed LoC-Path code directory: |
|
|
| ```bash |
| export LOCPATH_DATA_ROOT=$PWD/data/LoC-Path-Data |
| export CONFIG="$LOCPATH_MODEL_ROOT/configs/locpath_stage2.py" |
| export CHECKPOINT="$LOCPATH_MODEL_ROOT/checkpoints/stage2.pth" |
| export FEATURE_ROOT="$LOCPATH_DATA_ROOT/features/conch_v1" |
| export DATA_DIR="$LOCPATH_DATA_ROOT/annotations/test/wsi_bench" |
| export NPROC_PER_NODE=1 |
| |
| CUDA_VISIBLE_DEVICES=0 uv run --locked bash scripts/test_vis_adapter.sh \ |
| --image_feature_suffix .npy --router-top-m 96 \ |
| --sample_num 61440 --max_new_tokens 512 |
| ``` |
|
|
| For BCNB, set `DATA_DIR="$LOCPATH_DATA_ROOT/annotations/test/external"` and run |
| the same command. Its JSON points to the included numeric feature CSVs. |
| To use an already downloaded Qwen base, set |
| `LOCPATH_LLM_PATH=/path/to/Qwen2.5-7B-Instruct`. |
|
|
| Use the supplied configs: all three stages have 128 latent queries. Set `CONFIG` |
| to the corresponding file when invoking a training launcher: |
|
|
| | Launcher | Config | Initialization | |
| |---|---|---| |
| | `scripts/train_resampler_pretrain.sh` | `configs/locpath_mae.py` | Train MAE from scratch, or initialize using the MAE weights | |
| | `scripts/train_stage1.sh` | `configs/locpath_stage1.py` | `LOCPATH_RESAMPLER_CKPT` points to `checkpoints/mae.pth` | |
| | `scripts/train_stage2.sh` | `configs/locpath_stage2.py` | `LOCPATH_RESAMPLER_CKPT` points to MAE; `LOCPATH_STAGE2_PRETRAINED_PTH` points to Stage 1 | |
|
|
| Set feature and annotation environment variables as described in the training |
| package README. Training starts a new optimizer/scheduler run; these exports |
| are weights for initialization and inference. |
|
|
| ## Validation and use |
|
|
| The three exports were loaded into their matching model architectures with all |
| required LoC-Path tensors present and exact loaded values. The dedicated |
| 128-latent Stage-2 model was tested on an actual Qwen2.5-7B backbone using |
| prepared WSI-Bench and BCNB samples. Execution checks are distinct from a full |
| benchmark-metric reproduction. Use this model for computational pathology |
| research with the corresponding source-data and feature-extractor terms. |
|
|
| LoC-Path weights are released under Apache-2.0. Attribute the LoC-Path project |
| and the Qwen base model. Data and CONCH terms are documented in the prepared-data |
| repository and apply to those assets separately. |
|
|
