Med3DINO

Three geometry-aware 3D medical encoders for representation learning and downstream research.

Evaluation code · Project page · Checkpoint guide

Med3DINO-ISO, REL and SA are self-supervised ViT-L/16 encoders trained from random initialization. The variants compare different geometric policies during pretraining.

Checkpoints

Variant	Pretraining geometry	REFORM-3D key	Weight file
ISO	Isotropic normalization to 1.5 × 1.5 × 1.5 mm, followed by voxel-relative cropping	med3dino_iso_96	c96_iso_teacher_checkpoint.pth
REL	Native spacing with voxel-relative cropping	med3dino_rel_96	c96_rel_teacher_checkpoint.pth
SA	Native spacing with relative in-plane and physical through-plane sampling; anisotropy-aware cross-plane rotations	med3dino_sa_96	c96_sa_teacher_checkpoint.pth

All three use a single-channel 3D ViT-L/16 backbone with 24 transformer blocks, 16 attention heads and 1,024-dimensional embeddings. The registered evaluation input is B × 1 × 96 × 96 × 96, producing a 6 × 6 × 6 patch grid. Each implemented backbone has 306.78M parameters, including positional embeddings and mask tokens and excluding pretraining projection heads. Each checkpoint file is 1.42GB.

ISO, REL and SA identify the geometric policies used during pretraining; inference preprocessing is described under Input preparation.

The REFORM-3D project page describes a curated pretraining corpus of 62,638 volumes spanning CT, MR, PET, CTA and CBCT. See the corpus summary and geometric-policy comparison.

Training scans are not distributed here, and their provider terms remain separate from the weight license.

Pretraining and evaluation

The Med3DINO pretraining recipe combines DINO self-distillation on CLS tokens, iBOT masked patch prediction and KoLeo regularization, with an exponential-moving-average teacher. It uses two global crops of size 96³, eight local crops of size 48³ and AdamW optimization.

Download

Install REFORM-3D using its installation instructions. From the cloned REFORM3D repository, with the same environment active, install the Hugging Face client:

python -m pip install -c requirements.txt huggingface_hub

Authenticate with hf auth login when needed, then download the same fixed weight revision used by the REFORM-3D README. Fetch the checksum file separately from main:

hf download maregu2023/Med3DINO \
  med3dino_iso/c96_iso_teacher_checkpoint.pth \
  med3dino_rel/c96_rel_teacher_checkpoint.pth \
  med3dino_sa/c96_sa_teacher_checkpoint.pth \
  --revision 3404b24c7fb060215c15998aec6de42c7d6febc1 \
  --local-dir checkpoints
hf download maregu2023/Med3DINO SHA256SUMS --revision main --local-dir checkpoints

Downloads are placed in checkpoints/ under the REFORM3D repository. For YAML files in configs/, set checkpoint_root: ../checkpoints; paths resolve beside the YAML file:

checkpoints/
├── med3dino_iso/c96_iso_teacher_checkpoint.pth
├── med3dino_rel/c96_rel_teacher_checkpoint.pth
└── med3dino_sa/c96_sa_teacher_checkpoint.pth

The reference weight revision is 3404b24c7fb060215c15998aec6de42c7d6febc1. SHA256SUMS records the three complete-file hashes, also recorded in REFORM-3D's checkpoint metadata. Verify them from the REFORM3D repository:

(cd checkpoints && sha256sum -c SHA256SUMS)

Input preparation

For the registered 96³ evaluation inputs, prepare single-channel tensors with shape [B, 1, 96, 96, 96] and spatial axes ordered as [D, H, W]. REFORM-3D's evaluation preprocessing clips each source volume at its 0.05th and 99.95th intensity percentiles and scales intensities to [-1, 1]. The extractor converts tensors to its configured dtype, which defaults to float32 through the factory below.

Spatial preprocessing follows the selected evaluation protocol, including its spacing, axis-order, cropping and resampling rules. Use the configured evaluation pipeline for benchmark comparisons; resizing a scan to 96³ alone does not reproduce that preparation.

Extract frozen features

This example loads SA once and defines a function for preprocessed batches. The encoder is frozen and placed in evaluation mode by REFORM-3D.

import torch
from huggingface_hub import hf_hub_download
from reform3d.encoders.factory import create_feature_extractor

device = "cuda:0" if torch.cuda.is_available() else "cpu"
checkpoint = hf_hub_download(
    repo_id="maregu2023/Med3DINO",
    filename="med3dino_sa/c96_sa_teacher_checkpoint.pth",
    revision="3404b24c7fb060215c15998aec6de42c7d6febc1",
)
encoder = create_feature_extractor(
    "med3dino_sa_96", checkpoint_path=checkpoint, device=device
)

def encode(batch):
    # batch: preprocessed [B, 1, 96, 96, 96] tensor.
    with torch.inference_mode():
        features = encoder.extract(batch.to(device), normalize=True)
    return features["cls"], features["avg_pool"]

Both returned tensors have shape [B, 1024]. cls is the global token; avg_pool is the mean of the final patch tokens. normalize=True L2-normalizes the embeddings, not raw scan intensities. For ISO or REL, change both the filename and registry key.

Use the configured evaluation pipeline for spacing sensitivity, CT–MR alignment and low-shot anatomical assessment.

Intended use and limitations

These encoders support research on frozen representations and downstream adaptation. Task-specific readouts or decoders and appropriate validation are required for segmentation or classification. Performance can depend on acquisition geometry, modality, anatomy and the target dataset. The checkpoints do not establish clinical diagnostic performance.

License and attribution

The three Med3DINO checkpoint files are offered under CC BY-NC 4.0; see LICENSE. This permits sharing and adaptation for noncommercial purposes with the required attribution and notices. Commercial use requires separate permission from the applicable rights holders.

This weight license does not replace the terms of REFORM-3D's implementation or third-party components. The implementation includes DINOv2- and 3DINO-derived components with separately recorded source licenses; see source notices.

📖 Citation & acknowledgment

Please acknowledge Med3DINO and REFORM-3D when using these checkpoints in your research.

Maregu Assefa, Muzammal Naseer, Divya Velayudhan, Gedamu Alemu Kumie and Naoufel Werghi. REFORM-3D: A Representation-Centric Evaluation Framework for 3D Medical Vision Foundation Models. See the project page for citation details.

@misc{assefa_reform3d,
  title = {{REFORM-3D}: A Representation-Centric Evaluation Framework for 3D Medical Vision Foundation Models},
  author = {Assefa, Maregu and Naseer, Muzammal and Velayudhan, Divya and Kumie, Gedamu Alemu and Werghi, Naoufel},
  year = {2026},
  url = {https://endarzboy.github.io/reform3d-website/}
}

Contact: Maregu Assefa.