README.md

---
license: mit
library_name: pytorch
pipeline_tag: image-feature-extraction
tags:
- self-supervised-learning
- electron-microscopy
- 4d-stem
- scientific-imaging
- dinov2
- mae
- simclr
- vicregl
- i-jepa
---

# Physics-Aligned Self-Supervised Learning for Scientific Imaging

Pretrained ViT-B encoders from the GCPR 2026 paper
*Physics-Aligned Self-Supervised Learning for Scientific Imaging*.

**Code:** [DL4EM/physics-aligned-ssl](https://github.com/DL4EM/physics-aligned-ssl)

Five SSL methods (DINOv2, I-JEPA, MAE, SimCLR, VICRegL) were each pretrained
on two electron-microscopy modalities under two augmentation regimes:

- **`*_domain`** — physics-aligned augmentations (T_phys): measurement-consistent
  symmetries plus acquisition-driven perturbations (noise, intensity variation,
  reciprocal-space scaling, diffraction tilt, ...).
- **`*_original`** — standard natural-image augmentations (T_orig): random crop,
  horizontal flip, blur, photometric perturbations.

Pretraining data:

- **`cem500k/`** — real-space cellular EM ([CEM500K](https://doi.org/10.7554/eLife.65894) subset, 10k images).
- **`4dstem/`** — simulated LiNiO2 4D-STEM diffraction patterns
  ([Scheunert et al.](https://doi.org/10.5281/zenodo.17360572) subset, 10k patterns).

All encoders are single-channel (grayscale) ViT-B backbones.

## Available models

| Path | Method | Pretraining data | Augmentations |
|---|---|---|---|
| `cem500k/dinov2_domain` | dinov2 | cem500k | physics-aligned |
| `cem500k/dinov2_original` | dinov2 | cem500k | natural-image |
| `cem500k/ijepa_domain` | ijepa | cem500k | physics-aligned |
| `cem500k/ijepa_original` | ijepa | cem500k | natural-image |
| `cem500k/mae_domain` | MAE | cem500k | physics-aligned |
| `cem500k/mae_original` | MAE | cem500k | natural-image |
| `cem500k/simclr_domain` | simclr | cem500k | physics-aligned |
| `cem500k/simclr_original` | simclr | cem500k | natural-image |
| `cem500k/vicregl_domain` | vicregl | cem500k | physics-aligned |
| `cem500k/vicregl_original` | vicregl | cem500k | natural-image |
| `4dstem/dinov2_domain` | dinov2 | 4dstem | physics-aligned |
| `4dstem/dinov2_original` | dinov2 | 4dstem | natural-image |
| `4dstem/ijepa_domain` | ijepa | 4dstem | physics-aligned |
| `4dstem/ijepa_original` | ijepa | 4dstem | natural-image |
| `4dstem/mae_domain` | MAE | 4dstem | physics-aligned |
| `4dstem/mae_original` | MAE | 4dstem | natural-image |
| `4dstem/simclr_domain` | simclr | 4dstem | physics-aligned |
| `4dstem/simclr_original` | simclr | 4dstem | natural-image |
| `4dstem/vicregl_domain` | vicregl | 4dstem | physics-aligned |
| `4dstem/vicregl_original` | vicregl | 4dstem | natural-image |

## Usage

With the accompanying code (https://github.com/DL4EM/physics-aligned-ssl):

```python
from em_ssl.hub import load_encoder

encoder = load_encoder("cem500k/dinov2_domain", repo_id="DL4EM/physics-aligned-ssl")

import torch
images = torch.randn(4, 1, 128, 128)   # grayscale EM crops in [0, 1]
features = encoder(images)
```

Without the codebase, each `encoder.pt` is a plain PyTorch checkpoint:

```python
import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("DL4EM/physics-aligned-ssl", "cem500k/dinov2_domain/encoder.pt")
ckpt = torch.load(path, map_location="cpu", weights_only=True)
state_dict = ckpt["encoder_state_dict"]     # ViT-B weights
print(ckpt["backbone"], ckpt["backbone_kwargs"])
```

Inputs are single-channel images normalised to [0, 1] (percentile
normalisation was used during pretraining). Real-space EM models were trained
on 128x128 crops; 4D-STEM models on 224x224 crops.

## Citation

```bibtex
@inproceedings{kazimi2026physicsaligned,
  title     = {Physics-Aligned Self-Supervised Learning for Scientific Imaging},
  author    = {Kazimi, Bashir and Sandfeld, Stefan},
  booktitle = {DAGM German Conference on Pattern Recognition (GCPR)},
  year      = {2026}
}
```

## About

Developed by the [Deep Learning for Electron Microscopy (DL4EM)](https://www.fz-juelich.de/en/ias/ias-9/research/deep-learning-for-electron-microscopy)
group at the [Institute for Materials Data Science and Informatics (IAS-9)](https://www.fz-juelich.de/en/ias/ias-9),
Forschungszentrum Jülich. For questions, please open an issue on the
[GitHub repository](https://github.com/DL4EM/physics-aligned-ssl).