File size: 4,326 Bytes
a669fdb 62499af a669fdb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | ---
license: mit
library_name: pytorch
pipeline_tag: image-feature-extraction
tags:
- self-supervised-learning
- electron-microscopy
- 4d-stem
- scientific-imaging
- dinov2
- mae
- simclr
- vicregl
- i-jepa
---
# Physics-Aligned Self-Supervised Learning for Scientific Imaging
Pretrained ViT-B encoders from the GCPR 2026 paper
*Physics-Aligned Self-Supervised Learning for Scientific Imaging*.
**Code:** [DL4EM/physics-aligned-ssl](https://github.com/DL4EM/physics-aligned-ssl)
Five SSL methods (DINOv2, I-JEPA, MAE, SimCLR, VICRegL) were each pretrained
on two electron-microscopy modalities under two augmentation regimes:
- **`*_domain`** — physics-aligned augmentations (T_phys): measurement-consistent
symmetries plus acquisition-driven perturbations (noise, intensity variation,
reciprocal-space scaling, diffraction tilt, ...).
- **`*_original`** — standard natural-image augmentations (T_orig): random crop,
horizontal flip, blur, photometric perturbations.
Pretraining data:
- **`cem500k/`** — real-space cellular EM ([CEM500K](https://doi.org/10.7554/eLife.65894) subset, 10k images).
- **`4dstem/`** — simulated LiNiO2 4D-STEM diffraction patterns
([Scheunert et al.](https://doi.org/10.5281/zenodo.17360572) subset, 10k patterns).
All encoders are single-channel (grayscale) ViT-B backbones.
## Available models
| Path | Method | Pretraining data | Augmentations |
|---|---|---|---|
| `cem500k/dinov2_domain` | dinov2 | cem500k | physics-aligned |
| `cem500k/dinov2_original` | dinov2 | cem500k | natural-image |
| `cem500k/ijepa_domain` | ijepa | cem500k | physics-aligned |
| `cem500k/ijepa_original` | ijepa | cem500k | natural-image |
| `cem500k/mae_domain` | MAE | cem500k | physics-aligned |
| `cem500k/mae_original` | MAE | cem500k | natural-image |
| `cem500k/simclr_domain` | simclr | cem500k | physics-aligned |
| `cem500k/simclr_original` | simclr | cem500k | natural-image |
| `cem500k/vicregl_domain` | vicregl | cem500k | physics-aligned |
| `cem500k/vicregl_original` | vicregl | cem500k | natural-image |
| `4dstem/dinov2_domain` | dinov2 | 4dstem | physics-aligned |
| `4dstem/dinov2_original` | dinov2 | 4dstem | natural-image |
| `4dstem/ijepa_domain` | ijepa | 4dstem | physics-aligned |
| `4dstem/ijepa_original` | ijepa | 4dstem | natural-image |
| `4dstem/mae_domain` | MAE | 4dstem | physics-aligned |
| `4dstem/mae_original` | MAE | 4dstem | natural-image |
| `4dstem/simclr_domain` | simclr | 4dstem | physics-aligned |
| `4dstem/simclr_original` | simclr | 4dstem | natural-image |
| `4dstem/vicregl_domain` | vicregl | 4dstem | physics-aligned |
| `4dstem/vicregl_original` | vicregl | 4dstem | natural-image |
## Usage
With the accompanying code (https://github.com/DL4EM/physics-aligned-ssl):
```python
from em_ssl.hub import load_encoder
encoder = load_encoder("cem500k/dinov2_domain", repo_id="DL4EM/physics-aligned-ssl")
import torch
images = torch.randn(4, 1, 128, 128) # grayscale EM crops in [0, 1]
features = encoder(images)
```
Without the codebase, each `encoder.pt` is a plain PyTorch checkpoint:
```python
import torch
from huggingface_hub import hf_hub_download
path = hf_hub_download("DL4EM/physics-aligned-ssl", "cem500k/dinov2_domain/encoder.pt")
ckpt = torch.load(path, map_location="cpu", weights_only=True)
state_dict = ckpt["encoder_state_dict"] # ViT-B weights
print(ckpt["backbone"], ckpt["backbone_kwargs"])
```
Inputs are single-channel images normalised to [0, 1] (percentile
normalisation was used during pretraining). Real-space EM models were trained
on 128x128 crops; 4D-STEM models on 224x224 crops.
## Citation
```bibtex
@inproceedings{kazimi2026physicsaligned,
title = {Physics-Aligned Self-Supervised Learning for Scientific Imaging},
author = {Kazimi, Bashir and Sandfeld, Stefan},
booktitle = {DAGM German Conference on Pattern Recognition (GCPR)},
year = {2026}
}
```
## About
Developed by the [Deep Learning for Electron Microscopy (DL4EM)](https://www.fz-juelich.de/en/ias/ias-9/research/deep-learning-for-electron-microscopy)
group at the [Institute for Materials Data Science and Informatics (IAS-9)](https://www.fz-juelich.de/en/ias/ias-9),
Forschungszentrum Jülich. For questions, please open an issue on the
[GitHub repository](https://github.com/DL4EM/physics-aligned-ssl).
|