MicroStructFormer (Swin-Large)

MicroStructFormer is a high-capacity Mask2Former model built on a Swin-Large backbone, designed for the semantic segmentation of ultrastructures in Electron Microscopy (EM) images.

Model Description

This model has been trained to accurately segment the following critical ultrastructural elements:

• Abnormalities
• Mitochondria (and mitochondria-like organelles)
• Periaxonal Space

Note: While the model architecture theoretically supports axon and myelin segmentation, this specific release is optimized and calibrated for the classes listed below due to dataset annotations.

Usage

You can use this model easily via the transformers library.

1. Using Pipeline (Recommended)

from transformers import pipeline
from PIL import Image

# Initialize the segmentation pipeline
pipe = pipeline("image-segmentation", model="Dnq2025/MicroStructFormer")

# Load your EM image
image = Image.open("examples/example_tem.png").convert("RGB")

# Run inference
results = pipe(image)
print(results)

2. Manual Inference with AutoImageProcessor

import torch
import numpy as np
from PIL import Image
from transformers import AutoImageProcessor, Mask2FormerForUniversalSegmentation

repo_id = "Dnq2025/MicroStructFormer"

# Load processor and model
processor = AutoImageProcessor.from_pretrained(repo_id)
model = Mask2FormerForUniversalSegmentation.from_pretrained(repo_id)
model.eval()

# Load image
image = Image.open("examples/example_tem.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

# Post-process to get the semantic segmentation map
pred = processor.post_process_semantic_segmentation(
    outputs,
    target_sizes=[image.size[::-1]]
)[0].cpu().numpy().astype(np.uint8)

print(f"Prediction shape: {pred.shape}")
print(f"Unique classes found: {np.unique(pred)}")

Label Map

ID	Class
0	background
1	abnormality
2	mitochondria-like organelles
3	mitochondria
4	mitochondria outside axon
5	periaxonal space

Example

Input EM image:

Predicted segmentation overlay:

Model Use, License, and Citation

This Hugging Face model repository is associated with the following bioRxiv preprint:

Deng N, Miao G, Khadra A, Peterson AC, Bagheri H. Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging. bioRxiv. 2026. DOI: 10.64898/2026.05.26.727755.

These model weights and associated architectures are provided for non-commercial academic and research use under the CC-BY-NC-SA 4.0 license selected for this Hugging Face repository. Users must provide appropriate attribution to the original authors and cite the associated preprint when using these weights, architectures, derived representations, or any models fine-tuned from this project in publications, preprints, theses, presentations, software repositories, benchmarks, or derivative analyses.

Use of this model for commercial purposes is not permitted without prior written permission from the authors. Any redistributed or modified version of these weights or derived model variants must preserve attribution to the original authors and must be shared under the same or a compatible non-commercial share-alike license, unless separate written permission is obtained from the authors.

Users must not claim ownership of the original architectures, pretrained weights, Hugging Face model versions, or associated derived resources.

Please cite:

@article {Deng2026.05.26.727755,
    author = {Deng, Nianqi and Miao, Guillaume and Khadra, Anmar and Peterson, Alan C and Bagheri, Hooman},
    title = {Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging},
    elocation-id = {2026.05.26.727755},
    year = {2026},
    doi = {10.64898/2026.05.26.727755},
    publisher = {Cold Spring Harbor Laboratory},
    URL = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755},
    eprint = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755.full.pdf},
    journal = {bioRxiv}
}