metadata
license: mit
tags:
- biology
- microscopy
- instance-segmentation
- condensate-detection
- biomolecular-condensates
- fluorescence-microscopy
- efficientnet
- feature-pyramid-network
datasets:
- nimbusimage
library_name: pytorch
pipeline_tag: image-segmentation
CondensatePose
CondensatePose is a deep learning model for detecting and segmenting biomolecular condensates in fluorescence microscopy images.
Model Description
This model uses EfficientNetV2 as the encoder with a sophisticated Feature Pyramid Network decoder that includes style-based feature modulation and dual residual blocks for accurate condensate detection.
Architecture Highlights
- Encoder: EfficientNetV2-RW-S (pretrained, adapted for grayscale microscopy)
- Decoder: Multi-scale Feature Pyramid Network with:
- Style-based feature modulation for adaptive processing
- Dual residual blocks for deep feature fusion
- Multi-scale upsampling refinement
- Spatial attention for focusing on sparse objects
- Output: Binary mask logits + flow field vectors
- Parameters: ~18M trainable parameters
Key Features
- Style Modulation: Adaptive feature processing based on global image context
- Multi-Scale Processing: Captures condensates at various scales
- Spatial Attention: Focuses on sparse condensate regions
- Flow Fields: Cellpose-style flow vectors for instance segmentation
- Single-Channel: Works on grayscale fluorescence images
- Variable Size: Handles different image dimensions
Training Data
Trained on annotated fluorescence microscopy images from the Nimbus Image platform, containing manually annotated biomolecular condensates with diverse:
- Morphologies (spherical to irregular)
- Sizes (10-500 pixels in diameter)
- Intensities (bright to dim)
- Densities (sparse to crowded fields)
Intended Use
Primary Applications
- Detecting biomolecular condensates in live-cell imaging
- Segmenting phase-separated compartments
- Tracking condensate dynamics over time
- Quantifying condensate properties (size, number, morphology, intensity)
How to Use
Installation
pip install torch timm huggingface-hub
Quick Start
from huggingface_hub import hf_hub_download
import torch
import numpy as np
from model import load_condensatepose_model
# Download and load model
model_path = hf_hub_download(
repo_id="rajlab/condensatepose-model",
filename="model_weights.pth"
)
model = load_condensatepose_model(model_path, device='cuda')
model.eval()
# Prepare your image (H, W) grayscale
image = ... # Your microscopy image as numpy array
# Normalize
image = image.astype(np.float32)
if image.max() > 1:
image = image / 255.0
# Convert to tensor
img_tensor = torch.from_numpy(image).unsqueeze(0).unsqueeze(0) # (1, 1, H, W)
img_tensor = img_tensor.to('cuda')
# Run inference
with torch.no_grad():
outputs = model(img_tensor)
# Get outputs
mask_logits = outputs['mask'][0, 0].cpu().numpy() # (H, W)
flow_vectors = outputs['flows'][0].cpu().numpy() # (2, H, W)
# Apply sigmoid to get probabilities
mask_prob = 1 / (1 + np.exp(-mask_logits))
# Threshold to get binary mask
binary_mask = (mask_prob > 0.5).astype(np.uint8)
# Generate instance segmentation
from scipy.ndimage import label
instance_mask, num_objects = label(binary_mask)
print(f"Detected {num_objects} condensates")
Model Configuration
{
"model_type": "condensatepose",
"architecture": "efficientnetv2-fpn-style-modulation",
"task": "instance-segmentation",
"application": "biomolecular-condensate-detection",
"framework": "pytorch",
"encoder_variant": "rw_s",
"encoder_backbone": "efficientnetv2",
"pyramid_channels": [
24,
48,
64,
160
],
"pyramid_dim": 32,
"use_spatial_attention": true,
"spatial_kernel_size": 11,
"dropout_rate": 0.15,
"input_channels": 1,
"output_channels": 3,
"patch_size": 256,
"trained_on": "nimbus_image_data",
"microscopy_type": "fluorescence",
"target": "biomolecular_condensates",
"style_modulation": true,
"dual_residual_blocks": true,
"multi_scale_upsampling": true
}