BiliSakura/CROMA-transformers

CROMA Transformers Models

Self-contained HuggingFace model checkpoints for CROMA. Each subfolder ships its own model, image processor, and custom pipeline code — no external croma package required.

CROMA expects dual-modality Sentinel-1 (2-channel SAR) and Sentinel-2 (12-channel optical) inputs at 120×120 px by default. Per-channel mean ± 2σ normalization is applied by the bundled image processor.

Available checkpoints

Folder	Hidden size	Optical layers	Heads
croma-base-patch8-120/	768	12	16
croma-large-patch8-120/	1024	24	16

Usage

Processors default to do_resize: false. CROMA expects SAR (2, H, W) and optical (12, H, W) at the same native spatial size (120×120 at pretraining; larger sizes work without forced resize).

from transformers import pipeline

MODEL = "/path/to/CROMA-transformers/croma-base-patch8-120"

pipe = pipeline(
    task="croma-feature-extraction",
    model=MODEL,
    trust_remote_code=True,
)

import numpy as np

sar = np.random.randn(2, 256, 256).astype(np.float32)
optical = np.random.randn(12, 256, 256).astype(np.float32)

joint_gap = pipe(
    sar_images=sar,
    optical_images=optical,
    use_8_bit=True,
    pool=True,
    return_tensors=True,
)

sar_tokens = pipe(
    sar_images=sar,
    output_mode="sar",
    pool=False,
    return_tensors=True,
)

Opt in to 120×120 resize:

joint_gap = pipe(
    sar_images=sar,
    optical_images=optical,
    use_8_bit=True,
    pool=True,
    return_tensors=True,
    image_processor_kwargs={"do_resize": True},
)

Or load components directly:

from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained(MODEL, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(MODEL, trust_remote_code=True)

batch = processor(sar_images=sar, optical_images=optical, return_tensors="pt")
outputs = model(**batch)
print(outputs.pooler_output.shape)  # joint GAP

Dependencies

• transformers
• torch
• einops
• numpy
• opencv-python (only when resizing non-RGB inputs)

Rebuild from raw checkpoints

conda activate rsgen
python /path/to/projects/CROMA/scripts/build_transformers_repos.py