# MMDiff: Multi-modal Remote Sensing Image Generation via Cross-Modality Spatial Feature Transfer

ISPRS 2026 🔥

**Haojun Tang**¹ · **Wenda Zhao**^1,* · **Hengshuai Cui**¹ · **Haipeng Wang**² ¹ Dalian University of Technology ² Unit 92728 of PLA

[![Website](https://img.shields.io/badge/Homepage-Website-87CEEB)](#) [![ISPRS](https://img.shields.io/badge/ISPRS-Paper-2563EB)](#) [![HuggingFace](https://img.shields.io/badge/HuggingFace-Model-F9D371)](https://huggingface.co/XinRan-Tang/MM-Diff) [![Dataset](https://img.shields.io/badge/Dataset-Access-green)](https://huggingface.co/datasets/XinRan-Tang/Optical-SAR-Infrared)

^* **Corresponding author:**

## Abstract

Collecting spatially consistent multi-modal remote sensing (MMRS) images remains challenging due to different sensors vary in the imaging principles and acquisition times. This hinders the development of data-driven MMRS technologies, which rely on large-scale training samples. This paper proposes MMDiff, the first text-driven diffusion framework explicitly designed for jointly generating structurally consistent optical (OPT), synthetic aperture radar (SAR), and infrared (IR) remote sensing images from a single text prompt via cross-modality spatial feature transfer. MMDiff first trains the OPT branch with paired optical image-text data to capture rich semantic content, and then trains the SAR/IR branches with simple modality-specific text templates to learn the corresponding style attributes, without relying on complex linguistic descriptions. Specifically, we introduce a LoRA-based modality translation adaptation mechanism to translate the style attributes of optical spatial representations to SAR and IR style attributes while preserving the underlying semantic content. The translated representations are then transferred into the SAR and IR generation branches through the proposed spatial feature transfer mechanism, enabling rich spatial details in the generated SAR/IR images while maintaining cross-modal spatial consistency. Extensive experiments demonstrate that MMDiff achieves superior image quality in terms of modality similarity and semantic consistency compared to the state-of-the-art methods. Furthermore, MMDiff benefits downstream data-driven MMRS applications, e.g., multi-modal image fusion and object classification.

Generated multi-modal remote sensing images

From top to bottom: optical (OPT), synthetic aperture radar (SAR), and infrared (IR) images.

## Model Overview This directory contains the MMDiff model weights for jointly generating spatially consistent optical (OPT), synthetic aperture radar (SAR), and infrared (IR) remote sensing images from text prompts. ## Training Data These weights were trained on the [Optical-SAR-Infrared dataset](https://huggingface.co/datasets/XinRan-Tang/Optical-SAR-Infrared), which provides modality-specific image-text pairs for OPT, SAR, and IR remote sensing imagery. ## Available Checkpoints ```text Optical-SAR-Infrared/ ├── mmdiff-opt/ # Optical image generation branch ├── mmdiff-sar/ # SAR image generation branch └── mmdiff-ir/ # Infrared image generation branch ``` The three branches are used together by the MMDiff sampling pipeline. Refer to the main [MMDiff repository](https://github.com/XinR-Tang/MMDiff) for installation, sampling commands, and configuration options. ## Loading the Weights Pass the model directory to the sampling configuration or command line. For example: ```bash python mmdiff/sampling_OPT.py \ --model_path /path/to/MMDiff/models/Optical-SAR-Infrared/mmdiff-opt ``` The SAR and IR branches use their corresponding checkpoint directories during the complete OPT → SAR → IR sampling workflow. ## Intended Use These checkpoints are intended for research on text-driven multi-modal remote sensing image generation, cross-modality spatial feature transfer, multi-modal image fusion, and downstream remote sensing applications.