---
license: creativeml-openrail-m
library_name: diffusers
pipeline_tag: image-to-image
tags:
  - infrared
  - image-fusion
  - adapter
  - stable-diffusion
base_model: runwayml/stable-diffusion-inpainting
---

# IRecon MCM 128

Inference-only modulation and conditioning module for registered SWIR, MWIR, and LWIR fusion.

## Architecture

- 12 input channels: noisy latent, frozen base noise prediction, three bands, and object mask.
- 96 hidden channels, six time-conditioned residual blocks.
- 8 outputs: four noise residual channels and four spatial gates.
- 1,371,080 trainable parameters.
- Frozen Stable Diffusion 1.5 inpainting base, 10-step DDIM inference at 128 x 128.

## Training provenance

- Object-disjoint Objaverse simulation.
- Direct online-noise training from a randomly initialized MCM; no clean pretraining is contained in this checkpoint.
- Selected checkpoint: cumulative step 145,000, training seed 3409.
- Validation masked image MAE at selection: 0.1384654571.
- Loss: diffusion noise MSE + 0.1 image Smooth-L1 + 0.05 first-gradient loss.
- Online perturbation: band-dependent global noise and local spherical-region noise.
- Global noise sigma (SWIR/MWIR/LWIR): 0.08 / 0.025 / 0.01.
- Local spherical-region sigma (SWIR/MWIR/LWIR): 0.20 / 0.10 / 0.05.

The original checkpoint also contained optimizer and scaler state. The released file strips those states and stores only the MCM state dict and non-identifying inference metadata.

## Evaluation boundary

The thesis reports an MAE of 0.21288, PSNR of 13.9054 dB, and SSIM of 0.54345 for one validation-selected checkpoint averaged over four inference perturbation/sampling seeds. Independent training-seed results are reported separately. The model improves simulated intensity restoration over fixed linear fusion but does not yet improve the downstream GSO pose/mesh initializer over noisy SWIR. It has not been calibrated on a real three-band sensor.

## Required base model

Supply a legally obtained Stable Diffusion 1.5 inpainting Diffusers directory. This adapter does not include base-model weights.

## File integrity

SHA256: `573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739`

Repository: [github.com/ss00sxt/IRecon](https://github.com/ss00sxt/IRecon)