File size: 2,261 Bytes
09d2eec e3b802b 09d2eec e3b802b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
license: creativeml-openrail-m
library_name: diffusers
pipeline_tag: image-to-image
tags:
- infrared
- image-fusion
- adapter
- stable-diffusion
base_model: runwayml/stable-diffusion-inpainting
---
# IRecon MCM 128
Inference-only modulation and conditioning module for registered SWIR, MWIR, and LWIR fusion.
## Architecture
- 12 input channels: noisy latent, frozen base noise prediction, three bands, and object mask.
- 96 hidden channels, six time-conditioned residual blocks.
- 8 outputs: four noise residual channels and four spatial gates.
- 1,371,080 trainable parameters.
- Frozen Stable Diffusion 1.5 inpainting base, 10-step DDIM inference at 128 x 128.
## Training provenance
- Object-disjoint Objaverse simulation.
- Direct online-noise training from a randomly initialized MCM; no clean pretraining is contained in this checkpoint.
- Selected checkpoint: cumulative step 145,000, training seed 3409.
- Validation masked image MAE at selection: 0.1384654571.
- Loss: diffusion noise MSE + 0.1 image Smooth-L1 + 0.05 first-gradient loss.
- Online perturbation: band-dependent global noise and local spherical-region noise.
- Global noise sigma (SWIR/MWIR/LWIR): 0.08 / 0.025 / 0.01.
- Local spherical-region sigma (SWIR/MWIR/LWIR): 0.20 / 0.10 / 0.05.
The original checkpoint also contained optimizer and scaler state. The released file strips those states and stores only the MCM state dict and non-identifying inference metadata.
## Evaluation boundary
The thesis reports an MAE of 0.21288, PSNR of 13.9054 dB, and SSIM of 0.54345 for one validation-selected checkpoint averaged over four inference perturbation/sampling seeds. Independent training-seed results are reported separately. The model improves simulated intensity restoration over fixed linear fusion but does not yet improve the downstream GSO pose/mesh initializer over noisy SWIR. It has not been calibrated on a real three-band sensor.
## Required base model
Supply a legally obtained Stable Diffusion 1.5 inpainting Diffusers directory. This adapter does not include base-model weights.
## File integrity
SHA256: `573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739`
Repository: [github.com/ss00sxt/IRecon](https://github.com/ss00sxt/IRecon)
|