| --- |
| license: creativeml-openrail-m |
| library_name: diffusers |
| pipeline_tag: image-to-image |
| tags: |
| - infrared |
| - image-fusion |
| - adapter |
| - stable-diffusion |
| base_model: runwayml/stable-diffusion-inpainting |
| --- |
| |
| # IRecon MCM 128 |
|
|
| Inference-only modulation and conditioning module for registered SWIR, MWIR, and LWIR fusion. |
|
|
| ## Architecture |
|
|
| - 12 input channels: noisy latent, frozen base noise prediction, three bands, and object mask. |
| - 96 hidden channels, six time-conditioned residual blocks. |
| - 8 outputs: four noise residual channels and four spatial gates. |
| - 1,371,080 trainable parameters. |
| - Frozen Stable Diffusion 1.5 inpainting base, 10-step DDIM inference at 128 x 128. |
|
|
| ## Training provenance |
|
|
| - Object-disjoint Objaverse simulation. |
| - Direct online-noise training from a randomly initialized MCM; no clean pretraining is contained in this checkpoint. |
| - Selected checkpoint: cumulative step 145,000, training seed 3409. |
| - Validation masked image MAE at selection: 0.1384654571. |
| - Loss: diffusion noise MSE + 0.1 image Smooth-L1 + 0.05 first-gradient loss. |
| - Online perturbation: band-dependent global noise and local spherical-region noise. |
| - Global noise sigma (SWIR/MWIR/LWIR): 0.08 / 0.025 / 0.01. |
| - Local spherical-region sigma (SWIR/MWIR/LWIR): 0.20 / 0.10 / 0.05. |
|
|
| The original checkpoint also contained optimizer and scaler state. The released file strips those states and stores only the MCM state dict and non-identifying inference metadata. |
|
|
| ## Evaluation boundary |
|
|
| The thesis reports an MAE of 0.21288, PSNR of 13.9054 dB, and SSIM of 0.54345 for one validation-selected checkpoint averaged over four inference perturbation/sampling seeds. Independent training-seed results are reported separately. The model improves simulated intensity restoration over fixed linear fusion but does not yet improve the downstream GSO pose/mesh initializer over noisy SWIR. It has not been calibrated on a real three-band sensor. |
|
|
| ## Required base model |
|
|
| Supply a legally obtained Stable Diffusion 1.5 inpainting Diffusers directory. This adapter does not include base-model weights. |
|
|
| ## File integrity |
|
|
| SHA256: `573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739` |
|
|
| Repository: [github.com/ss00sxt/IRecon](https://github.com/ss00sxt/IRecon) |
|
|
