Sunmj commited on
Commit
e3b802b
·
verified ·
1 Parent(s): 09d2eec

Upload IRecon MCM inference adapter

Browse files
Files changed (3) hide show
  1. README.md +47 -0
  2. mcm_fusion_128.pt +3 -0
  3. mcm_fusion_128.sha256 +1 -0
README.md CHANGED
@@ -1,3 +1,50 @@
1
  ---
2
  license: creativeml-openrail-m
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: creativeml-openrail-m
3
+ library_name: diffusers
4
+ pipeline_tag: image-to-image
5
+ tags:
6
+ - infrared
7
+ - image-fusion
8
+ - adapter
9
+ - stable-diffusion
10
+ base_model: runwayml/stable-diffusion-inpainting
11
  ---
12
+
13
+ # IRecon MCM 128
14
+
15
+ Inference-only modulation and conditioning module for registered SWIR, MWIR, and LWIR fusion.
16
+
17
+ ## Architecture
18
+
19
+ - 12 input channels: noisy latent, frozen base noise prediction, three bands, and object mask.
20
+ - 96 hidden channels, six time-conditioned residual blocks.
21
+ - 8 outputs: four noise residual channels and four spatial gates.
22
+ - 1,371,080 trainable parameters.
23
+ - Frozen Stable Diffusion 1.5 inpainting base, 10-step DDIM inference at 128 x 128.
24
+
25
+ ## Training provenance
26
+
27
+ - Object-disjoint Objaverse simulation.
28
+ - Direct online-noise training from a randomly initialized MCM; no clean pretraining is contained in this checkpoint.
29
+ - Selected checkpoint: cumulative step 145,000, training seed 3409.
30
+ - Validation masked image MAE at selection: 0.1384654571.
31
+ - Loss: diffusion noise MSE + 0.1 image Smooth-L1 + 0.05 first-gradient loss.
32
+ - Online perturbation: band-dependent global noise and local spherical-region noise.
33
+ - Global noise sigma (SWIR/MWIR/LWIR): 0.08 / 0.025 / 0.01.
34
+ - Local spherical-region sigma (SWIR/MWIR/LWIR): 0.20 / 0.10 / 0.05.
35
+
36
+ The original checkpoint also contained optimizer and scaler state. The released file strips those states and stores only the MCM state dict and non-identifying inference metadata.
37
+
38
+ ## Evaluation boundary
39
+
40
+ The thesis reports an MAE of 0.21288, PSNR of 13.9054 dB, and SSIM of 0.54345 for one validation-selected checkpoint averaged over four inference perturbation/sampling seeds. Independent training-seed results are reported separately. The model improves simulated intensity restoration over fixed linear fusion but does not yet improve the downstream GSO pose/mesh initializer over noisy SWIR. It has not been calibrated on a real three-band sensor.
41
+
42
+ ## Required base model
43
+
44
+ Supply a legally obtained Stable Diffusion 1.5 inpainting Diffusers directory. This adapter does not include base-model weights.
45
+
46
+ ## File integrity
47
+
48
+ SHA256: `573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739`
49
+
50
+ Repository: [github.com/ss00sxt/IRecon](https://github.com/ss00sxt/IRecon)
mcm_fusion_128.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739
3
+ size 5507658
mcm_fusion_128.sha256 ADDED
@@ -0,0 +1 @@
 
 
1
+ 573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739 mcm_fusion_128.pt