xocialize commited on
Commit
210447d
Β·
verified Β·
1 Parent(s): d5e796f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +85 -0
README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: mlx
3
+ license: mit
4
+ license_link: https://github.com/swz30/Restormer/blob/main/LICENSE.md
5
+ base_model: swz30/Restormer
6
+ pipeline_tag: image-to-image
7
+ tags:
8
+ - mlx
9
+ - image-restoration
10
+ - deblurring
11
+ - denoising
12
+ - restormer
13
+ ---
14
+
15
+ # mlx-community/Restormer-motion-deblurring-fp32
16
+
17
+ [Restormer](https://github.com/swz30/Restormer) β€” **Motion deblur** β€” converted to **Apple MLX** for
18
+ Apple-Silicon inference via the
19
+ [`mlx-restormer-swift`](https://github.com/xocialize/mlx-restormer-swift) Swift package.
20
+
21
+ Zamir et al., *Restormer: Efficient Transformer for High-Resolution Image Restoration*, **CVPR
22
+ 2022**. **26,126,644 parameters** (494 tensors, `LayerNorm_type=WithBias`).
23
+
24
+ Camera-shake and subject-motion deblur. GoPro 32.92.
25
+
26
+ > **Licence note.** Restormer is **plain MIT** with full commercial rights. The *same author's*
27
+ > MIRNet / MIRNetv2 / MPRNet / CycleISP carry an **Academic Public License** β€” easy to conflate,
28
+ > so it is worth stating explicitly.
29
+
30
+ ## Use with mlx-restormer-swift
31
+
32
+ ```swift
33
+ import RestormerMLXCore
34
+
35
+ var cfg = Restormer.Configuration()
36
+ cfg.normKind = .withBias
37
+ let model = Restormer(cfg)
38
+ try model.loadWeights(from: weightsURL)
39
+ let restored = model.restoreTiled(imageNHWC) // NHWC RGB in [0,1]
40
+ ```
41
+
42
+ Or as an MLXEngine `imageRestore` ModelPackage (`MLXRestormer.RestormerRestorePackage`), which
43
+ resolves this repo via the Hub.
44
+
45
+ ## ⚠️ Tile β€” do not run this full-frame
46
+
47
+ Measured on an M5 Max: a single 1080p frame full-frame costs **15.50 GB of MLX allocation / 48.02 GB
48
+ process footprint**. Tiled, the MLX peak is **flat at ~2.6 GB** from 512Β² to 1080p, because the peak
49
+ is one-tile-sized β€” a bigger image runs *more* tiles, not bigger ones.
50
+
51
+ **Tile geometry should be 8-aligned.** Three `pixelUnshuffle(2)` stages mean the 2Γ—2 grouping grid is
52
+ measured from the tile origin at strides 2, 4 and 8 full-resolution pixels, so an unaligned origin
53
+ shifts that grid out of phase with the full-frame decomposition.
54
+
55
+ On overlap, measured by **seam visibility** rather than PSNR: overlap 0 leaves a faint but real seam
56
+ (boundary gradient 1.31Γ— the interior), while every aligned overlap β‰₯ 8 measures clean (1.09–1.20Γ—).
57
+ Note PSNR-against-full-frame actually *prefers* overlap 0 β€” which is why it is the wrong metric for
58
+ a tiler.
59
+
60
+ ## Conversion
61
+
62
+ MLX **NHWC** layout: conv `(O,I,kH,kW) β†’ (O,kH,kW,I)` (186 standard + 88 depthwise), everything else
63
+ passthrough. The upstream `.pth` nests its state dict under `raw['params']` (BasicSR convention).
64
+
65
+ ## Parity
66
+
67
+ Gated against the PyTorch oracle on the CPU stream, fp32, judged on relative error:
68
+
69
+ - **key contract** β€” 494 tensors / 26,126,644 params / 0 missing / 0 unused, strict load
70
+ - **primitives** β€” both LayerNorm variants, and the pixel-shuffle pair (worst 2.6e-06)
71
+ - **blocks** β€” MDTA, GDFN, TransformerBlock (worst 2.6e-07)
72
+ - **full model** β€” cosine **1.00000000** at 64Β² / 128Β² / 256Β² (worst 1.0e-06)
73
+
74
+ The pixel-shuffle gate also runs the *wrong* channel ordering β€” the `(r,r,C)` split that compiles
75
+ and silently scrambles the image β€” and measures it at **493,451Γ— the observed rounding**, so the
76
+ tolerance keeps ~5 orders of discrimination.
77
+
78
+ ## Honest limitation
79
+
80
+ Restormer's attention is spatially **global**, so it is weak on *spatially varying* blur. Benchmark
81
+ rank is also a weak predictor of real-world value: *"Deblurring in the Wild"* (2026) found every
82
+ GoPro-trained method scored below the blurry input on real smartphone blur. Validate on your own
83
+ photographs.
84
+
85
+ Weights: MIT (swz30/Restormer, first-party GitHub release v1.0). Port code: MIT.