Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: mlx
|
| 4 |
+
tags:
|
| 5 |
+
- mlx
|
| 6 |
+
- face-restoration
|
| 7 |
+
- image-restoration
|
| 8 |
+
- vqgan
|
| 9 |
+
- restoreformer
|
| 10 |
+
base_model: wzhouxiff/RestoreFormerPlusPlus
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# RestoreFormerPlusPlus-fp32 (MLX)
|
| 14 |
+
|
| 15 |
+
[RestoreFormer++](https://github.com/wzhouxiff/RestoreFormerPlusPlus) (TPAMI 2023) blind
|
| 16 |
+
face restoration converted to MLX NHWC safetensors for Apple Silicon. 73,472,579
|
| 17 |
+
parameters (441 tensors), fp32.
|
| 18 |
+
|
| 19 |
+
- **Architecture:** `VQVAEGANMultiHeadTransformer` — VQ-GAN encoder/decoder over a
|
| 20 |
+
1024×256 ROHQD codebook, multi-scale multi-head cross-attention (enc attn @16, dec
|
| 21 |
+
@[16, 32]). 512×512 aligned face crops, RGB in [-1, 1]. Fully deterministic.
|
| 22 |
+
- **Source:** the author's official `RestoreFormer++.ckpt` (v1.0.0 GitHub release);
|
| 23 |
+
`vqvae.*` state re-exported through the instantiated architecture.
|
| 24 |
+
- **Layout:** MLX NHWC. Conv `(O,kH,kW,I)`; GroupNorm vectors, biases, and the codebook
|
| 25 |
+
embedding pass through. Keys mirror the upstream state dict (prefix stripped).
|
| 26 |
+
- **dtype:** fp32. Measured alternatives: fp16 50.1 dB vs the fp32 golden (viable), bf16
|
| 27 |
+
38.6 dB (mantissa-bound — fp16 beats bf16 here). Late-decoder activations reach ±14k.
|
| 28 |
+
|
| 29 |
+
## License
|
| 30 |
+
|
| 31 |
+
Apache-2.0 — upstream is plain Apache-2.0 with no third-party carve-outs. Trained on FFHQ
|
| 32 |
+
(dataset compilation CC-BY-NC-SA — the standard unsettled dataset-to-weights question).
|
| 33 |
+
|
| 34 |
+
## Consume
|
| 35 |
+
|
| 36 |
+
Swift (Apple Silicon): `mlx-restoreformer-swift` —
|
| 37 |
+
`RestoreFormerMLXCore.RestoreFormer` + `MLXRestoreFormer.RestoreFormerRestorePackage`
|
| 38 |
+
(MLXEngine `imageRestore`, Vision-based detect/align/paste).
|
| 39 |
+
|
| 40 |
+
Parity vs the PyTorch reference: key contract 441/441 tensors; per-stage taps 40/40
|
| 41 |
+
≤ 5e-4 relative (fp32, CPU stream); codebook indices 256/256 exact.
|