Upload folder using huggingface_hub
Browse files- README.md +108 -0
- drunet_color.onnx +3 -0
- export_drunet_onnx.py +28 -0
README.md
CHANGED
|
@@ -1,3 +1,111 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: onnx
|
| 4 |
+
tags:
|
| 5 |
+
- image-denoising
|
| 6 |
+
- onnx
|
| 7 |
+
- drunet
|
| 8 |
+
- raw-photography
|
| 9 |
---
|
| 10 |
+
|
| 11 |
+
# DRUNet (colour) — ONNX export
|
| 12 |
+
|
| 13 |
+
An ONNX export of **DRUNet**, the plug-and-play denoiser from Kai Zhang's
|
| 14 |
+
[KAIR](https://github.com/cszn/KAIR) toolbox. Nothing here is retrained or modified: this is the
|
| 15 |
+
original `drunet_color.pth` weights, exported to ONNX so they can be run from ONNX Runtime.
|
| 16 |
+
|
| 17 |
+
Published for use in [PxlMonk](https://pxlmonk.com), a desktop raw photo editor, whose build fetches
|
| 18 |
+
this file at build time rather than committing a 130 MB binary.
|
| 19 |
+
|
| 20 |
+
## Provenance
|
| 21 |
+
|
| 22 |
+
| | |
|
| 23 |
+
|---|---|
|
| 24 |
+
| Architecture | `UNetRes` from [cszn/KAIR](https://github.com/cszn/KAIR) — MIT |
|
| 25 |
+
| Weights | `drunet_color.pth`, from the KAIR v1.0 release — unmodified |
|
| 26 |
+
| Export | `torch.onnx.export`, opset 17, dynamic batch/height/width |
|
| 27 |
+
| Paper | Zhang et al., *Plug-and-Play Image Restoration with Deep Denoiser Prior*, TPAMI 2021 |
|
| 28 |
+
|
| 29 |
+
The weights were trained by the original author and are redistributed unchanged, in a different file
|
| 30 |
+
format. No training data is redistributed here.
|
| 31 |
+
|
| 32 |
+
### A note on the training corpora
|
| 33 |
+
|
| 34 |
+
DRUNet was trained on `train400`, **DIV2K**, **Flickr2K** and the Waterloo Exploration Database. Some of
|
| 35 |
+
those carry their own terms — DIV2K states:
|
| 36 |
+
|
| 37 |
+
> This dataset is made available for academic research purpose only. All the images are collected from
|
| 38 |
+
> the Internet, and the copyright belongs to the original owners.
|
| 39 |
+
|
| 40 |
+
Whether such dataset terms reach the resulting model weights is unsettled, and this repository takes no
|
| 41 |
+
position on it. What it does do is state the provenance plainly, so anyone using these weights can make
|
| 42 |
+
that call with the facts in front of them rather than discovering them later. The MIT tag above reflects
|
| 43 |
+
the licence under which the original author published the weights.
|
| 44 |
+
|
| 45 |
+
## Interface
|
| 46 |
+
|
| 47 |
+
Input `input`: `float32[batch, 4, height, width]`, NCHW.
|
| 48 |
+
|
| 49 |
+
- Channels 0–2: RGB, **sRGB display-referred**, range 0…1. The model was trained on display images —
|
| 50 |
+
feeding scene-linear data gives poor results.
|
| 51 |
+
- Channel 3: the **noise level map**, a constant `sigma / 255` across the plane. This is DRUNet's
|
| 52 |
+
strength control: sigma is in 0…255 units, so `sigma = 20` means a plane filled with `20/255`.
|
| 53 |
+
|
| 54 |
+
Output `output`: `float32[batch, 3, height, width]`, same convention, denoised.
|
| 55 |
+
|
| 56 |
+
Height and width must be divisible by 8 (three downsampling stages). Pad or tile accordingly.
|
| 57 |
+
|
| 58 |
+
## Choosing sigma
|
| 59 |
+
|
| 60 |
+
There is no single right value — it is the strength dial. Measured on a 24 MP frame shot at ISO 16000,
|
| 61 |
+
using mean deviation from the local 3×3 mean as a noise figure (lower is cleaner):
|
| 62 |
+
|
| 63 |
+
| sigma | noise figure | |
|
| 64 |
+
|---|---|---|
|
| 65 |
+
| — (untouched) | 7.41 | |
|
| 66 |
+
| 10 | 1.67 | visible noise remains |
|
| 67 |
+
| **20** | **0.23** | clean, fine detail intact |
|
| 68 |
+
| 35 | 0.17 | over-smoothed — eyelashes go waxy |
|
| 69 |
+
|
| 70 |
+
## Reproducing this export
|
| 71 |
+
|
| 72 |
+
The weights are stored **without bias terms**, so the network must be constructed with `bias=False`.
|
| 73 |
+
Loading with `strict=False` silently leaves 64 randomly-initialised bias tensors in place and produces
|
| 74 |
+
a model that runs but denoises badly — always load strictly.
|
| 75 |
+
|
| 76 |
+
```python
|
| 77 |
+
from models.network_unet import UNetRes # from cszn/KAIR
|
| 78 |
+
import torch
|
| 79 |
+
|
| 80 |
+
net = UNetRes(in_nc=4, out_nc=3, nc=[64, 128, 256, 512], nb=4, act_mode="R",
|
| 81 |
+
downsample_mode="strideconv", upsample_mode="convtranspose", bias=False)
|
| 82 |
+
net.load_state_dict(torch.load("drunet_color.pth", map_location="cpu", weights_only=True), strict=True)
|
| 83 |
+
net.eval()
|
| 84 |
+
|
| 85 |
+
torch.onnx.export(
|
| 86 |
+
net, torch.randn(1, 4, 256, 256), "drunet_color.onnx",
|
| 87 |
+
input_names=["input"], output_names=["output"],
|
| 88 |
+
dynamic_axes={"input": {0: "b", 2: "h", 3: "w"}, "output": {0: "b", 2: "h", 3: "w"}},
|
| 89 |
+
opset_version=17,
|
| 90 |
+
)
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
## Execution providers
|
| 94 |
+
|
| 95 |
+
Measured on Apple Silicon (10 cores), 256×256 tile, best of three after a warm-up:
|
| 96 |
+
|
| 97 |
+
| | |
|
| 98 |
+
|---|---|
|
| 99 |
+
| CPU | 560 ms |
|
| 100 |
+
| CoreML | **117 ms** |
|
| 101 |
+
| WebGPU | 154 ms |
|
| 102 |
+
|
| 103 |
+
DRUNet is a plain CNN, so it maps cleanly onto accelerators — unlike transformer-based denoisers, where
|
| 104 |
+
the graph fragments into hundreds of partitions and the copying between them costs more than the
|
| 105 |
+
compute saves.
|
| 106 |
+
|
| 107 |
+
## Licence
|
| 108 |
+
|
| 109 |
+
MIT, matching KAIR. Include the original copyright notice when redistributing:
|
| 110 |
+
|
| 111 |
+
> Copyright (c) 2019 Kai Zhang
|
drunet_color.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ae3ab5eb15daac2ee79be984d584b908ce7f0f60b27be87d005f728c2aa0087
|
| 3 |
+
size 130589232
|
export_drunet_onnx.py
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DRUNet (KAIR, MIT) nach ONNX. Die Gewichte sind cszns Original; Architektur aus demselben Repo,
|
| 2 |
+
# damit die Schlüssel ohne Umbenennen passen.
|
| 3 |
+
import sys, torch
|
| 4 |
+
sys.path.insert(0, "/private/tmp/claude-501/-Users-ckret-wkspaces-pxlmonk/35a64bed-d9bb-4a13-9d30-2c08f92e2fd2/scratchpad/kair")
|
| 5 |
+
from models.network_unet import UNetRes
|
| 6 |
+
|
| 7 |
+
SP = "/private/tmp/claude-501/-Users-ckret-wkspaces-pxlmonk/35a64bed-d9bb-4a13-9d30-2c08f92e2fd2/scratchpad"
|
| 8 |
+
net = UNetRes(in_nc=4, out_nc=3, nc=[64,128,256,512], nb=4, act_mode='R',
|
| 9 |
+
downsample_mode="strideconv", upsample_mode="convtranspose", bias=False)
|
| 10 |
+
sd = torch.load(f"{SP}/kair/drunet_color.pth", map_location="cpu", weights_only=True)
|
| 11 |
+
net.load_state_dict(sd, strict=True) # strict: jede Abweichung ist ein Fehler, kein Zufallsgewicht
|
| 12 |
+
missing, unexpected = [], []
|
| 13 |
+
print("fehlend:", len(missing), " unerwartet:", len(unexpected))
|
| 14 |
+
if missing or unexpected:
|
| 15 |
+
print(" ", missing[:3], unexpected[:3])
|
| 16 |
+
net.eval()
|
| 17 |
+
|
| 18 |
+
# 4. Kanal ist die Rauschpegelkarte: konstant sigma/255 über das ganze Bild.
|
| 19 |
+
dummy = torch.randn(1, 4, 256, 256)
|
| 20 |
+
out = f"{SP}/models/drunet_color.onnx"
|
| 21 |
+
torch.onnx.export(
|
| 22 |
+
net, dummy, out,
|
| 23 |
+
input_names=["input"], output_names=["output"],
|
| 24 |
+
dynamic_axes={"input": {0: "b", 2: "h", 3: "w"}, "output": {0: "b", 2: "h", 3: "w"}},
|
| 25 |
+
opset_version=17,
|
| 26 |
+
)
|
| 27 |
+
import os
|
| 28 |
+
print(f"geschrieben: {out} ({os.path.getsize(out)/1e6:.1f} MB)")
|