Publish Ommatidium v0.1.0 model card
Browse files- README.md +83 -0
- config.ron +19 -0
- manifest.json +44 -0
README.md
CHANGED
|
@@ -1,3 +1,86 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: meganeura
|
| 4 |
+
pipeline_tag: image-to-image
|
| 5 |
+
datasets:
|
| 6 |
+
- mad-bot/ommatidia
|
| 7 |
+
tags:
|
| 8 |
+
- ray-tracing
|
| 9 |
+
- denoising
|
| 10 |
+
- upscaling
|
| 11 |
+
- vulkan
|
| 12 |
+
- metal
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
# Ommatidium
|
| 16 |
+
|
| 17 |
+
Ommatidium is a portable neural denoiser and 2× upscaler for real-time ray
|
| 18 |
+
tracing. This first checkpoint replaces Blade's variance-guided SVGF pass: it
|
| 19 |
+
accepts raw ReSTIR radiance and the renderer's depth, normal, diffuse albedo,
|
| 20 |
+
specular F0, and roughness buffers, then reconstructs a high-resolution frame
|
| 21 |
+
through [Meganeura](https://github.com/kvark/meganeura).
|
| 22 |
+
|
| 23 |
+
- **Source and integration:** https://github.com/kvark/ommatidia
|
| 24 |
+
- **Training and validation data:** https://huggingface.co/datasets/mad-bot/ommatidia
|
| 25 |
+
- **Runtime:** Meganeura on a graphics context shared with Blade
|
| 26 |
+
- **Backends:** Vulkan and Metal; the published performance result is Vulkan
|
| 27 |
+
|
| 28 |
+
## Checkpoint
|
| 29 |
+
|
| 30 |
+
The release contains `model.safetensors`, `config.ron`, and a machine-readable
|
| 31 |
+
`manifest.json`. The configuration is a direct, single-frame base-24 U-Net with
|
| 32 |
+
three resolution levels, one residual block per level, 649,200 parameters, and
|
| 33 |
+
a 2× output scale. It has no temporal history yet.
|
| 34 |
+
|
| 35 |
+
The input contract is six low-resolution plane groups in this order:
|
| 36 |
+
|
| 37 |
+
1. RGB radiance
|
| 38 |
+
2. depth
|
| 39 |
+
3. world-space normal
|
| 40 |
+
4. diffuse albedo
|
| 41 |
+
5. specular F0
|
| 42 |
+
6. roughness
|
| 43 |
+
|
| 44 |
+
Use the Ommatidium loader rather than constructing the tensor manually: it
|
| 45 |
+
applies the checkpoint's stored radiance compression and residual gain and
|
| 46 |
+
keeps the prediction on the shared GPU.
|
| 47 |
+
|
| 48 |
+
## Results
|
| 49 |
+
|
| 50 |
+
On the matched 128-scene validation capture:
|
| 51 |
+
|
| 52 |
+
| Reconstruction | Error |
|
| 53 |
+
|---|---:|
|
| 54 |
+
| Raw ReSTIR, nearest 2× | 0.005748 |
|
| 55 |
+
| Blade SVGF, nearest 2× | 0.004284 |
|
| 56 |
+
| Ommatidium from raw ReSTIR | **0.002876** |
|
| 57 |
+
|
| 58 |
+
That is 3.01 dB over raw nearest reconstruction and 1.73 dB over the Blade
|
| 59 |
+
filter it replaces. The canonical references are byte-identical between the
|
| 60 |
+
raw and SVGF captures.
|
| 61 |
+
|
| 62 |
+
The 104.1 GFLOP backbone measures **19.4 ms** for 720×720 input / 1440×1440
|
| 63 |
+
output—the same output pixel count as 1080p—on an idle Radeon RX 7900 XT with
|
| 64 |
+
RADV and Mesa 26.0.3. This is the Meganeura network step; Blade texture packing,
|
| 65 |
+
unpacking, and display post-processing are not included.
|
| 66 |
+
|
| 67 |
+
## Intended use and limitations
|
| 68 |
+
|
| 69 |
+
This is an early research checkpoint intended for Blade integration and for
|
| 70 |
+
developing portable neural reconstruction runtimes. It was trained entirely
|
| 71 |
+
on procedural Blade scenes at 128×128 input and 256×256 reference resolution.
|
| 72 |
+
It may fail on geometry, materials, lighting distributions, resolutions, or
|
| 73 |
+
renderers outside that training distribution. It is spatial-only, so it cannot
|
| 74 |
+
recover information from prior frames or guarantee temporal stability.
|
| 75 |
+
|
| 76 |
+
Pin the `v0.1.0` Hub revision, or its exact commit, in applications. Do not
|
| 77 |
+
download mutable `main` for a shipped build.
|
| 78 |
+
|
| 79 |
+
## Revisions
|
| 80 |
+
|
| 81 |
+
- Ommatidium: `7f08f025a3150c355643af513bc2825d88441520`
|
| 82 |
+
- Meganeura upstream integration: `256b906`
|
| 83 |
+
- Blade upstream integration: `3a8895a`
|
| 84 |
+
- Dataset: `mad-bot/ommatidia@v0.1.0`
|
| 85 |
+
|
| 86 |
+
The weights are released under the MIT license.
|
config.ron
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
(
|
| 2 |
+
scale: 2,
|
| 3 |
+
tile: 64,
|
| 4 |
+
batch: 8,
|
| 5 |
+
cond_planes: (63),
|
| 6 |
+
base_channels: 24,
|
| 7 |
+
level_multipliers: [
|
| 8 |
+
1,
|
| 9 |
+
2,
|
| 10 |
+
4,
|
| 11 |
+
],
|
| 12 |
+
blocks_per_level: 1,
|
| 13 |
+
num_groups: 8,
|
| 14 |
+
gn_eps: 0.00001,
|
| 15 |
+
time_input_dim: 64,
|
| 16 |
+
time_embed_dim: 256,
|
| 17 |
+
residual_gain: 12.969844,
|
| 18 |
+
objective: Direct,
|
| 19 |
+
)
|
manifest.json
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema_version": 1,
|
| 3 |
+
"project": "ommatidia",
|
| 4 |
+
"variant": "b24-2x-spatial-v0",
|
| 5 |
+
"objective": "direct",
|
| 6 |
+
"scale_factor": 2,
|
| 7 |
+
"parameters": 649200,
|
| 8 |
+
"input_contract": {
|
| 9 |
+
"name": "ommatidia-render-planes-v1",
|
| 10 |
+
"planes": [
|
| 11 |
+
"color",
|
| 12 |
+
"depth",
|
| 13 |
+
"normal",
|
| 14 |
+
"diffuse_albedo",
|
| 15 |
+
"specular_f0",
|
| 16 |
+
"roughness"
|
| 17 |
+
]
|
| 18 |
+
},
|
| 19 |
+
"files": {
|
| 20 |
+
"weights": {
|
| 21 |
+
"path": "model.safetensors",
|
| 22 |
+
"bytes": 7804448,
|
| 23 |
+
"sha256": "5272c28bd13e15cfcf67d0aed6b366362161522c880ea6ed62c09dba19379e9d"
|
| 24 |
+
},
|
| 25 |
+
"config": {
|
| 26 |
+
"path": "config.ron",
|
| 27 |
+
"bytes": 323,
|
| 28 |
+
"sha256": "247a0d40a72ca14d83d57bcb31332c2639e370080d328242c9b0e6918c09fef6"
|
| 29 |
+
}
|
| 30 |
+
},
|
| 31 |
+
"dataset": {
|
| 32 |
+
"repository": "mad-bot/ommatidia",
|
| 33 |
+
"revision": "v0.1.0",
|
| 34 |
+
"source": "blade-restir"
|
| 35 |
+
},
|
| 36 |
+
"source": {
|
| 37 |
+
"repository": "https://github.com/kvark/ommatidia",
|
| 38 |
+
"revision": "7f08f025a3150c355643af513bc2825d88441520"
|
| 39 |
+
},
|
| 40 |
+
"compatibility": {
|
| 41 |
+
"meganeura_revision": "256b906",
|
| 42 |
+
"blade_revision": "3a8895a"
|
| 43 |
+
}
|
| 44 |
+
}
|