| ---
|
| license: mit
|
| library_name: onnxruntime
|
| pipeline_tag: image-to-image
|
| tags:
|
| - super-resolution
|
| - upscaling
|
| - onnx
|
| - real-time
|
| - nrr
|
| ---
|
|
|
| # NRR upscalers - the released 2x model, and a 2.5x tier
|
|
|
| > **Licence: MIT.** Copyright (c) 2026 SamurAI-Official. Permission is hereby granted, free of charge, to any
|
| > person obtaining a copy of these model weights and this card to deal in them without restriction, including the
|
| > rights to use, copy, modify, merge, publish, distribute, sublicense and/or sell copies, subject to the terms in
|
| > [LICENSE](https://huggingface.co/SamurAI-Official/NRR/raw/main/LICENSE) (the MIT warranty disclaimer included).
|
| > These are the models already distributed under MIT, and they stay MIT.
|
| >
|
| > **Future models:** from the commit the repository moved to an All Rights Reserved licence, code and model
|
| > weights developed *after* that change are All Rights Reserved and are **not** covered by this MIT grant. A
|
| > Hugging Face repo carries one `license:` field, so any such model belongs in its own repo or a clearly-marked
|
| > path rather than silently inheriting the MIT tag above.
|
|
|
| A 2x neural upscaler as a single ONNX graph of about 461 KB, trained in the
|
| [NRR repository](https://github.com/SamurAI-Official/NRR) and used there as *the released model*: the one the
|
| Unreal, Godot and Unity plugins load, the one `docs/parity.md` measures, and the one this repository's headless
|
| verifications render.
|
|
|
| The same repository publishes a **2.5x** model under [`ratio25/`](#the-25x-tier-ratio25), which is what the
|
| 768x432 -> 1920x1080 tier needs. The released 2x model cannot be asked for that tier: the ratio is the graph's own
|
| head, so fed a 768x432 frame it returns 1536x864, and it reaches 1920x1080 only from a 960x540 input.
|
|
|
| It is a prototype model from a research repository rather than a shipping upscaler. It is published so the
|
| plugin side of that repository can be verified against a real model instead of a sample.
|
|
|
| ## The graph
|
|
|
| | | |
|
| | --- | --- |
|
| | inputs | `color` - the low-resolution frame; `jitter` - the sub-pixel offset the frame was rendered at (all zeros for a render that does not jitter); `scale` - **the tier token** |
|
| | output | one frame, **2x** the input on both axes |
|
| | spatial | dynamic H/W: the graph takes whatever size the input is, **including non-square**. Verified with `tools/verify_rect_upscale.py` in the NRR repository on a 160x90 rectangular frame: the output is exactly `(1, 3, 180, 320)` - 2x on both axes, finite, no square reshape anywhere. The 2.5x graphs below are checked the same way |
|
| | the `scale` token | `0.0` for the **128 -> 256** grid, `1.0` for the **256 -> 512** grid. The model was trained at both, so it is measured at both - `docs/parity.md` and `docs/parity-512.md` are two halves of one result rather than two results. NRR's own harness derives the token from the frame's width (`log2(width / 128)`), so each tier is handed the value it was trained with |
|
| | depth, motion | **not inputs.** This model declares neither; those belong to the temporal models in the repository, not to this one |
|
|
|
| ## Measured quality (this repository's own harness, not a vendor's)
|
|
|
| From `docs/parity.md` - all val scenes (`models/training-data/godot-v6-warp`, 48 frames, 2.00x), on an
|
| NVIDIA GeForce RTX 4070 Ti, driver 610.88. Every arm below was driven from the scene's own frames and scored
|
| against that scene's own targets:
|
|
|
| | arm | tier | PSNR dB | SSIM | MS-SSIM | LPIPS | DISTS | VMAF | detail |
|
| | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| | bilinear | 128x128->256x256 | 26.3563 | 0.87165 | 0.95257 | 0.1025 | 0.2732 | 36.793 | 0.2634 |
|
| | bicubic | 128x128->256x256 | 25.7689 | 0.86524 | 0.94596 | 0.0936 | 0.2795 | 40.399 | 0.4258 |
|
| | lanczos | 128x128->256x256 | 25.4109 | 0.85991 | 0.94447 | 0.1025 | 0.2846 | 40.912 | 0.4459 |
|
| | fsr1 (EASU + RCAS) | 128x128->256x256 | 24.6755 | 0.84641 | 0.93548 | 0.0947 | 0.2958 | 42.255 | 0.7668 |
|
| | **nrr - this model** | 128x128->256x256 | **26.4320** | **0.88151** | **0.95515** | **0.0570** | **0.1609** | 43.374 | 0.5616 |
|
| | DLSS (NGX) | 128x128->256x256 | 26.6829 | 0.88217 | 0.96077 | 0.1047 | 0.2576 | 52.253 | 0.2378 |
|
| | XeSS 2.0.2 | 128x128->256x256 | 26.7147 | 0.87228 | 0.95586 | 0.1319 | 0.2718 | 31.847 | 0.1774 |
|
|
|
| That is a comparison of *these frames at this tier*, which is what it is. No row here is a general claim about any
|
| upscaler, and a timing taken on a host doing other work is a timing on a busy host. The repository's
|
| `docs/evaluation-protocol.md` states the rules the table is held to.
|
|
|
| ## The 2.5x tier (`ratio25/`)
|
|
|
| `ratio25/upscale_25x_procedural_depth.onnx` upscales by **2.5** on both axes, and it is the first model in that
|
| repository trained at a ratio that is not a power of two *and* qualified at it by the repository's own gates - a
|
| model that fails one is refused, not exported with a warning next to it.
|
|
|
| | | |
|
| | --- | --- |
|
| | inputs | `color` - the low-resolution frame; `depth` - per-pixel ray distance on the input's grid |
|
| | output | one frame, **2.5x** the input on both axes (768x432 -> 1920x1080) |
|
| | spatial | dynamic H/W: the graph takes whatever size the input is |
|
| | parameters | 90,974 (369,025 bytes) |
|
| | sha256 | `86326c5d7c4ca5b7b5e7b5e206b55de2909f80a7fee7d1938d3d661545c545d9` |
|
| | how the ratio is reached | `PixelShuffle(5)` synthesises 25 polyphase components per colour channel, a learned stride-2 convolution selects the phase, and the skip is the 2.5x bilinear upsample - so nothing outside the graph resamples the frame, and an untrained model *is* that ratio's bilinear baseline |
|
|
|
| Measured on the run's own held-out split (120 procedural pairs, `color` being the noisy packed input the runtime
|
| supplies), with the 2.5x bilinear baseline scored on the same frames by the same code:
|
|
|
| | arm | L1 | SSIM | PSNR dB | MS-SSIM |
|
| | --- | --- | --- | --- | --- |
|
| | bilinear at 2.5x | 0.02542 | 0.8897 | 21.55 | 0.9427 |
|
| | **this model** | **0.01058** | **0.9479** | **27.38** | **0.9739** |
|
|
|
| 58.37% below the baseline on the metric the training gates score (the bar is 5%), with training progress 58.9% and
|
| a `depth` ablation of 0.02742 - the gate that refuses an input the model does not actually use. Through the
|
| runtime at the tier: **224.6 ms/frame** on an RTX 4070 Ti (CUDA provider, 102.1 ms of it inference) and 2104.3 ms
|
| on the same host's CPU execution provider; the benchmark seeds a synthetic depth ramp for depth-consuming models,
|
| so those rows measure speed and the quality numbers above come from the trainer's held-out split instead.
|
|
|
| What is **not** claimed about the procedural model. Its data is generated (`tools/gen_training_pairs.py
|
| --ratio 2.5`: each scene rendered twice, at 128 and at 320, so the target is a real render at the ratio rather
|
| than a resample onto it), so its 58.37% is an **in-family** number: held-out pairs, but the same scenes, the same
|
| renderer and no held-out content. It is not comparable to the 2x table below either, which is measured on
|
| captured frames by a different harness at a different tier.
|
|
|
| ## The same ratio trained on captured frames (`ratio25/`)
|
|
|
| These are the out-of-family numbers: the same ratio, trained on frames *captured* from Godot - the content
|
| pipeline the released 2x model was trained on - over 840 pairs (360 train / 480 held out) of seven scenes the
|
| model has never seen.
|
|
|
| | model | inputs | parameters | held-out L1 vs the 2.5x bilinear baseline | SSIM | PSNR dB | MS-SSIM |
|
| | --- | --- | --- | --- | --- | --- | --- |
|
| | `ratio25/upscale_25x_captured_nj_depth_s2.onnx` | `color`, `depth` | 90,974 | **0.01359 vs 0.01516 → 10.37% better** | 0.9035 vs 0.8947 | 26.81 vs **27.22** | 0.9715 vs 0.9624 |
|
| | `ratio25/upscale_25x_captured_depthjitter.onnx` | `color`, `depth`, `jitter` | 93,430 | **0.01399 vs 0.01541 → 9.24% better** | 0.9008 vs 0.8936 | 27.66 vs 27.10 | 0.9689 vs 0.9600 |
|
|
|
| Three things a reader should know before choosing between them:
|
|
|
| * **The `color,depth` model loses PSNR.** It is 0.41 dB *below* the bilinear baseline (26.81 against 27.22)
|
| while gaining on L1, SSIM and MS-SSIM. The training gates score L1, and the column that disagrees is shown
|
| rather than dropped. The `color,depth,jitter` model is above the baseline on all four metrics.
|
| * **The `color,depth` configuration is seed-fragile on captured content: 1 of 4 seeds qualified.** One seed froze
|
| at the bilinear plane and then diverged (residual L1 0.01756 → 0.29974), one reached 1.9% progress with its
|
| output 0.000302 from the plane, one reached 2.5%. The published weights are the seed that qualified
|
| (20261112). The spread is part of the result - the procedural configuration in the section above qualified on
|
| its first seed with 58.37%.
|
| * **The phase is what makes a jittered capture learnable.** On the *jittered* capture - the low-resolution
|
| sample moving up to half a pixel between frames - `color,depth` measured val L1 equal to the baseline's to five
|
| decimals (0.01541 against 0.01541), with an output 0.000155 from the plane: a half-pixel misalignment the model
|
| was never told about has the bilinear plane as its L1 optimum. Declaring `jitter` on the same frames reached
|
| 9.24%, with both declared inputs measurably used (depth 0.00540, jitter 0.00695).
|
|
|
| Both graphs are trained at 128 -> 320 and are dynamic in H/W, so both render 768x432 -> 1920x1080; the runtime
|
| rows are `docs/measurements/ratio25-captured-*` in the repository, and the training configuration, the captures
|
| and the packing script are there too.
|
|
|
| One thing to know before comparing this model's output across hardware: ONNX Runtime's CUDA execution provider
|
| computes convolutions in TF32 by default, so this graph's CPU and CUDA outputs differ by up to **6.341e-04** (mean
|
| 1.558e-05) while agreeing to **2.176e-06** with `use_tf32=0`. The two recorded runtime rows therefore carry
|
| different frame hashes for a named reason rather than a mysterious one, and a re-run on one provider reproduces
|
| its hash exactly.
|
|
|
| ## The 2.5x tier against the field (`docs/parity-ratio25.md`)
|
|
|
| `docs/parity-ratio25.md` scores the 2.5x ratio against the offline field and one commercial arm, all driven from
|
| the same captured frames and scored against the same targets. It is a *2.5x* table, so its offline baselines
|
| upscale by 2.5 (a 2x-tier table's baselines upscale by 2 - a row that says `768x768->1920x1920` while its bilinear
|
| arm upscaled by 2 would be scoring one grid and labelling it another):
|
|
|
| | arm | tier | PSNR dB | SSIM | MS-SSIM | LPIPS | DISTS | detail |
|
| | --- | --- | --- | --- | --- | --- | --- | --- |
|
| | bilinear | 768x768->1920x1920 | 27.1135 | 0.88770 | 0.94527 | 0.1168 | 0.2992 | 0.1330 |
|
| | bicubic | 768x768->1920x1920 | 26.4337 | 0.87777 | 0.93720 | 0.1343 | 0.3095 | 0.2052 |
|
| | lanczos | 768x768->1920x1920 | 26.1083 | 0.87372 | 0.93362 | 0.1432 | 0.3106 | 0.2178 |
|
| | fsr1 (EASU + RCAS) | 768x768->1920x1920 | 25.6000 | 0.86508 | 0.92990 | 0.1402 | 0.3447 | 0.3919 |
|
| | nrr - the procedural model | 768x768->1920x1920 | 26.9605 | 0.87093 | 0.94155 | 0.1332 | 0.3312 | 0.2919 |
|
| | DLSS (NGX) | 768x768->1920x1920 | 26.9670 | 0.89735 | 0.95684 | 0.0934 | 0.2684 | 0.1567 |
|
|
|
| 24 frames, one held-out scene, RTX 4070 Ti, driver 610.88. What a reader should hold on to:
|
|
|
| * **The grid is square, not 16:9.** The capture renders a square viewport, so this tier is `768x768 -> 1920x1920`
|
| rather than the `768x432 -> 1920x1080` the model's README names. The ratio is the same, the crop is not; the
|
| graph is dynamic in H/W and renders either, so the measured table is the square one.
|
| * **The procedural model loses to bilinear here, and that is the point of the section above.** Its 58.37% is an
|
| *in-family* number from generated data; scored on captured frames it lands below its own bilinear baseline on
|
| PSNR (26.96 vs 27.11), SSIM and MS-SSIM, keeping only `detail` (0.2919 vs 0.1330). That is the out-of-family gap
|
| the captured models above close - `upscale_25x_captured_depthjitter.onnx` is the nrr row this table wants next.
|
| * **XeSS has no 2.5x row.** Its presets are fixed input sizes: `balanced` at a 1920 output wants 960, which is
|
| exactly 2.0x. The row is absent because XeSS refuses to produce one, not because it was dropped.
|
| * **FSR 2/4 are absent by structure, not omission.** FSR 4 is not distributed as a runtime this harness can call,
|
| and FSR 2/3/4 are temporal and consume an engine's motion/jitter/history planes, so their rows come from the
|
| engines that ship them.
|
| * **VMAF is skipped** (`--skip-vmaf`): `evaluate_model`'s VMAF path writes into a temp directory that is not
|
| there, so this run omits the column rather than being quietly shortened.
|
|
|
| ## The 16:9 model (`rect/rect-model.onnx`) - a plumbing proof, not a quality result
|
|
|
| `rect-model.onnx` is **not** a released model and is **not** an alternative to the 2x graph above. It exists to
|
| answer one question: does a *non-square* canvas render end to end, or does something in the pipeline assume a
|
| square? It renders. `tools/verify_rect_upscale.py` in the NRR repository feeds a 160x90 frame through it and
|
| asserts the output is exactly `(1, 3, 180, 320)` - 2x on both axes, finite, no square reshape anywhere.
|
|
|
| Read its numbers with that in mind, because they are **worse than bilinear**:
|
|
|
| | arm | 160x90 -> 320x180, 6 held-out val pairs | PSNR dB | SSIM | MS-SSIM |
|
| | --- | --- | --- | --- | --- |
|
| | bilinear (the baseline) | 18 captured pairs, 12 train / 6 val | 28.7766 | 0.90028 | 0.96080 |
|
| | rect-model.onnx | same split | 28.7725 | 0.89984 | 0.96075 |
|
|
|
| It was exported with `--export-unqualified`, and its own `rect-model.report.json` records the three gates it
|
| fails:
|
|
|
| * **does not beat the baseline** - val L1 0.011708 against the baseline's 0.011638, which is not 5% lower;
|
| * **is the baseline** - its output differs from the plane it corrects by only 0.000724, against a 0.0010 floor,
|
| so the network learned almost nothing;
|
| * **ignores motion** - zeroing the `motion` input changes the output by 0.000000, against a 0.0001 floor.
|
|
|
| So: 112,907 parameters trained for 3 epochs (6.91 s on CPU) over **18 pairs** from a single capture, which is
|
| a smoke test's worth of data. Its `motion` and `depth` ablations are near zero because the ablation, not the
|
| training, is the interesting part of the check - and `verify_rect_upscale.py` is what actually passes here.
|
|
|
| It is published beside the released model for the same reason the 2.5x tier is: it is the artifact a claim in
|
| the repository points at, and a claim a reader cannot download is a claim they have to take on trust. Its
|
| provenance travels with it as `rect/rect-model.report.json`, which is gitignored in the repository itself.
|
|
|
|
|
| ## Files
|
|
|
| | file | bytes | what it is |
|
| | --- | --- | --- |
|
| | `upscale_msreal_scale.onnx` | 461,501 | the released model: the two-tier (`scale` token) upscaler described above |
|
| | `rect/rect-model.onnx` | 456,548 | the 16:9 plumbing proof: 160x90 -> 320x180, inputs `color`, `depth`, `motion`, 112,907 parameters, exported `--export-unqualified` and **below its bilinear baseline** on all three image metrics - see the section above |
|
| | `rect/rect-model.report.json` | 3,714 | its provenance, including the three release gates it fails |
|
| | `ratio25/upscale_25x_procedural_depth.onnx` | 369,025 | the 2.5x model: 768x432 -> 1920x1080, inputs `color`, `depth` |
|
| | `ratio25/upscale_25x_procedural_depth.report.json` | 2,934 | that model's gate report: the configuration, the sha256, the held-out numbers and the baselines they were scored against - the repository gitignores `models/**/*.json`, so this copy is the one that travels with the weights |
|
| | `ratio25/upscale_25x_captured_nj_depth_s2.onnx` | 369,025 | the same ratio trained on **captured** frames: inputs `color`, `depth`, 10.37% below its ratio's bilinear baseline on 480 held-out captured pairs |
|
| | `ratio25/upscale_25x_captured_nj_depth_s2.report.json` | 2,955 | its gate report, including the baselines every number was scored against |
|
| | `ratio25/upscale_25x_captured_depthjitter.onnx` | 384,090 | the same capture with the phase declared: inputs `color`, `depth`, `jitter`, 9.24% better and above the baseline on all four metrics |
|
| | `ratio25/upscale_25x_captured_depthjitter.report.json` | 2,991 | its gate report |
|
| | `phases/upscale_subsampled.onnx` | 424,038 | the 128 -> 256 model `docs/evaluation-protocol.md` names when it explains why a model must be judged at the tier it was trained for |
|
| | `phases/upscale_mid.onnx` | 424,038 | mid-training model, kept because the phase tables compare against it |
|
| | `phases/upscale_mid_regression.onnx` | 424,038 | the mid model as a regression baseline |
|
| | `probes/*.onnx` | 790 - 45,082 | the sample model and the small probe graphs the test suite and the plugin drift guards load (`nrr_upscaler_v0.1`, `nrr_passthrough_2x`, `nrr_scale_token_probe`, `nrr_unrecognised_input_probe`, `nrr_history_probe`) |
|
| | `tests/p4_tjit_20261025.onnx` | 499,854 | the reference model the Unity verification project renders with: its tests score their output against `engine_plugins/unity_verify/Assets/StreamingAssets/smoke_fixture/fixture.json`, whose recorded reference means were produced by this exact file. The tests select the highest-dated `models/p5/p4_tjit_*.onnx`, so this is a real training checkpoint - one of the history, whose other 118 checkpoints are in [SamurAI-Official/NRR-checkpoints](https://huggingface.co/SamurAI-Official/NRR-checkpoints) |
|
|
|
| ## Using it
|
|
|
| With NRR itself:
|
|
|
| ```c
|
| nrr_model_load(device, "upscale_msreal_scale.onnx", &model);
|
| nrr_render(device, model, /*references=*/NULL, &input, &output); /* the runtime derives scale from the width */
|
| ```
|
|
|
| The third argument is the reference set, not the inputs: NRR hands a model the planes it *declares*, and takes
|
| them from the frame - `NRRFrameInput.color` / `.depth` / `.motion_vectors`, `NRRTemporalState.jitter` and
|
| `.previous_output` for a temporal model, and `scale` derived from the frame's width. That is the whole difference
|
| between the two contracts on this page. The released 2x model declares `color`, `jitter`, `scale`, which the
|
| runtime already has; the 2.5x model declares `color` and `depth`, so a caller has to populate `input.depth`:
|
|
|
| ```c
|
| nrr_model_load(device, "ratio25/upscale_25x_procedural_depth.onnx", &model);
|
| nrr_render(device, model, /*references=*/NULL, &input, &output); /* input.depth carries the frame's depth */
|
| ```
|
|
|
| Fetching it into a checkout, which puts it exactly where the repository expects it
|
| (`engine_plugins/unreal_verify/setup.ps1` and the test suite load it from there):
|
|
|
| ```powershell
|
| hf download SamurAI-Official/NRR upscale_msreal_scale.onnx --local-dir models/phase4
|
| ```
|
|
|
| Anywhere else it is a plain ONNX graph:
|
|
|
| ```python
|
| import onnxruntime
|
| session = onnxruntime.InferenceSession("upscale_msreal_scale.onnx")
|
| ```
|
|
|
| ## Provenance, and what is not claimed
|
|
|
| Trained in the NRR repository's own trainer on frames captured from Godot; the training configuration, the
|
| comparison harness, the plugins that load it and every number above live in that repository. The released model
|
| is not tracked by git there (`models/phase4/` is working state), which is why it is here.
|
|
|
| The 2.5x model in `ratio25/` was trained in the same repository, by `tools/run_ratio25.ps1` on data it generates
|
| rather than data it captured (`tools/gen_training_pairs.py --ratio 2.5`), and its gate report travels with it
|
| because the repository gitignores `models/**/*.json` - without that copy the artifact's provenance would live only
|
| in a changelog. The weights are tracked there, so the two copies can be compared byte for byte.
|
|
|
| Verification status, stated because a model that loads is not a model that ran: NRR's Unreal project renders this
|
| model at three tiers headless and prints `RESULT: PASS` (`engine_plugins/unreal_verify/README.md`), and the Godot
|
| addon is verified end to end (`CHANGELOG.md`). Those runs are on a host where the CUDA execution provider
|
| *crashes* during model load - a host/driver result, recorded in that README - so the numbers there are CPU
|
| numbers.
|
|
|
| One description in the repository does **not** match this model: `models/architecture.md` there still describes an
|
| aspirational 1080p -> 4K color+depth+motion network, which is not what is published here. The contract above is
|
| this graph's real one, read from the model.
|
|
|
| Licence: **All Rights Reserved** - see `LICENSE` (Copyright (c) 2026
|
| SamurAI-Official). The weights and this card are covered by the same terms as the
|
| repository; nothing here is granted under the MIT licence the repository
|
| originally carried.
|
|
|
