---
license: mit
library_name: onnxruntime
pipeline_tag: image-to-image
tags:
  - super-resolution
  - upscaling
  - onnx
  - real-time
  - nrr
---

# NRR upscalers - the released 2x model, and a 2.5x tier

A 2x neural upscaler as a single ONNX graph of about 461 KB, trained in the
[NRR repository](https://github.com/SamurAI-Official/NRR) and used there as *the released model*: the one the
Unreal, Godot and Unity plugins load, the one `docs/parity.md` measures, and the one this repository's headless
verifications render.

The same repository publishes a **2.5x** model under [`ratio25/`](#the-25x-tier-ratio25), which is what the
768x432 -> 1920x1080 tier needs. The released 2x model cannot be asked for that tier: the ratio is the graph's own
head, so fed a 768x432 frame it returns 1536x864, and it reaches 1920x1080 only from a 960x540 input.

It is a prototype model from a research repository rather than a shipping upscaler. It is published so the
plugin side of that repository can be verified against a real model instead of a sample.

## The graph

| | |
| --- | --- |
| inputs | `color` - the low-resolution frame; `jitter` - the sub-pixel offset the frame was rendered at (all zeros for a render that does not jitter); `scale` - **the tier token** |
| output | one frame, **2x** the input on both axes |
| spatial | dynamic H/W: the graph takes whatever size the input is |
| the `scale` token | `0.0` for the **128 -> 256** grid, `1.0` for the **256 -> 512** grid. The model was trained at both, so it is measured at both - `docs/parity.md` and `docs/parity-512.md` are two halves of one result rather than two results. NRR's own harness derives the token from the frame's width, so each tier is handed the value it was trained with |
| depth, motion | **not inputs.** This model declares neither; those belong to the temporal models in the repository, not to this one |

## Measured quality (this repository's own harness, not a vendor's)

From `docs/parity.md` - all val scenes (`models/training-data/godot-v6-warp`, 48 frames, 2.00x), on an
NVIDIA GeForce RTX 4070 Ti, driver 610.88. Every arm below was driven from the scene's own frames and scored
against that scene's own targets:

| arm | tier | PSNR dB | SSIM | MS-SSIM | LPIPS | DISTS | VMAF | detail |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| bilinear | 128x128->256x256 | 26.3563 | 0.87165 | 0.95257 | 0.1025 | 0.2732 | 36.793 | 0.2634 |
| bicubic | 128x128->256x256 | 25.7689 | 0.86524 | 0.94596 | 0.0936 | 0.2795 | 40.399 | 0.4258 |
| lanczos | 128x128->256x256 | 25.4109 | 0.85991 | 0.94447 | 0.1025 | 0.2846 | 40.912 | 0.4459 |
| fsr1 (EASU + RCAS) | 128x128->256x256 | 24.6755 | 0.84641 | 0.93548 | 0.0947 | 0.2958 | 42.255 | 0.7668 |
| **nrr - this model** | 128x128->256x256 | **26.4320** | **0.88151** | **0.95515** | **0.0570** | **0.1609** | 43.374 | 0.5616 |
| DLSS (NGX) | 128x128->256x256 | 26.6829 | 0.88217 | 0.96077 | 0.1047 | 0.2576 | 52.253 | 0.2378 |
| XeSS 2.0.2 | 128x128->256x256 | 26.7147 | 0.87228 | 0.95586 | 0.1319 | 0.2718 | 31.847 | 0.1774 |

That is a comparison of *these frames at this tier*, which is what it is. No row here is a general claim about any
upscaler, and a timing taken on a host doing other work is a timing on a busy host. The repository's
`docs/evaluation-protocol.md` states the rules the table is held to.

## The 2.5x tier (`ratio25/`)

`ratio25/upscale_25x_procedural_depth.onnx` upscales by **2.5** on both axes, and it is the first model in that
repository trained at a ratio that is not a power of two *and* qualified at it by the repository's own gates - a
model that fails one is refused, not exported with a warning next to it.

| | |
| --- | --- |
| inputs | `color` - the low-resolution frame; `depth` - per-pixel ray distance on the input's grid |
| output | one frame, **2.5x** the input on both axes (768x432 -> 1920x1080) |
| spatial | dynamic H/W: the graph takes whatever size the input is |
| parameters | 90,974 (369,025 bytes) |
| sha256 | `86326c5d7c4ca5b7b5e7b5e206b55de2909f80a7fee7d1938d3d661545c545d9` |
| how the ratio is reached | `PixelShuffle(5)` synthesises 25 polyphase components per colour channel, a learned stride-2 convolution selects the phase, and the skip is the 2.5x bilinear upsample - so nothing outside the graph resamples the frame, and an untrained model *is* that ratio's bilinear baseline |

Measured on the run's own held-out split (120 procedural pairs, `color` being the noisy packed input the runtime
supplies), with the 2.5x bilinear baseline scored on the same frames by the same code:

| arm | L1 | SSIM | PSNR dB | MS-SSIM |
| --- | --- | --- | --- | --- |
| bilinear at 2.5x | 0.02542 | 0.8897 | 21.55 | 0.9427 |
| **this model** | **0.01058** | **0.9479** | **27.38** | **0.9739** |

58.37% below the baseline on the metric the training gates score (the bar is 5%), with training progress 58.9% and
a `depth` ablation of 0.02742 - the gate that refuses an input the model does not actually use. Through the
runtime at the tier: **224.6 ms/frame** on an RTX 4070 Ti (CUDA provider, 102.1 ms of it inference) and 2104.3 ms
on the same host's CPU execution provider; the benchmark seeds a synthetic depth ramp for depth-consuming models,
so those rows measure speed and the quality numbers above come from the trainer's held-out split instead.

What is **not** claimed about the procedural model. Its data is generated (`tools/gen_training_pairs.py
--ratio 2.5`: each scene rendered twice, at 128 and at 320, so the target is a real render at the ratio rather
than a resample onto it), so its 58.37% is an **in-family** number: held-out pairs, but the same scenes, the same
renderer and no held-out content. It is not comparable to the 2x table below either, which is measured on
captured frames by a different harness at a different tier.

## The same ratio trained on captured frames (`ratio25/`)

These are the out-of-family numbers: the same ratio, trained on frames *captured* from Godot - the content
pipeline the released 2x model was trained on - over 840 pairs (360 train / 480 held out) of seven scenes the
model has never seen.

| model | inputs | parameters | held-out L1 vs the 2.5x bilinear baseline | SSIM | PSNR dB | MS-SSIM |
| --- | --- | --- | --- | --- | --- | --- |
| `ratio25/upscale_25x_captured_nj_depth_s2.onnx` | `color`, `depth` | 90,974 | **0.01359 vs 0.01516 → 10.37% better** | 0.9035 vs 0.8947 | 26.81 vs **27.22** | 0.9715 vs 0.9624 |
| `ratio25/upscale_25x_captured_depthjitter.onnx` | `color`, `depth`, `jitter` | 93,430 | **0.01399 vs 0.01541 → 9.24% better** | 0.9008 vs 0.8936 | 27.66 vs 27.10 | 0.9689 vs 0.9600 |

Three things a reader should know before choosing between them:

* **The `color,depth` model loses PSNR.** It is 0.41 dB *below* the bilinear baseline (26.81 against 27.22)
  while gaining on L1, SSIM and MS-SSIM. The training gates score L1, and the column that disagrees is shown
  rather than dropped. The `color,depth,jitter` model is above the baseline on all four metrics.
* **The `color,depth` configuration is seed-fragile on captured content: 1 of 4 seeds qualified.** One seed froze
  at the bilinear plane and then diverged (residual L1 0.01756 → 0.29974), one reached 1.9% progress with its
  output 0.000302 from the plane, one reached 2.5%. The published weights are the seed that qualified
  (20261112). The spread is part of the result - the procedural configuration in the section above qualified on
  its first seed with 58.37%.
* **The phase is what makes a jittered capture learnable.** On the *jittered* capture - the low-resolution
  sample moving up to half a pixel between frames - `color,depth` measured val L1 equal to the baseline's to five
  decimals (0.01541 against 0.01541), with an output 0.000155 from the plane: a half-pixel misalignment the model
  was never told about has the bilinear plane as its L1 optimum. Declaring `jitter` on the same frames reached
  9.24%, with both declared inputs measurably used (depth 0.00540, jitter 0.00695).

Both graphs are trained at 128 -> 320 and are dynamic in H/W, so both render 768x432 -> 1920x1080; the runtime
rows are `docs/measurements/ratio25-captured-*` in the repository, and the training configuration, the captures
and the packing script are there too.

One thing to know before comparing this model's output across hardware: ONNX Runtime's CUDA execution provider
computes convolutions in TF32 by default, so this graph's CPU and CUDA outputs differ by up to **6.341e-04** (mean
1.558e-05) while agreeing to **2.176e-06** with `use_tf32=0`. The two recorded runtime rows therefore carry
different frame hashes for a named reason rather than a mysterious one, and a re-run on one provider reproduces
its hash exactly.

## Files

| file | bytes | what it is |
| --- | --- | --- |
| `upscale_msreal_scale.onnx` | 461,501 | the released model: the two-tier (`scale` token) upscaler described above |
| `ratio25/upscale_25x_procedural_depth.onnx` | 369,025 | the 2.5x model: 768x432 -> 1920x1080, inputs `color`, `depth` |
| `ratio25/upscale_25x_procedural_depth.report.json` | 2,934 | that model's gate report: the configuration, the sha256, the held-out numbers and the baselines they were scored against - the repository gitignores `models/**/*.json`, so this copy is the one that travels with the weights |
| `ratio25/upscale_25x_captured_nj_depth_s2.onnx` | 369,025 | the same ratio trained on **captured** frames: inputs `color`, `depth`, 10.37% below its ratio's bilinear baseline on 480 held-out captured pairs |
| `ratio25/upscale_25x_captured_nj_depth_s2.report.json` | 2,955 | its gate report, including the baselines every number was scored against |
| `ratio25/upscale_25x_captured_depthjitter.onnx` | 384,090 | the same capture with the phase declared: inputs `color`, `depth`, `jitter`, 9.24% better and above the baseline on all four metrics |
| `ratio25/upscale_25x_captured_depthjitter.report.json` | 2,991 | its gate report |
| `phases/upscale_subsampled.onnx` | 424,038 | the 128 -> 256 model `docs/evaluation-protocol.md` names when it explains why a model must be judged at the tier it was trained for |
| `phases/upscale_mid.onnx` | 424,038 | mid-training model, kept because the phase tables compare against it |
| `phases/upscale_mid_regression.onnx` | 424,038 | the mid model as a regression baseline |
| `probes/*.onnx` | 790 - 45,082 | the sample model and the small probe graphs the test suite and the plugin drift guards load (`nrr_upscaler_v0.1`, `nrr_passthrough_2x`, `nrr_scale_token_probe`, `nrr_unrecognised_input_probe`, `nrr_history_probe`) |
| `tests/p4_tjit_20261025.onnx` | 499,854 | the reference model the Unity verification project renders with: its tests score their output against `engine_plugins/unity_verify/Assets/StreamingAssets/smoke_fixture/fixture.json`, whose recorded reference means were produced by this exact file. The tests select the highest-dated `models/p5/p4_tjit_*.onnx`, so this is a real training checkpoint - one of the history, whose other 118 checkpoints are in [SamurAI-Official/NRR-checkpoints](https://huggingface.co/SamurAI-Official/NRR-checkpoints) |

## Using it

With NRR itself:

```c
nrr_model_load(device, "upscale_msreal_scale.onnx", &model);
nrr_render(device, model, /*references=*/NULL, &input, &output);   /* the runtime derives scale from the width */
```

The third argument is the reference set, not the inputs: NRR hands a model the planes it *declares*, and takes
them from the frame - `NRRFrameInput.color` / `.depth` / `.motion_vectors`, `NRRTemporalState.jitter` and
`.previous_output` for a temporal model, and `scale` derived from the frame's width. That is the whole difference
between the two contracts on this page. The released 2x model declares `color`, `jitter`, `scale`, which the
runtime already has; the 2.5x model declares `color` and `depth`, so a caller has to populate `input.depth`:

```c
nrr_model_load(device, "ratio25/upscale_25x_procedural_depth.onnx", &model);
nrr_render(device, model, /*references=*/NULL, &input, &output);   /* input.depth carries the frame's depth */
```

Fetching it into a checkout, which puts it exactly where the repository expects it
(`engine_plugins/unreal_verify/setup.ps1` and the test suite load it from there):

```powershell
hf download SamurAI-Official/NRR upscale_msreal_scale.onnx --local-dir models/phase4
```

Anywhere else it is a plain ONNX graph:

```python
import onnxruntime
session = onnxruntime.InferenceSession("upscale_msreal_scale.onnx")
```

## Provenance, and what is not claimed

Trained in the NRR repository's own trainer on frames captured from Godot; the training configuration, the
comparison harness, the plugins that load it and every number above live in that repository. The released model
is not tracked by git there (`models/phase4/` is working state), which is why it is here.

The 2.5x model in `ratio25/` was trained in the same repository, by `tools/run_ratio25.ps1` on data it generates
rather than data it captured (`tools/gen_training_pairs.py --ratio 2.5`), and its gate report travels with it
because the repository gitignores `models/**/*.json` - without that copy the artifact's provenance would live only
in a changelog. The weights are tracked there, so the two copies can be compared byte for byte.

Verification status, stated because a model that loads is not a model that ran: NRR's Unreal project renders this
model at three tiers headless and prints `RESULT: PASS` (`engine_plugins/unreal_verify/README.md`), and the Godot
addon is verified end to end (`CHANGELOG.md`). Those runs are on a host where the CUDA execution provider
*crashes* during model load - a host/driver result, recorded in that README - so the numbers there are CPU
numbers.

One description in the repository does **not** match this model: `models/architecture.md` there still describes an
aspirational 1080p -> 4K color+depth+motion network, which is not what is published here. The contract above is
this graph's real one, read from the model.

Licence: MIT - see `LICENSE`.