CorridorKey Real-Time Edge Refiner
A 23k-parameter CNN that brings back native-resolution hair and edge detail when CorridorKey 2.0 runs at reduced resolution for live green-screen keying, e.g. a 4K or 1080p facecam in OBS at 30–60 fps.
Footage: CorridorKey 1.0 Quality Benchmark (Niko Pueringer / Corridor Digital, Apache-2.0). Real-time pipeline: CorridorKey 2.0 Stage 1 on a 512×512 person crop + this refiner.
What problem it solves
CorridorKey 2.0 produces excellent mattes, but at full 4K it is an offline model. To key live video, the network has to run on a small grid: for example a 512×512 crop around the person, or 512×256 for the whole frame. Its matte then has to be scaled back up to the camera resolution, and that step destroys the edge:
- Bilinear upsampling gives a blurry, blocky edge with a green or dark halo.
- A colour guided filter recovers some structure, but it smears hair, leaves fringes and fails where foreground and screen have similar brightness.
- This refiner looks at the native-resolution pixels and learns how CorridorKey 2.0 would have keyed them at full resolution. It predicts corrections for alpha and foreground colour (despill / unmixing) at every native pixel.
Top row: composite over a dark slate. Bottom row: alpha. All crops are 1:1 native pixels, enlarged with nearest neighbour. "CorridorKey 2.0 offline" is the full model at native resolution, the target the refiner was trained on.
How it works
The refiner is a fully convolutional network: 11 input channels, a 3×3 input convolution, four residual dilated 3×3 blocks (dilations 1, 2, 4, 1), GELU, and a 1×1 output convolution. It has 24 channels in v12 and v10w and 16 in v9b. Its receptive field is ±9 pixels. Per native pixel it receives:
| Channels | Input |
|---|---|
| 0–2 | I: native RGB frame (sRGB) |
| 3–5 | B: smooth background estimate (sRGB), hole-filled from pixels CorridorKey calls background |
| 6–8 | F: CorridorKey foreground transferred onto the native frame (I + upsample(F_low − I_low)) |
| 9 | a_up: CorridorKey alpha, bilinear upsampled |
| 10 | a_gf: alpha of a colour guided filter driven by CorridorKey's low-resolution alpha |
It outputs residuals on top of a_gf (alpha) and F (foreground). The result is used only inside an
uncertainty band around CorridorKey's edges. Outside the band the output is the colour-guided upsampling of
CorridorKey's matte, which in solid foreground and clean background is CorridorKey's own 0 or 1. The band is weighted
by nearby foreground, which keeps the refiner out of dark or noisy screen areas.
In the real-time engine the refiner only runs on 32×32 tiles that intersect the band, with a fixed tile budget
(most uncertain tiles first). In fp16 that costs about 5–6 ms per frame with 256 tiles and 10–11 ms with 512 tiles
on an RTX 5070 Ti.
corridorkey_refiner.py in this repository is a standalone full-frame reference implementation of the same maths.
On a test frame it matches the tiled engine to a mean absolute difference of 0.0001 in alpha.
Files
| File | Description |
|---|---|
corridorkey_edge_refiner_v12.safetensors |
recommended (October 2026), 24 channels, same architecture and cost as v10w, lower edge error and flicker |
corridorkey_edge_refiner_v10w.safetensors |
previous version, 24 channels |
corridorkey_edge_refiner_v9b.safetensors |
16 channels, 10,948 parameters, slightly cheaper, slightly worse |
corridorkey_refiner.py |
model definition, loader, input construction and full-frame refine() |
LICENSE |
Corridor Key Licence, Version 2.0 |
Usage
You need CorridorKey 2.0 (code and weights from https://huggingface.co/nikopueringer/CorridorKey2.0, not included here) to produce the low-resolution matte. Then:
import torch
from corridorkey_refiner import load_refiner, refine
net = load_refiner("corridorkey_edge_refiner_v12.safetensors") # fp16, CUDA
# rgb: (1, 3, H, W) native frame, sRGB in [0, 1]
# rgb_low: (1, 3, h, w) the (crop of the) frame as CorridorKey saw it
# alpha_low: (1, 1, h', w') CorridorKey alpha at its output resolution (e.g. Stage-1 aux head, 256x256)
# fg_low: (1, 3, h', w') CorridorKey straight foreground, sRGB
alpha, fg = refine(net, rgb, rgb_low, alpha_low, fg_low) # native resolution
If you key a crop, pass the native-resolution crop as rgb and paste the result back. Apply despill and alpha
contrast after the refiner, as you would for CorridorKey's own output.
Results
Edge error = mean absolute alpha error inside the edge band (×1000), measured against CorridorKey 2.0 offline at native resolution; flicker = temporal error of the frame-to-frame alpha change in the edge band (×1000). Lower is better. 32 consecutive frames per segment. None of these clips were used for training.
- Real stock footage: 6 Pexels / Mixkit green-screen clips (4K, 4K DCI, 1080p, 720p), curly / wavy / long hair, headphones, headset, glasses.
- Real 4K, hair and Real 4K, transparency: the two clips of the CorridorKey 1.0 Quality Benchmark (2 segments each). The druid has curly hair, a beard and fur; the clown has tulle and water glasses.
- Generated streamers: the 8 series-3 test clips of version 2 of the dataset. They are deliberately hard (props in front of the person, partially covered screens, people leaving the frame).
Person crop, 512×512 (CorridorKey 2.0 Stage 1 on a tracked square around the person):
| Upsampling | Real stock | Real 4K, hair | Real 4K, transparency | Generated streamers | Flicker (real stock) | GPU ms / frame |
|---|---|---|---|---|---|---|
| Bilinear | 121.3 | 91.2 | 80.1 | 134.4 | 75.3 | — |
| Colour guided filter | 51.2 | 45.3 | 64.1 | 103.8 | 33.1 | 14.4 |
| Edge Refiner v9b | 32.5 | 32.7 | 61.9 | 88.9 | 22.0 | 24.4 |
| Edge Refiner v10w | 30.9 | 30.6 | 62.3 | 84.9 | 20.5 | 25.8 |
| Edge Refiner v12 | 29.3 | 28.6 | 61.8 | 83.0 | 19.7 | 25.8 |
Full frame, 512×256 (cheaper, for 60 fps):
| Upsampling | Real stock | Real 4K, hair | Real 4K, transparency | Generated streamers | Flicker (real stock) | GPU ms / frame |
|---|---|---|---|---|---|---|
| Bilinear | 174.7 | 117.8 | 106.7 | 187.8 | 91.5 | — |
| Colour guided filter | 61.3 | 53.4 | 76.4 | 137.5 | 34.9 | 11.8 |
| Edge Refiner v9b | 41.7 | 46.3 | 73.6 | 127.7 | 26.4 | 16.5 |
| Edge Refiner v10w | 39.4 | 45.2 | 73.5 | 125.0 | 25.0 | 17.6 |
| Edge Refiner v12 | 38.0 | 43.9 | 72.5 | 123.5 | 24.4 | 17.6 |
GPU time is end-to-end per frame, CorridorKey 2.0 Stage 1 included, on an RTX 5070 Ti. It is the median over the 6 real stock clips, measured without video I/O. v12 has the same architecture as v10w and therefore the same cost. The refiner itself costs about 5–6 ms with a budget of 256 tiles and 10–11 ms with 512 tiles. The tile count, not the frame resolution, dominates the cost.
- Compared with the guided filter, the refiner (v12) reduces the edge error by 38–43 % on the real stock clips and by 18–37 % on the 4K druid clip. Flicker drops by 30–40 %.
- v12 against v10w, at the same cost: lower edge error on every test set, by 3–7 % on hair (real stock, druid) and 1 % on the semi-transparent clown, and 2–4 % less flicker on the real clips.
- On hair (druid) the gain is largest. On semi-transparent material (clown) it is small: there the result is limited by what the low-resolution network sees.
Training
Target: CorridorKey 2.0 offline at native resolution (alpha and foreground). The refiner is a student of CorridorKey: it cannot be sharper than its teacher, and it inherits the teacher's mistakes.
Inputs during training: exactly what the real-time engine produces at run time. Training patches were cut by running the real-time pipeline (512×512 person crop and 512×256 full frame, Stage 1 only) over every training clip and computing the 11 input channels, mostly from the edge band.
Data:
- 32 segments of real stock green-screen footage (Pexels, Mixkit)
- the same frames re-composited from the teacher's foreground and alpha over procedural green/teal screens, also passed through a "bad camera" model (sensor noise, Bayer demosaicing, 4:2:0 chroma, JPEG, exposure errors)
- the 207 training clips of version 2 of the CorridorKey Synthetic Streamers dataset: every clip reviewed by eye, broken labels removed, everything in front of the screen kept as foreground (mugs, microphones, a second person)
Real footage is sampled 3× more often than its share. 110,000 patches, loss on alpha, alpha gradients, composite and foreground. v12: 96,000 steps (v10w: 24,000 steps on the unreviewed version 1 of the dataset).
Tried and dropped for v12: removing screen colour from the foreground before re-compositing (makes the synthetic frames cleaner than real camera frames, edge error +3–5 %) and 25 % blue synthetic screens (worse on every test set).
Evaluation data was never used for training.
Limitations
- The teacher is the ceiling. Where CorridorKey 2.0 is soft at native resolution, the refiner is soft too.
- It can only refine what the low-resolution network found. Single strands far outside CorridorKey's low-resolution matte are outside the refiner band and are lost, as in the image above. Semi-transparent objects (glass, tulle, water) depend almost entirely on the low-resolution network.
- Trained for green (and teal / blue-green) screens with people in front, mostly facecam and upper-body framing. It is not a general background-removal model.
- The numbers above compare against CorridorKey 2.0, not against hand-made ground truth.
License
Corridor Key Licence, Version 2.0 (CC BY-NC-SA 4.0 with Additional Terms by Corridor Digital LLC). The full
text is in LICENSE.
This refiner was trained on the outputs of CorridorKey 2.0 and only works together with it. It is therefore released as a CorridorKey derivative under the same licence and keeps the CorridorKey name (Additional Terms §4, §5). In short:
- Free of charge. You may not sell it, repackage it as a standalone or competing product, or offer it as a paid API or inference service.
- Share-alike: derivatives must be released under the same licence, free of charge, and keep the "CorridorKey" name.
- Images and video you produce with it are yours.
- Commercial software integrations require a separate agreement with Corridor Digital (contact@corridordigital.com). Organisations above US $20M revenue need CorridorKey plugin seats (https://corridorkey.com).
Attribution and modifications
- CorridorKey 2.0 © Corridor Digital LLC. The model was designed and trained by Zachary Harris in collaboration with Niko Pueringer and Corridor Digital. https://huggingface.co/nikopueringer/CorridorKey2.0 · https://corridorkey.com
- CorridorKey 2.0 builds on frozen foundation models under their own licences (MoGe-2 by Microsoft, RVM by Google DeepMind, optional C-RADIO by NVIDIA). None of them, and no CorridorKey weights, are contained in this repository.
- Changes relative to CorridorKey 2.0: this repository adds a new, separately trained edge-refinement network and its input pipeline. The CorridorKey 2.0 weights and code are not modified or redistributed.
- Example footage: CorridorKey 1.0 Quality Benchmark (https://huggingface.co/datasets/nikopueringer/CorridorKey1.0_QualityBenchmark, Apache-2.0), Pexels (Pexels licence), and the CorridorKey Synthetic Streamers dataset.
- This is an independent community project. It is not affiliated with, sponsored by or endorsed by Corridor Digital LLC.