Image Segmentation
ONNX
MLX
onnxruntime
torch-tensorrt
matting
keying
vfx
green-screen
blue-screen
corridorkey
Instructions to use alexandrealvaro/CorridorKey with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use alexandrealvaro/CorridorKey with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir CorridorKey alexandrealvaro/CorridorKey
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
docs: rewrite README as spec (drop devlog narrative, focus on what each variant is)
Browse files
README.md
CHANGED
|
@@ -15,122 +15,116 @@ tags:
|
|
| 15 |
pipeline_tag: image-segmentation
|
| 16 |
---
|
| 17 |
|
| 18 |
-
# CorridorKey —
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
Silicon, plus the auxiliary tracker fixtures).
|
| 26 |
|
| 27 |
## Upstream sources
|
| 28 |
|
| 29 |
-
|
| 30 |
-
here
|
| 31 |
|
| 32 |
-
| Variant | Upstream
|
| 33 |
|---|---|
|
| 34 |
-
| Green
|
| 35 |
-
| Blue
|
| 36 |
|
| 37 |
-
|
| 38 |
|
| 39 |
-
##
|
| 40 |
|
| 41 |
-
|
| 42 |
-
CorridorKey Runtime build pipeline. Re-exports are deterministic; if
|
| 43 |
-
upstream re-trains, regenerate the artefacts and bump the runtime
|
| 44 |
-
version that consumes them.
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
| File | Resolution | Approx size |
|
| 49 |
|---|---|---|
|
| 50 |
| `onnx/fp16/corridorkey_fp16_512.onnx` | 512 × 512 | ~145 MB |
|
| 51 |
| `onnx/fp16/corridorkey_fp16_1024.onnx` | 1024 × 1024 | ~156 MB |
|
| 52 |
| `onnx/fp16/corridorkey_fp16_1536.onnx` | 1536 × 1536 | ~174 MB |
|
| 53 |
| `onnx/fp16/corridorkey_fp16_2048.onnx` | 2048 × 2048 | ~200 MB |
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
|
| 58 |
-
the
|
| 59 |
-
|
| 60 |
|
| 61 |
-
| File | Resolution |
|
| 62 |
|---|---|---|
|
| 63 |
| `onnx/fp16_ctx/corridorkey_fp16_512_ctx.onnx` | 512 × 512 | ~167 MB |
|
| 64 |
| `onnx/fp16_ctx/corridorkey_fp16_1024_ctx.onnx` | 1024 × 1024 | ~178 MB |
|
| 65 |
| `onnx/fp16_ctx/corridorkey_fp16_1536_ctx.onnx` | 1536 × 1536 | ~197 MB |
|
| 66 |
| `onnx/fp16_ctx/corridorkey_fp16_2048_ctx.onnx` | 2048 × 2048 | ~223 MB |
|
| 67 |
|
| 68 |
-
##
|
| 69 |
|
| 70 |
-
Quantised
|
| 71 |
-
|
| 72 |
-
|
| 73 |
|
| 74 |
-
| File | Resolution |
|
| 75 |
|---|---|---|
|
| 76 |
| `onnx/int8/corridorkey_int8_512.onnx` | 512 × 512 | ~79 MB |
|
| 77 |
| `onnx/int8/corridorkey_int8_768.onnx` | 768 × 768 | ~88 MB |
|
| 78 |
| `onnx/int8/corridorkey_int8_1024.onnx` | 1024 × 1024 | ~101 MB |
|
| 79 |
|
| 80 |
-
##
|
| 81 |
|
| 82 |
Pre-compiled TorchScript engines (Torch-TensorRT 2.8.0 + TensorRT 10.12,
|
| 83 |
-
CUDA 12.8). Loaded
|
| 84 |
-
|
| 85 |
-
|
| 86 |
|
| 87 |
-
| File | Resolution | Precision |
|
| 88 |
|---|---|---|---|
|
| 89 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_512.ts` | 512 × 512 | FP16 | ~408 MB |
|
| 90 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_1024.ts` | 1024 × 1024 | FP16 | ~430 MB |
|
| 91 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_1536.ts` | 1536 × 1536 | FP16 | ~502 MB |
|
| 92 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_2048.ts` | 2048 × 2048 | FP16 | ~558 MB |
|
| 93 |
|
| 94 |
-
##
|
|
|
|
|
|
|
| 95 |
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
|
| 101 |
-
| File | Resolution | Precision |
|
| 102 |
|---|---|---|---|
|
| 103 |
-
| `torchtrt/fp16-blue/corridorkey_blue_torchtrt_fp16_512.ts` | 512 × 512 | FP16 |
|
| 104 |
| `torchtrt/fp16-blue/corridorkey_blue_torchtrt_fp16_1024.ts` | 1024 × 1024 | FP16 | ~258 MB |
|
| 105 |
| `torchtrt/fp32-blue/corridorkey_blue_torchtrt_fp32_1536.ts` | 1536 × 1536 | FP32 | ~488 MB |
|
| 106 |
-
| `torchtrt/fp32-blue/corridorkey_blue_torchtrt_fp32_2048.ts` | 2048 × 2048 | FP32 |
|
| 107 |
|
| 108 |
-
##
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
|
|
|
|
|
|
| 112 |
|
| 113 |
-
| File | Description |
|
| 114 |
|---|---|---|
|
| 115 |
-
| `mlx/corridorkey_mlx.safetensors` | Re-exported base weights
|
| 116 |
| `mlx/corridorkey_mlx_bridge_512.mlxfn` | 512 × 512 bridge | ~289 MB |
|
| 117 |
| `mlx/corridorkey_mlx_bridge_768.mlxfn` | 768 × 768 bridge | ~298 MB |
|
| 118 |
| `mlx/corridorkey_mlx_bridge_1024.mlxfn` | 1024 × 1024 bridge | ~311 MB |
|
| 119 |
| `mlx/corridorkey_mlx_bridge_1536.mlxfn` | 1536 × 1536 bridge | ~348 MB |
|
| 120 |
| `mlx/corridorkey_mlx_bridge_2048.mlxfn` | 2048 × 2048 bridge | ~399 MB |
|
| 121 |
|
| 122 |
-
|
| 123 |
-
expected layout — it differs byte-wise from the upstream
|
| 124 |
-
`nikopueringer/CorridorKey_v1.0.safetensors` and is required by the MLX
|
| 125 |
-
bridges below it.
|
| 126 |
|
| 127 |
-
|
|
|
|
|
|
|
|
|
|
| 128 |
|
| 129 |
-
|
| 130 |
-
from the upstream MobileSAM and Cutie projects under their own licences
|
| 131 |
-
(see model cards on Hugging Face).
|
| 132 |
-
|
| 133 |
-
| File | Approx size |
|
| 134 |
|---|---|
|
| 135 |
| `hint/mobilesam_image_encoder_fp16.onnx` | ~14 MB |
|
| 136 |
| `hint/mobilesam_prompt_decoder_fp16.onnx` | ~8 MB |
|
|
@@ -138,52 +132,25 @@ from the upstream MobileSAM and Cutie projects under their own licences
|
|
| 138 |
| `hint/cutie_init_memory_fp16.onnx` | ~1 MB |
|
| 139 |
| `hint/cutie_tracking_memory_fp16.onnx` | ~1 MB |
|
| 140 |
|
| 141 |
-
##
|
| 142 |
-
|
| 143 |
-
Standalone CorridorKey Runtime installer artefacts (Windows OFX plugin
|
| 144 |
-
bundle for DaVinci Resolve / Foundry Nuke). Each subdirectory under
|
| 145 |
-
`releases/` is a single tagged build of the
|
| 146 |
-
[CorridorKey-Runtime repo](https://github.com/alexandremendoncaalvaro/CorridorKey-Runtime).
|
| 147 |
-
|
| 148 |
-
## What this repo deliberately does NOT publish
|
| 149 |
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
Windows blue pack) — those ship inside the OFX installer artefact,
|
| 155 |
-
not as a model file.
|
| 156 |
|
| 157 |
## Usage
|
| 158 |
|
| 159 |
```bash
|
| 160 |
-
# Single
|
| 161 |
hf download alexandrealvaro/CorridorKey onnx/fp16/corridorkey_fp16_1024.onnx
|
| 162 |
|
| 163 |
-
# All ONNX FP16
|
| 164 |
hf download alexandrealvaro/CorridorKey --include "onnx/fp16/*.onnx"
|
| 165 |
|
| 166 |
-
# All Torch-
|
| 167 |
hf download alexandrealvaro/CorridorKey --include "torchtrt/**/*.ts"
|
| 168 |
|
| 169 |
# Direct HTTP
|
| 170 |
-
# https://huggingface.co/alexandrealvaro/CorridorKey/resolve/main/
|
| 171 |
```
|
| 172 |
-
|
| 173 |
-
The CorridorKey Runtime ships a wrapper that pulls the right subset for
|
| 174 |
-
each platform: `scripts/fetch_models.ps1 -Profile windows-rtx`,
|
| 175 |
-
`-Profile windows-rtx-blue`, `-Profile apple`, etc.
|
| 176 |
-
|
| 177 |
-
## Runtime
|
| 178 |
-
|
| 179 |
-
Models above are consumed by
|
| 180 |
-
[CorridorKey Runtime](https://github.com/alexandremendoncaalvaro/CorridorKey-Runtime),
|
| 181 |
-
shipped as:
|
| 182 |
-
|
| 183 |
-
- OFX plugin for DaVinci Resolve and Foundry Nuke (Windows RTX, macOS Apple Silicon)
|
| 184 |
-
- CLI for offline batch processing
|
| 185 |
-
- Browser demo (uses the INT8 ONNX rung above)
|
| 186 |
-
|
| 187 |
-
## License
|
| 188 |
-
|
| 189 |
-
CC BY-NC-SA 4.0 (matches upstream).
|
|
|
|
| 15 |
pipeline_tag: image-segmentation
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# CorridorKey — model variants
|
| 19 |
|
| 20 |
+
Ready-to-load inference variants of the CorridorKey chroma-keying network
|
| 21 |
+
in the formats common runtimes consume: ONNX (FP16 + INT8), Torch-TensorRT
|
| 22 |
+
TorchScript engines, and MLX safetensors + bridges for Apple Silicon.
|
| 23 |
+
|
| 24 |
+
License: **CC BY-NC-SA 4.0**.
|
|
|
|
| 25 |
|
| 26 |
## Upstream sources
|
| 27 |
|
| 28 |
+
Training checkpoints (`.pth`, base `.safetensors`) are not republished
|
| 29 |
+
here; fetch them from the original repos:
|
| 30 |
|
| 31 |
+
| Variant | Upstream |
|
| 32 |
|---|---|
|
| 33 |
+
| Green | [`nikopueringer/CorridorKey_v1.0`](https://huggingface.co/nikopueringer/CorridorKey_v1.0) |
|
| 34 |
+
| Blue | [`nikopueringer/CorridorKeyBlue_1.0`](https://huggingface.co/nikopueringer/CorridorKeyBlue_1.0) |
|
| 35 |
|
| 36 |
+
Everything in this repo is derived from those checkpoints.
|
| 37 |
|
| 38 |
+
## ONNX FP16
|
| 39 |
|
| 40 |
+
Static-shape models for ONNX Runtime (TensorRT, DirectML, CUDA, CPU EPs).
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
| File | Resolution | Size |
|
|
|
|
|
|
|
| 43 |
|---|---|---|
|
| 44 |
| `onnx/fp16/corridorkey_fp16_512.onnx` | 512 × 512 | ~145 MB |
|
| 45 |
| `onnx/fp16/corridorkey_fp16_1024.onnx` | 1024 × 1024 | ~156 MB |
|
| 46 |
| `onnx/fp16/corridorkey_fp16_1536.onnx` | 1536 × 1536 | ~174 MB |
|
| 47 |
| `onnx/fp16/corridorkey_fp16_2048.onnx` | 2048 × 2048 | ~200 MB |
|
| 48 |
|
| 49 |
+
## ONNX FP16 context cache
|
| 50 |
|
| 51 |
+
Companion `_ctx.onnx` files for ONNX Runtime's TensorRT EP. When paired
|
| 52 |
+
with the matching FP16 model, the EP loads a pre-built TensorRT engine
|
| 53 |
+
instead of compiling on first run. Pair by resolution.
|
| 54 |
|
| 55 |
+
| File | Resolution | Size |
|
| 56 |
|---|---|---|
|
| 57 |
| `onnx/fp16_ctx/corridorkey_fp16_512_ctx.onnx` | 512 × 512 | ~167 MB |
|
| 58 |
| `onnx/fp16_ctx/corridorkey_fp16_1024_ctx.onnx` | 1024 × 1024 | ~178 MB |
|
| 59 |
| `onnx/fp16_ctx/corridorkey_fp16_1536_ctx.onnx` | 1536 × 1536 | ~197 MB |
|
| 60 |
| `onnx/fp16_ctx/corridorkey_fp16_2048_ctx.onnx` | 2048 × 2048 | ~223 MB |
|
| 61 |
|
| 62 |
+
## ONNX INT8
|
| 63 |
|
| 64 |
+
Quantised FP16 → INT8 models. Lower memory footprint, lower precision
|
| 65 |
+
than FP16; useful where GPU memory or model size matters more than
|
| 66 |
+
edge fidelity (browsers, lower-tier hardware).
|
| 67 |
|
| 68 |
+
| File | Resolution | Size |
|
| 69 |
|---|---|---|
|
| 70 |
| `onnx/int8/corridorkey_int8_512.onnx` | 512 × 512 | ~79 MB |
|
| 71 |
| `onnx/int8/corridorkey_int8_768.onnx` | 768 × 768 | ~88 MB |
|
| 72 |
| `onnx/int8/corridorkey_int8_1024.onnx` | 1024 × 1024 | ~101 MB |
|
| 73 |
|
| 74 |
+
## Torch-TensorRT — green
|
| 75 |
|
| 76 |
Pre-compiled TorchScript engines (Torch-TensorRT 2.8.0 + TensorRT 10.12,
|
| 77 |
+
CUDA 12.8 toolchain). Loaded via `torch::jit::load`. The runtime
|
| 78 |
+
process must have the matching `torch_tensorrt` runtime DLLs reachable
|
| 79 |
+
to the OS loader.
|
| 80 |
|
| 81 |
+
| File | Resolution | Precision | Size |
|
| 82 |
|---|---|---|---|
|
| 83 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_512.ts` | 512 × 512 | FP16 | ~408 MB |
|
| 84 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_1024.ts` | 1024 × 1024 | FP16 | ~430 MB |
|
| 85 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_1536.ts` | 1536 × 1536 | FP16 | ~502 MB |
|
| 86 |
| `torchtrt/fp16/corridorkey_torchtrt_fp16_2048.ts` | 2048 × 2048 | FP16 | ~558 MB |
|
| 87 |
|
| 88 |
+
## Torch-TensorRT — blue
|
| 89 |
+
|
| 90 |
+
Same toolchain as green, dedicated to the CorridorKeyBlue checkpoint.
|
| 91 |
|
| 92 |
+
**Precision split:** FP16 for 512 / 1024, FP32 for 1536 / 2048. The
|
| 93 |
+
blue checkpoint produces NaN activations at higher resolutions when
|
| 94 |
+
compiled at FP16; FP32 above 1024 is the workaround until the
|
| 95 |
+
checkpoint is retrained.
|
| 96 |
|
| 97 |
+
| File | Resolution | Precision | Size |
|
| 98 |
|---|---|---|---|
|
| 99 |
+
| `torchtrt/fp16-blue/corridorkey_blue_torchtrt_fp16_512.ts` | 512 × 512 | FP16 | _not yet published_ |
|
| 100 |
| `torchtrt/fp16-blue/corridorkey_blue_torchtrt_fp16_1024.ts` | 1024 × 1024 | FP16 | ~258 MB |
|
| 101 |
| `torchtrt/fp32-blue/corridorkey_blue_torchtrt_fp32_1536.ts` | 1536 × 1536 | FP32 | ~488 MB |
|
| 102 |
+
| `torchtrt/fp32-blue/corridorkey_blue_torchtrt_fp32_2048.ts` | 2048 × 2048 | FP32 | _not yet published_ |
|
| 103 |
|
| 104 |
+
## MLX — Apple Silicon
|
| 105 |
|
| 106 |
+
Safetensors weights for the MLX framework plus per-resolution
|
| 107 |
+
pre-compiled bridges. The `.safetensors` here is re-exported for the
|
| 108 |
+
MLX layout and differs byte-wise from the upstream
|
| 109 |
+
`CorridorKey_v1.0.safetensors`; use it together with the bridges below.
|
| 110 |
|
| 111 |
+
| File | Description | Size |
|
| 112 |
|---|---|---|
|
| 113 |
+
| `mlx/corridorkey_mlx.safetensors` | Re-exported base weights | ~399 MB |
|
| 114 |
| `mlx/corridorkey_mlx_bridge_512.mlxfn` | 512 × 512 bridge | ~289 MB |
|
| 115 |
| `mlx/corridorkey_mlx_bridge_768.mlxfn` | 768 × 768 bridge | ~298 MB |
|
| 116 |
| `mlx/corridorkey_mlx_bridge_1024.mlxfn` | 1024 × 1024 bridge | ~311 MB |
|
| 117 |
| `mlx/corridorkey_mlx_bridge_1536.mlxfn` | 1536 × 1536 bridge | ~348 MB |
|
| 118 |
| `mlx/corridorkey_mlx_bridge_2048.mlxfn` | 2048 × 2048 bridge | ~399 MB |
|
| 119 |
|
| 120 |
+
## Hint trackers
|
|
|
|
|
|
|
|
|
|
| 121 |
|
| 122 |
+
Auxiliary models used by alpha-hint generation pipelines (interactive
|
| 123 |
+
matte refinement, segment-then-track flows). Exported from
|
| 124 |
+
[MobileSAM](https://github.com/ChaoningZhang/MobileSAM) and
|
| 125 |
+
[Cutie](https://github.com/hkchengrex/Cutie) under their own licences.
|
| 126 |
|
| 127 |
+
| File | Size |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
|---|---|
|
| 129 |
| `hint/mobilesam_image_encoder_fp16.onnx` | ~14 MB |
|
| 130 |
| `hint/mobilesam_prompt_decoder_fp16.onnx` | ~8 MB |
|
|
|
|
| 132 |
| `hint/cutie_init_memory_fp16.onnx` | ~1 MB |
|
| 133 |
| `hint/cutie_tracking_memory_fp16.onnx` | ~1 MB |
|
| 134 |
|
| 135 |
+
## Releases
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
|
| 137 |
+
`releases/v*/` contains tagged builds of the
|
| 138 |
+
[CorridorKey Runtime](https://github.com/alexandremendoncaalvaro/CorridorKey-Runtime)
|
| 139 |
+
OFX plugin and CLI for Windows, packaged with the model subset each
|
| 140 |
+
build expects.
|
|
|
|
|
|
|
| 141 |
|
| 142 |
## Usage
|
| 143 |
|
| 144 |
```bash
|
| 145 |
+
# Single file
|
| 146 |
hf download alexandrealvaro/CorridorKey onnx/fp16/corridorkey_fp16_1024.onnx
|
| 147 |
|
| 148 |
+
# All ONNX FP16
|
| 149 |
hf download alexandrealvaro/CorridorKey --include "onnx/fp16/*.onnx"
|
| 150 |
|
| 151 |
+
# All Torch-TensorRT (green + blue)
|
| 152 |
hf download alexandrealvaro/CorridorKey --include "torchtrt/**/*.ts"
|
| 153 |
|
| 154 |
# Direct HTTP
|
| 155 |
+
# https://huggingface.co/alexandrealvaro/CorridorKey/resolve/main/<path>
|
| 156 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|