Spaces:
Running on Zero
Running on Zero
update README: fix batch size, add GPU optimization docs
Browse filesSpeed comparison (264x480 not directly comparable to 4K):
- OLD (2026-04-30): 264x480, batch 32, 0.21s/fr, TIMED OUT at 120s (33% done)
- NEW (2026-05-02): 3840x2160 4K, batch 32, 0.32s/fr, COMPLETED in 49.2s
README.md
CHANGED
|
@@ -31,14 +31,21 @@ Based on [CorridorKey](https://github.com/nikopueringer/CorridorKey) by Corridor
|
|
| 31 |
|
| 32 |
## Inference Paths
|
| 33 |
|
| 34 |
-
- **GPU (ZeroGPU H200)**: PyTorch GreenFormer with batched inference (batch 32 at 1024, batch
|
| 35 |
- **CPU (fallback)**: ONNX Runtime sequential inference (batch 1)
|
| 36 |
|
| 37 |
## Pipeline
|
| 38 |
|
| 39 |
-
1. **BiRefNet** - Generates coarse foreground mask (ONNX)
|
| 40 |
2. **CorridorKey GreenFormer** - Refines alpha matte + extracts clean foreground (PyTorch on GPU, ONNX on CPU)
|
| 41 |
-
3. **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
## API
|
| 44 |
|
|
|
|
| 31 |
|
| 32 |
## Inference Paths
|
| 33 |
|
| 34 |
+
- **GPU (ZeroGPU H200)**: PyTorch GreenFormer with batched inference (batch 32 at 1024, batch 16 at 2048)
|
| 35 |
- **CPU (fallback)**: ONNX Runtime sequential inference (batch 1)
|
| 36 |
|
| 37 |
## Pipeline
|
| 38 |
|
| 39 |
+
1. **BiRefNet** - Generates coarse foreground mask (ONNX, or fast classical HSV for green screens)
|
| 40 |
2. **CorridorKey GreenFormer** - Refines alpha matte + extracts clean foreground (PyTorch on GPU, ONNX on CPU)
|
| 41 |
+
3. **GPU Postprocessing** - Despill, despeckle (connected components), resize — all on GPU via torchvision, single CPU transfer at the end
|
| 42 |
+
|
| 43 |
+
## GPU Optimizations
|
| 44 |
+
|
| 45 |
+
- **Full GPU pipeline**: preprocessing (resize + normalize) and postprocessing (despill, clean_matte, resize) stay on device — avoids CPU↔GPU round-trips per batch
|
| 46 |
+
- **TF32 tensor cores**: `torch.set_float32_matmul_precision('high')` for FP32 postprocessing ops
|
| 47 |
+
- **Conditional torch.compile**: on warm ZeroGPU containers, cached torch.inductor kernels enable compiled inference (~0s warmup). Cold starts skip compile (37s warmup exceeds 120s GPU budget). AOTI compilation with torch.inductor + triton cudagraphs gives 1.5-2x speed: native CUDA kernels, fused operations, no CPU-GPU sync overhead
|
| 48 |
+
- **Inductor cache persistence**: `TORCHINDUCTOR_CACHE_DIR` set so compiled kernels survive across sessions
|
| 49 |
|
| 50 |
## API
|
| 51 |
|