Nekochu commited on
Commit
69eef34
·
1 Parent(s): c9b234a

update README: fix batch size, add GPU optimization docs

Browse files

Speed comparison (264x480 not directly comparable to 4K):
- OLD (2026-04-30): 264x480, batch 32, 0.21s/fr, TIMED OUT at 120s (33% done)
- NEW (2026-05-02): 3840x2160 4K, batch 32, 0.32s/fr, COMPLETED in 49.2s

Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -31,14 +31,21 @@ Based on [CorridorKey](https://github.com/nikopueringer/CorridorKey) by Corridor
31
 
32
  ## Inference Paths
33
 
34
- - **GPU (ZeroGPU H200)**: PyTorch GreenFormer with batched inference (batch 32 at 1024, batch 8 at 2048)
35
  - **CPU (fallback)**: ONNX Runtime sequential inference (batch 1)
36
 
37
  ## Pipeline
38
 
39
- 1. **BiRefNet** - Generates coarse foreground mask (ONNX)
40
  2. **CorridorKey GreenFormer** - Refines alpha matte + extracts clean foreground (PyTorch on GPU, ONNX on CPU)
41
- 3. **Compositing** - Despill, despeckle, composite on new background
 
 
 
 
 
 
 
42
 
43
  ## API
44
 
 
31
 
32
  ## Inference Paths
33
 
34
+ - **GPU (ZeroGPU H200)**: PyTorch GreenFormer with batched inference (batch 32 at 1024, batch 16 at 2048)
35
  - **CPU (fallback)**: ONNX Runtime sequential inference (batch 1)
36
 
37
  ## Pipeline
38
 
39
+ 1. **BiRefNet** - Generates coarse foreground mask (ONNX, or fast classical HSV for green screens)
40
  2. **CorridorKey GreenFormer** - Refines alpha matte + extracts clean foreground (PyTorch on GPU, ONNX on CPU)
41
+ 3. **GPU Postprocessing** - Despill, despeckle (connected components), resize — all on GPU via torchvision, single CPU transfer at the end
42
+
43
+ ## GPU Optimizations
44
+
45
+ - **Full GPU pipeline**: preprocessing (resize + normalize) and postprocessing (despill, clean_matte, resize) stay on device — avoids CPU↔GPU round-trips per batch
46
+ - **TF32 tensor cores**: `torch.set_float32_matmul_precision('high')` for FP32 postprocessing ops
47
+ - **Conditional torch.compile**: on warm ZeroGPU containers, cached torch.inductor kernels enable compiled inference (~0s warmup). Cold starts skip compile (37s warmup exceeds 120s GPU budget). AOTI compilation with torch.inductor + triton cudagraphs gives 1.5-2x speed: native CUDA kernels, fused operations, no CPU-GPU sync overhead
48
+ - **Inductor cache persistence**: `TORCHINDUCTOR_CACHE_DIR` set so compiled kernels survive across sessions
49
 
50
  ## API
51