Spaces:
Running
Running
Commit Β·
e323c6f
1
Parent(s): 4a2b4cf
Enable GPU for InsightFace: auto-add CUDA12 + cuDNN9 DLL dirs on Windows
Browse filesonnxruntime-gpu silently fell back to CPU (LoadLibrary error 126) because
cuDNN 9 keeps its CUDA-12 DLLs in a versioned subfolder (CUDNN\vX\bin\12.x)
that isn't on PATH. detector._ensure_cuda_dlls() auto-discovers the CUDA toolkit
bin + cuDNN 12.x bin and prepends them before importing onnxruntime. Verified:
InsightFace, GFPGAN and RealESRGAN all run on the RTX 4060. No-op on Linux.
+ GPU_ENHANCEMENT_PLAN.md runbook (incl. batch test harness).
- GPU_ENHANCEMENT_PLAN.md +100 -0
- core/detector.py +35 -0
GPU_ENHANCEMENT_PLAN.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GPU Enhancement Runbook β DeepFace Studio
|
| 2 |
+
|
| 3 |
+
Executed once `torch+cu121` **and** `onnxruntime-gpu` are installed. Goal: best
|
| 4 |
+
realistic output, working **both locally (RTX 4060)** and on **HF Pro GPU**.
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## Phase 1 β Verify the GPU stack
|
| 9 |
+
1. `torch.cuda.is_available()` β True, `torch.cuda.get_device_name(0)` β RTX 4060.
|
| 10 |
+
2. `onnxruntime.get_available_providers()` β contains `CUDAExecutionProvider`.
|
| 11 |
+
3. Load InsightFace and confirm it initialises on `CUDAExecutionProvider`
|
| 12 |
+
(check `_ORT_PROVIDERS` in `core/detector.py` is `[CUDA, CPU]`).
|
| 13 |
+
4. Confirm `core/super_res.py::_device()` and `core/head_swap.py::_device()`
|
| 14 |
+
both return `cuda`.
|
| 15 |
+
5. **Fail-safe**: if onnxruntime-gpu can't find cuDNN, install matching cuDNN 9
|
| 16 |
+
or pin `onnxruntime-gpu==1.19.2` (CUDA 12 + cuDNN 9).
|
| 17 |
+
|
| 18 |
+
## Phase 2 β Render + benchmark a real swap on GPU
|
| 19 |
+
1. Run the full pipeline (head β face β GFPGAN β skin-match β hair β blend β
|
| 20 |
+
RealESRGAN Γ4) on a real source + a location target.
|
| 21 |
+
2. Save the output, **view it**, and time it (expect a few seconds vs ~60s CPU).
|
| 22 |
+
3. Confirm only the main subject is swapped (no background/poster faces).
|
| 23 |
+
|
| 24 |
+
## Phase 3 β Quality tuning (now that there's GPU headroom)
|
| 25 |
+
1. **Working resolution**: raise `resize_keep_aspect(..., 1024)` in `web_app.py`
|
| 26 |
+
to **1280β1536** (GPU can handle it β sharper swap, better landmarks).
|
| 27 |
+
2. **RealESRGAN Γ4**: real GPU upscale for the download (currently Lanczos on
|
| 28 |
+
CPU). Tune `realesrgan_weight` (~0.5) so skin isn't plastic.
|
| 29 |
+
3. **GFPGAN**: confirm it runs on GPU; keep `upscale=1` (RealESRGAN does scaling).
|
| 30 |
+
4. **Skin tone**: keep `match_skin_to_source` at 0.75; re-evaluate visually.
|
| 31 |
+
5. **Blend**: keep Laplacian 4-level over the face mask; check no hard seam.
|
| 32 |
+
6. Re-render and visually compare before/after each change (no guessing).
|
| 33 |
+
|
| 34 |
+
## Phase 4 β Real hair swap (HairFastGAN, GPU)
|
| 35 |
+
1. `python scripts/setup_hairfast.py` β clones `external/HairFastGAN` + weights,
|
| 36 |
+
patches ops, sets up the MSVC/CUDA launcher.
|
| 37 |
+
2. `pip install git+https://github.com/openai/CLIP.git face_alignment lpips kornia`.
|
| 38 |
+
3. Set `HAIRFAST_LOCAL=1`; `core/hair_transfer.py` auto-uses the local GPU path.
|
| 39 |
+
4. Wire it into the pipeline: after the face swap, transfer the **source's
|
| 40 |
+
hairstyle** onto the result (real hair, not the BiSeNet warp).
|
| 41 |
+
5. Render + view; first call is slow (op compile), then ~15β20s on the 4060.
|
| 42 |
+
6. **VRAM guard**: 8GB is tight β run HairFastGAN, free InsightFace/GFPGAN
|
| 43 |
+
between stages if needed; fall back gracefully if OOM.
|
| 44 |
+
|
| 45 |
+
## Phase 5 β Automated test harness (the "100 tests per location")
|
| 46 |
+
Feasible on GPU (~3-5s/swap). 100 Γ 19 locations Γ 2 genders β 3,800 swaps β
|
| 47 |
+
3-5 hrs on the RTX 4060 β run as a batch job.
|
| 48 |
+
|
| 49 |
+
**5a. Source face dataset**
|
| 50 |
+
- Need a folder of diverse SOURCE faces: light / medium / deep skin tones,
|
| 51 |
+
round / oval / long face shapes, short / long / curly hair, with / without
|
| 52 |
+
glasses and beard, balanced male + female.
|
| 53 |
+
- Source options: a face dataset (e.g. FFHQ samples / a provided folder).
|
| 54 |
+
Drop them in `tests/faces/male/` and `tests/faces/female/`. The harness
|
| 55 |
+
scales to however many are provided (10, 50, 100β¦).
|
| 56 |
+
|
| 57 |
+
**5b. Build `scripts/batch_test.py`**
|
| 58 |
+
- For each gender β each location β each source face: run the full GPU pipeline.
|
| 59 |
+
- Save outputs to `tests/output/<gender>/<location>/<source>.jpg`.
|
| 60 |
+
- Build a **contact sheet (grid)** per location for quick visual QA.
|
| 61 |
+
- Compute metrics per swap via `core/quality_checker.py`
|
| 62 |
+
(alignment, blend, ΞE, naturalness) β write `tests/report.csv`.
|
| 63 |
+
|
| 64 |
+
**5c. Analyse + tune**
|
| 65 |
+
- Aggregate metrics: flag locations / source-types with low scores
|
| 66 |
+
(e.g. deep skin tone + bright location, or long-hair sources).
|
| 67 |
+
- **View** the worst cases; tune the specific stage (skin-match strength,
|
| 68 |
+
blend, head-swap region, hair) and re-run that subset.
|
| 69 |
+
- Iterate until the score distribution + visual quality are consistently good
|
| 70 |
+
across all skin tones, face shapes and hairstyles.
|
| 71 |
+
|
| 72 |
+
**5d. Reality on "fine-tune the model itself"**
|
| 73 |
+
- The `inswapper_128` weights can't be fine-tuned here (no training pipeline /
|
| 74 |
+
dataset / time). What we *can* tune is the **pipeline around it** (resolution,
|
| 75 |
+
head/face/hair order, skin-match, blend, RealESRGAN, HairFastGAN). The harness
|
| 76 |
+
drives that tuning with real numbers + visuals. For a genuinely different
|
| 77 |
+
identity model, the upgrade path is a stronger swapper (e.g. SimSwap-HQ /
|
| 78 |
+
inswapper-512 / a GPU diffusion swapper) β also GPU-only.
|
| 79 |
+
|
| 80 |
+
## Phase 6 β GPU deploy for HF Pro
|
| 81 |
+
1. `requirements-gpu.txt`: `torch==2.3.0+cu121`, `torchvision==0.18.0+cu121`
|
| 82 |
+
(cu121 index), `onnxruntime-gpu==1.19.2`, + the rest of the deps.
|
| 83 |
+
2. `Dockerfile.gpu`: base `nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04`
|
| 84 |
+
(CUDA 12 + cuDNN 9), install Python 3.10, torch cu121, onnxruntime-gpu, deps,
|
| 85 |
+
build the React frontend (stage 1), copy app, `CMD ./startup.sh`.
|
| 86 |
+
3. README: note "switch HF Space to GPU hardware + use Dockerfile.gpu".
|
| 87 |
+
4. Keep the CPU `Dockerfile` as fallback so nothing breaks before the upgrade.
|
| 88 |
+
|
| 89 |
+
## Phase 7 β Ship
|
| 90 |
+
1. `npm run build` (frontend), syntax-check backend, run `pytest`.
|
| 91 |
+
2. Commit + push; confirm CI green and HF deploy green.
|
| 92 |
+
3. On HF: switch Space to **GPU hardware**, point to `Dockerfile.gpu`, set the
|
| 93 |
+
existing secrets; verify a live GPU swap.
|
| 94 |
+
|
| 95 |
+
---
|
| 96 |
+
|
| 97 |
+
### Guardrails
|
| 98 |
+
- **Verify every change by rendering + viewing the image** β no blind tuning.
|
| 99 |
+
- Keep CPU fallback intact (so the app still runs without a GPU).
|
| 100 |
+
- Don't break the green HF deploy until the GPU Space is confirmed working.
|
core/detector.py
CHANGED
|
@@ -1,7 +1,42 @@
|
|
| 1 |
import os
|
|
|
|
|
|
|
| 2 |
import cv2
|
| 3 |
import cv2.data
|
| 4 |
import numpy as np
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
try:
|
| 6 |
import onnxruntime as ort
|
| 7 |
_ORT_PROVIDERS = (
|
|
|
|
| 1 |
import os
|
| 2 |
+
import sys
|
| 3 |
+
import glob
|
| 4 |
import cv2
|
| 5 |
import cv2.data
|
| 6 |
import numpy as np
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def _ensure_cuda_dlls():
|
| 10 |
+
"""
|
| 11 |
+
Windows: make CUDA 12 + cuDNN 9 DLLs discoverable so onnxruntime-gpu can load
|
| 12 |
+
CUDAExecutionProvider β otherwise it silently falls back to CPU (LoadLibrary
|
| 13 |
+
error 126). cuDNN 9 ships its CUDA-12 DLLs in a versioned subfolder
|
| 14 |
+
(β¦\\CUDNN\\vX\\bin\\12.x) that isn't on PATH by default. We auto-discover the
|
| 15 |
+
CUDA-toolkit bin + the cuDNN 12.x bin and prepend them to PATH.
|
| 16 |
+
No-op on Linux (the HF Docker/CUDA base image already puts CUDA on the path).
|
| 17 |
+
"""
|
| 18 |
+
if sys.platform != "win32":
|
| 19 |
+
return
|
| 20 |
+
pf = os.environ.get("ProgramFiles", r"C:\Program Files")
|
| 21 |
+
patterns = [
|
| 22 |
+
os.path.join(pf, "NVIDIA GPU Computing Toolkit", "CUDA", "v12*", "bin"),
|
| 23 |
+
os.path.join(pf, "NVIDIA", "CUDNN", "v*", "bin", "12*"), # cuDNN9 for CUDA12
|
| 24 |
+
os.path.join(pf, "NVIDIA", "CUDNN", "v*", "bin"),
|
| 25 |
+
]
|
| 26 |
+
added = set()
|
| 27 |
+
for pat in patterns:
|
| 28 |
+
for d in sorted(glob.glob(pat), reverse=True): # newest version first
|
| 29 |
+
if os.path.isdir(d) and d not in added:
|
| 30 |
+
added.add(d)
|
| 31 |
+
os.environ["PATH"] = d + os.pathsep + os.environ.get("PATH", "")
|
| 32 |
+
try:
|
| 33 |
+
os.add_dll_directory(d)
|
| 34 |
+
except Exception:
|
| 35 |
+
pass
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
_ensure_cuda_dlls()
|
| 39 |
+
|
| 40 |
try:
|
| 41 |
import onnxruntime as ort
|
| 42 |
_ORT_PROVIDERS = (
|