aleph65 commited on
Commit
a593adc
·
verified ·
1 Parent(s): 4b8218a

makeup-extract: install + run scripts for Makeup Extraction of 3D Representation (EG2023) (single photo -> FLAME UV materials + extracted makeup layers)

Browse files
makeup-extract/README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # makeup-extract — 3D makeup extraction from a single photo
2
+
3
+ Support scripts for [YangXingchao/makeup-extract](https://github.com/YangXingchao/makeup-extract)
4
+ (*Makeup Extraction of 3D Representation via Illumination-Aware Image
5
+ Decomposition*, Yang, Taketomi, Kanamori — Computer Graphics Forum / Eurographics 2023).
6
+
7
+ From one face photo the pipeline reconstructs a coarse FLAME face, unwraps and
8
+ completes the UV texture, decomposes it into materials (diffuse albedo, normal,
9
+ diffuse/specular shading) and extracts the **makeup as a separate layer**:
10
+
11
+ | output | meaning |
12
+ |---|---|
13
+ | `bare_skin.png` | UV albedo with makeup removed |
14
+ | `make_base.png` | makeup base color (UV) |
15
+ | `make_alpha.png` | makeup alpha/opacity matte (UV, white = no makeup) |
16
+ | `make_blend.png` | premultiplied makeup layer (`base * alpha`, UV) |
17
+ | `albe_refine.png`, `norm_refine.png`, `diff_refine.png`, `rnsr_refine.png`, `tex_refine.png`, `completion.png` | refined UV materials / completed texture |
18
+ | `img_compose.png`, `img_compose_with_bg.png` | face re-rendered from the decomposed materials (sanity check — should resemble the aligned input) |
19
+
20
+ ## Install
21
+
22
+ ```bash
23
+ ./makeup_extract_install.sh
24
+ ```
25
+
26
+ One-shot, idempotent, installs into the **native** python env (no venv; a pip
27
+ constraints file protects torch/numpy/etc. from being touched). Tested on
28
+ Ubuntu 24.04, python 3.12, torch 2.8.0+cu128, H100 (image `plx1029/comfyui-qwen`).
29
+
30
+ What it does:
31
+
32
+ - apt-installs GL/EGL dev headers (`libglvnd-dev`, mesa dev packages) as a
33
+ safety net for nvdiffrast's GL path. Note: current nvdiffrast (2026)
34
+ deprecates `RasterizeGLContext` and silently maps it to the CUDA
35
+ rasterizer, so the upstream `dr.RasterizeGLContext()` calls run headless
36
+ with no OpenGL involved at all.
37
+ - pip-installs `ninja`, `gdown`, `nvdiffrast` (from GitHub,
38
+ `--no-build-isolation` so it compiles against native torch), and a patched
39
+ `chumpy` (py3.12/numpy2 fixes; only needed to unpickle FLAME).
40
+ - clones the upstream repo to `$MAKEUP_EXTRACT_HOME/makeup-extract`
41
+ (default `/workspace/makeup-extract/makeup-extract`).
42
+ - downloads weights:
43
+ - pipeline checkpoints (`coarse_reconstruction.pkl`, `uv_completion.pkl`,
44
+ `makeup_extraction.pkl`) from the paper's official Google Drive folder;
45
+ - FLAME assets `generic_model.pkl` (FLAME2020), `FLAME_masks.pkl`,
46
+ `FLAME_texture.npz` — from the **official** site when
47
+ `FLAME_USERNAME`/`FLAME_PASSWORD` are set, otherwise from the public HF
48
+ mirror `camenduru/show`. Either way the
49
+ [FLAME license](https://flame.is.tue.mpg.de/modellicense.html)
50
+ (non-commercial scientific research) applies — register at
51
+ flame.is.tue.mpg.de;
52
+ - `albedoModel2020_FLAME_albedoPart.npz` from the official
53
+ [AlbedoMM](https://github.com/waps101/AlbedoMM) GitHub release (its own
54
+ non-commercial license applies).
55
+
56
+ ## Run
57
+
58
+ ```bash
59
+ ./makeup_extract_run.sh <photo.jpg> [out_dir]
60
+ # e.g.
61
+ ./makeup_extract_run.sh /workspace/face2.jpg
62
+ ```
63
+
64
+ Runs all six upstream steps (preprocess → coarse reconstruction → UV
65
+ completion → material refinement → makeup extraction → re-render). Curated
66
+ outputs land in `/workspace/outputs/makeup-extract/<stem>/`; raw per-step
67
+ outputs are cached in `$MAKEUP_EXTRACT_HOME/results/<stem>/` (delete to
68
+ re-run). ~1–2 min per image on an H100 (step 3 is a 500-iteration
69
+ optimization; first run adds nvdiffrast JIT compile + VGG19/resnet18 aux
70
+ downloads, cached under `weights_cache/torch`).
71
+
72
+ ## Deviations from upstream
73
+
74
+ - **No upstream file is modified.** All adaptation is done via environment and
75
+ wrapper:
76
+ - `TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1` at runtime (repo predates the
77
+ torch ≥2.6 `torch.load(weights_only=True)` default; checkpoints contain
78
+ non-tensor objects).
79
+ - `TORCH_HOME` redirected to `weights_cache/torch` so auto-downloaded VGG19 /
80
+ resnet18 weights survive pod restarts (`/root` is ephemeral).
81
+ - Non-square inputs are padded to square by the run script before step 0
82
+ (upstream's landmark scaling assumes square input).
83
+ - Dependency versions differ from upstream's `requirements.txt` pins
84
+ (numpy 1.23 → 2.1, kornia 0.6 → 0.8, opencv 4.7 → 5.0, scipy native): the
85
+ code is compatible as-is; no `np.int`-era aliases are used. `chumpy` is
86
+ patched for py3.12/numpy2 (`getargspec` → `getfullargspec`, removed-alias
87
+ import) as in the `deca/` implementation.
88
+ - FLAME assets fall back to a public HF mirror when no FLAME credentials are
89
+ provided (see Install above).
90
+
91
+ ## Layout
92
+
93
+ ```
94
+ /workspace/makeup-extract/
95
+ makeup_extract_install.sh this installer
96
+ makeup_extract_run.sh single-image runner
97
+ makeup-extract/ upstream clone + checkpoints/ + resources/
98
+ staging/<stem>/ squared input copy
99
+ results/<stem>/ raw per-step outputs (cache)
100
+ weights_cache/torch/ VGG19 / resnet18 hub cache
101
+ /workspace/outputs/makeup-extract/<stem>/ curated outputs
102
+ ```
makeup-extract/makeup_extract_install.sh ADDED
@@ -0,0 +1,227 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ # =============================================================================
3
+ # makeup_extract_install.sh — one-shot installer for makeup-extract
4
+ # (Eurographics 2023: "Makeup Extraction of 3D Representation via
5
+ # Illumination-Aware Image Decomposition", Yang et al.)
6
+ # single face photo -> FLAME-based UV materials (albedo / normal / shading)
7
+ # + extracted makeup layers (bare skin, makeup base, alpha, blend)
8
+ # + re-rendered composite.
9
+ #
10
+ # Installs https://github.com/YangXingchao/makeup-extract into the NATIVE
11
+ # python environment (no venv, no conda, no torch reinstall). Tested on:
12
+ # Ubuntu 24.04, python 3.12, torch 2.8.0+cu128, H100
13
+ # (RunPod image plx1029/comfyui-qwen).
14
+ #
15
+ # Idempotent: safe to re-run; finished steps are skipped.
16
+ #
17
+ # Requirements:
18
+ # - torch with CUDA already installed (this script will not install torch)
19
+ # - nvcc + gcc (nvdiffrast CUDA/GL plugins JIT-compile on first use)
20
+ # - NVIDIA EGL driver libs (libEGL_nvidia.so.0) for the headless GL
21
+ # rasterizer context — present on this image
22
+ # - FLAME assets (see below): official downloads need FLAME account creds
23
+ # via env vars FLAME_USERNAME / FLAME_PASSWORD
24
+ # (https://flame.is.tue.mpg.de). Without creds the script falls back to
25
+ # a public Hugging Face mirror (camenduru/show). Either way you must
26
+ # comply with the FLAME license (non-commercial scientific research).
27
+ #
28
+ # Downloads:
29
+ # - pretrained pipeline checkpoints (~1 GB, official Google Drive folder)
30
+ # - FLAME2020 generic_model.pkl (~54 MB), FLAME_masks.pkl (~0.2 MB),
31
+ # FLAME_texture.npz (~1.3 GB)
32
+ # - AlbedoMM albedoModel2020_FLAME_albedoPart.npz (~1.8 GB, official
33
+ # GitHub release, CC BY 4.0 non-commercial per AlbedoMM README)
34
+ #
35
+ # Usage:
36
+ # ./makeup_extract_install.sh
37
+ #
38
+ # Layout (override with MAKEUP_EXTRACT_HOME):
39
+ # $MAKEUP_EXTRACT_HOME/makeup-extract upstream clone + weights
40
+ # $MAKEUP_EXTRACT_HOME/staging per-image input staging
41
+ # $MAKEUP_EXTRACT_HOME/results per-image raw outputs
42
+ # $MAKEUP_EXTRACT_HOME/weights_cache torch hub cache (VGG19, resnet18)
43
+ # =============================================================================
44
+ set -euo pipefail
45
+
46
+ if [ "${1:-}" = "--help" ] || [ "${1:-}" = "-h" ]; then grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0; fi
47
+
48
+ M_HOME="${MAKEUP_EXTRACT_HOME:-/workspace/makeup-extract}"
49
+ REPO="$M_HOME/makeup-extract"
50
+
51
+ log() { echo -e "\n\033[1;36m[makeup-extract-install]\033[0m $*"; }
52
+
53
+ mkdir -p "$M_HOME" "$M_HOME/staging" "$M_HOME/results" "$M_HOME/weights_cache"
54
+ cd "$M_HOME"
55
+
56
+ # ---------------------------------------------------------------- sanity checks
57
+ log "checking python/torch/nvcc"
58
+ python3 - <<'EOF'
59
+ import torch
60
+ assert torch.cuda.is_available(), "CUDA torch required"
61
+ print(f" torch {torch.__version__} cuda {torch.version.cuda} ok")
62
+ EOF
63
+ command -v nvcc >/dev/null || echo " WARNING: nvcc not found - nvdiffrast JIT compile will fail at runtime"
64
+
65
+ # ---------------------------------------------------- constraints (protect env)
66
+ # Pin currently-installed shared heavyweights so pip cannot up/downgrade them
67
+ # while installing deps (keeps the ComfyUI env safe).
68
+ log "generating pip constraints from current env"
69
+ python3 - > "$M_HOME/constraints.txt" <<'EOF'
70
+ from importlib.metadata import version, PackageNotFoundError
71
+ for p in ["numpy","torch","torchvision","pillow","scipy","kornia",
72
+ "opencv-python","opencv-python-headless","requests","einops",
73
+ "safetensors","huggingface-hub"]:
74
+ try: print(f"{p}=={version(p)}")
75
+ except PackageNotFoundError: pass
76
+ EOF
77
+ CONS="-c $M_HOME/constraints.txt"
78
+
79
+ # ------------------------------------------------- system GL/EGL dev packages
80
+ # nvdiffrast's GL rasterizer plugin JIT-compiles against EGL/GLES headers.
81
+ # The NVIDIA EGL driver (libEGL_nvidia.so.0) is already on the image; we only
82
+ # add the glvnd dispatcher + headers.
83
+ if ! dpkg -s libglvnd-dev >/dev/null 2>&1; then
84
+ log "installing GL/EGL dev packages (apt)"
85
+ apt-get update -qq
86
+ DEBIAN_FRONTEND=noninteractive apt-get install -y -qq \
87
+ pkg-config libglvnd0 libgl1 libglx0 libegl1 libgles2 \
88
+ libglvnd-dev libgl1-mesa-dev libegl1-mesa-dev libgles2-mesa-dev
89
+ else
90
+ log "GL/EGL dev packages already installed"
91
+ fi
92
+
93
+ # ------------------------------------------------------------------ pip deps
94
+ log "installing python dependencies (native env, constrained)"
95
+ pip install -q $CONS ninja gdown
96
+
97
+ if ! python3 -c "import nvdiffrast" 2>/dev/null; then
98
+ log "installing nvdiffrast from GitHub (compiles CUDA extension against native torch)"
99
+ TORCH_CUDA_ARCH_LIST="$(python3 -c 'import torch; print("%d.%d" % torch.cuda.get_device_capability())')" \
100
+ pip install -q $CONS --no-build-isolation "git+https://github.com/NVlabs/nvdiffrast.git"
101
+ fi
102
+
103
+ # chumpy: only needed to unpickle FLAME. Needs patching for py>=3.11 / numpy>=2.
104
+ if ! python3 -c "import chumpy" 2>/dev/null; then
105
+ log "installing + patching chumpy for py3.12/numpy2"
106
+ pip install -q --no-deps --no-build-isolation chumpy
107
+ CHDIR=$(python3 -c "import importlib.util,os; print(os.path.dirname(importlib.util.find_spec('chumpy').origin))")
108
+ grep -rl "inspect.getargspec" "$CHDIR"/*.py | xargs -r sed -i "s/inspect\.getargspec/inspect.getfullargspec/g"
109
+ sed -i "s/^from numpy import bool, int, float, complex, object, unicode, str, nan, inf$/from numpy import nan, inf/" "$CHDIR/__init__.py"
110
+ python3 -c "import chumpy" || { echo "chumpy patch failed"; exit 1; }
111
+ fi
112
+
113
+ # ------------------------------------------------------------------ repo
114
+ if [ ! -d "$REPO" ]; then
115
+ log "cloning makeup-extract"
116
+ git clone --depth 1 https://github.com/YangXingchao/makeup-extract.git "$REPO"
117
+ fi
118
+
119
+ # -------------------------------------------------------- pipeline checkpoints
120
+ # Official Google Drive folder from the README.
121
+ CKPTS="coarse_reconstruction.pkl uv_completion.pkl makeup_extraction.pkl"
122
+ need_ckpt=0
123
+ for c in $CKPTS; do [ -s "$REPO/checkpoints/$c" ] || need_ckpt=1; done
124
+ if [ "$need_ckpt" = 1 ]; then
125
+ log "downloading pretrained checkpoints (official Google Drive)"
126
+ gdown --folder 1lwkR9JcrbZ7fNylTSJQQEiGnt3s2LQYq -O "$REPO/checkpoints" -q
127
+ fi
128
+ for c in $CKPTS; do
129
+ [ -s "$REPO/checkpoints/$c" ] || { echo "checkpoint $c missing after download"; exit 1; }
130
+ done
131
+
132
+ # ------------------------------------------------------------------ FLAME assets
133
+ RES="$REPO/resources"
134
+
135
+ flame_official() { # $1 = sfile, $2 = out zip
136
+ urle () { local LANG=C i x; for (( i = 0; i < ${#1}; i++ )); do x="${1:i:1}"; [[ "${x}" == [a-zA-Z0-9.~-] ]] && echo -n "${x}" || printf '%%%02X' "'${x}"; done; }
137
+ local U P; U=$(urle "$FLAME_USERNAME"); P=$(urle "$FLAME_PASSWORD")
138
+ wget -q --post-data "username=$U&password=$P" \
139
+ "https://download.is.tue.mpg.de/download.php?domain=flame&sfile=$1&resume=1" \
140
+ -O "$2" --no-check-certificate --continue
141
+ }
142
+
143
+ hf_mirror() { # $1 = path in camenduru/show, $2 = dest file
144
+ python3 - "$1" "$2" <<'EOF'
145
+ import shutil, sys
146
+ from huggingface_hub import hf_hub_download
147
+ src = hf_hub_download("camenduru/show", sys.argv[1])
148
+ shutil.copyfile(src, sys.argv[2])
149
+ EOF
150
+ }
151
+
152
+ HAVE_CREDS=0
153
+ [ -n "${FLAME_USERNAME:-}" ] && [ -n "${FLAME_PASSWORD:-}" ] && HAVE_CREDS=1
154
+
155
+ if [ ! -s "$RES/generic_model.pkl" ]; then
156
+ P3DM_FLAME="/workspace/p3dm/pixel3dmm/src/pixel3dmm/preprocessing/MICA/data/FLAME2020/generic_model.pkl"
157
+ if [ -f "$P3DM_FLAME" ]; then
158
+ log "reusing FLAME 2020 from p3dm install"
159
+ cp "$P3DM_FLAME" "$RES/generic_model.pkl"
160
+ elif [ "$HAVE_CREDS" = 1 ]; then
161
+ log "downloading FLAME 2020 (official, needs FLAME account)"
162
+ flame_official "FLAME2020.zip" "$RES/FLAME2020.zip"
163
+ unzip -o -q "$RES/FLAME2020.zip" -d "$RES/FLAME2020_tmp"
164
+ mv "$RES/FLAME2020_tmp/generic_model.pkl" "$RES/generic_model.pkl"
165
+ rm -rf "$RES/FLAME2020.zip" "$RES/FLAME2020_tmp"
166
+ else
167
+ log "downloading FLAME 2020 generic_model.pkl (HF mirror camenduru/show; set FLAME_USERNAME/FLAME_PASSWORD for official source)"
168
+ hf_mirror "models/models_MICA/FLAME2020/generic_model.pkl" "$RES/generic_model.pkl"
169
+ fi
170
+ fi
171
+
172
+ if [ ! -s "$RES/FLAME_masks.pkl" ]; then
173
+ if [ "$HAVE_CREDS" = 1 ]; then
174
+ log "downloading FLAME vertex masks (official)"
175
+ flame_official "FLAME_masks.zip" "$RES/FLAME_masks.zip"
176
+ unzip -o -q "$RES/FLAME_masks.zip" -d "$RES/FLAME_masks_tmp"
177
+ mv "$RES/FLAME_masks_tmp/FLAME_masks.pkl" "$RES/FLAME_masks.pkl"
178
+ rm -rf "$RES/FLAME_masks.zip" "$RES/FLAME_masks_tmp"
179
+ else
180
+ log "downloading FLAME_masks.pkl (HF mirror)"
181
+ hf_mirror "data/FLAME_masks.pkl" "$RES/FLAME_masks.pkl"
182
+ fi
183
+ fi
184
+
185
+ if [ ! -s "$RES/FLAME_texture.npz" ]; then
186
+ if [ "$HAVE_CREDS" = 1 ]; then
187
+ log "downloading FLAME texture space (official, ~1.3 GB)"
188
+ flame_official "TextureSpace.zip" "$RES/TextureSpace.zip"
189
+ unzip -o -q "$RES/TextureSpace.zip" -d "$RES/TextureSpace_tmp"
190
+ mv "$RES/TextureSpace_tmp/FLAME_texture.npz" "$RES/FLAME_texture.npz"
191
+ rm -rf "$RES/TextureSpace.zip" "$RES/TextureSpace_tmp"
192
+ else
193
+ log "downloading FLAME_texture.npz (HF mirror, ~1.3 GB)"
194
+ hf_mirror "data/flame/FLAME_texture.npz" "$RES/FLAME_texture.npz"
195
+ fi
196
+ fi
197
+
198
+ if [ ! -s "$RES/albedoModel2020_FLAME_albedoPart.npz" ]; then
199
+ log "downloading AlbedoMM albedo model (official GitHub release, ~1.8 GB)"
200
+ wget -q --show-progress --continue \
201
+ "https://github.com/waps101/AlbedoMM/releases/download/v1.0/albedoModel2020_FLAME_albedoPart.npz" \
202
+ -O "$RES/albedoModel2020_FLAME_albedoPart.npz"
203
+ fi
204
+
205
+ # ------------------------------------------------------------------ verify
206
+ log "verifying imports + files"
207
+ REPO_DIR="$REPO" python3 - <<'EOF'
208
+ import os, sys, warnings
209
+ warnings.filterwarnings("ignore")
210
+ import chumpy, gdown, kornia, cv2, nvdiffrast
211
+ repo = os.environ["REPO_DIR"]
212
+ sys.path.insert(0, repo)
213
+ import networks # pulls torchvision resnet50 + icosahedron
214
+ for f in ["checkpoints/coarse_reconstruction.pkl", "checkpoints/uv_completion.pkl",
215
+ "checkpoints/makeup_extraction.pkl",
216
+ "resources/generic_model.pkl", "resources/FLAME_masks.pkl",
217
+ "resources/FLAME_texture.npz",
218
+ "resources/albedoModel2020_FLAME_albedoPart.npz",
219
+ "resources/landmark_embedding.npy", "resources/skin_mask.png",
220
+ "libs/face_parsing/module_weight/79999_iter.pth",
221
+ "libs/face_landmark/checkpoint/mobilenet_224_model_best_gdconv_external.pth.tar"]:
222
+ p = os.path.join(repo, f)
223
+ assert os.path.exists(p) and os.path.getsize(p) > 0, p
224
+ print(" all good")
225
+ EOF
226
+
227
+ log "install complete. run: ./makeup_extract_run.sh <photo.jpg>"
makeup-extract/makeup_extract_run.sh ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ # =============================================================================
3
+ # makeup_extract_run.sh — single photo -> 3D makeup decomposition
4
+ #
5
+ # Runs the full 6-step makeup-extract pipeline (Yang et al., EG 2023) on one
6
+ # face image:
7
+ # step 0 landmark crop + face parsing -> aligned_img / mask
8
+ # step 1 coarse FLAME material reconstruction -> UV albedo/normal/shading
9
+ # step 2 UV texture completion (DSD-GAN)
10
+ # step 3 material refinement (500-iter optimization, VGG/TV losses)
11
+ # step 4 makeup extraction (bare skin / base / alpha / blend)
12
+ # step 5 re-render composite with extracted textures
13
+ #
14
+ # Usage:
15
+ # ./makeup_extract_run.sh <photo.jpg> [out_dir]
16
+ #
17
+ # out_dir default: /workspace/outputs/makeup-extract/<image-stem>/
18
+ #
19
+ # Notes:
20
+ # - non-square inputs are padded to square first (upstream assumes square)
21
+ # - raw per-step outputs are cached in $MAKEUP_EXTRACT_HOME/results/<stem>/;
22
+ # delete that folder to force a re-run
23
+ # =============================================================================
24
+ set -euo pipefail
25
+
26
+ if [ $# -lt 1 ] || [ "${1:-}" = "--help" ] || [ "${1:-}" = "-h" ]; then
27
+ grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0
28
+ fi
29
+
30
+ M_HOME="${MAKEUP_EXTRACT_HOME:-/workspace/makeup-extract}"
31
+ REPO="$M_HOME/makeup-extract"
32
+
33
+ log() { echo -e "\n\033[1;36m[makeup-extract]\033[0m $*"; }
34
+
35
+ IMG="$(realpath "$1")"
36
+ [ -f "$IMG" ] || { echo "input image not found: $IMG"; exit 1; }
37
+
38
+ for f in checkpoints/coarse_reconstruction.pkl resources/generic_model.pkl \
39
+ resources/FLAME_texture.npz resources/albedoModel2020_FLAME_albedoPart.npz; do
40
+ [ -s "$REPO/$f" ] || { echo "missing $REPO/$f - run ./makeup_extract_install.sh first"; exit 1; }
41
+ done
42
+
43
+ STEM=$(basename "$IMG" | sed 's/\.[^.]*$//' | tr -c 'a-zA-Z0-9' '_' | sed 's/_*$//;s/^_*//' | cut -c1-60)
44
+ STAGE="$M_HOME/staging/$STEM"
45
+ RESULTS="$M_HOME/results/$STEM"
46
+ OUT_DIR="${2:-/workspace/outputs/makeup-extract/$STEM}"
47
+ mkdir -p "$STAGE" "$RESULTS" "$OUT_DIR"
48
+
49
+ export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 # repo predates torch 2.6 torch.load change
50
+ export TORCH_HOME="$M_HOME/weights_cache/torch" # VGG19/resnet18 aux weights (keep off ephemeral /root)
51
+
52
+ START=$(date +%s)
53
+
54
+ # ---------------------------------------------------------------- stage input
55
+ # Upstream step_0 assumes a square image; pad to square (centered, black).
56
+ log "staging input (pad to square if needed)"
57
+ IMG_PATH="$IMG" STAGED="$STAGE/input.png" python3 - <<'EOF'
58
+ import os, cv2
59
+ img = cv2.imread(os.environ["IMG_PATH"])
60
+ h, w = img.shape[:2]
61
+ if h != w:
62
+ s = max(h, w)
63
+ top = (s - h) // 2; left = (s - w) // 2
64
+ img = cv2.copyMakeBorder(img, top, s - h - top, left, s - w - left,
65
+ cv2.BORDER_CONSTANT, value=(0, 0, 0))
66
+ print(f" padded {w}x{h} -> {s}x{s}")
67
+ cv2.imwrite(os.environ["STAGED"], img)
68
+ EOF
69
+
70
+ cd "$REPO"
71
+
72
+ # ---------------------------------------------------------------- pipeline
73
+ if [ -s "$RESULTS/render_texture/img_compose_with_bg.png" ]; then
74
+ log "cached results found in $RESULTS (delete to re-run)"
75
+ else
76
+ log "step 0/5: landmark crop + face parsing"
77
+ python3 step_0_preprocess.py -i "$STAGE/input.png" -o "$RESULTS"
78
+
79
+ log "step 1/5: coarse facial material reconstruction"
80
+ python3 step_1_coarse_reconstruction.py \
81
+ --aligned_img_path "$RESULTS/aligned_img.png" \
82
+ --segmented_img_path "$RESULTS/segmented_img.png" \
83
+ -o "$RESULTS/coarse_reconstruction"
84
+
85
+ log "step 2/5: UV completion"
86
+ python3 step_2_uv_completion.py \
87
+ --uv_mm_path "$RESULTS/coarse_reconstruction/uv_rncr.png" \
88
+ --uv_tex_path "$RESULTS/coarse_reconstruction/uv_tex.png" \
89
+ -o "$RESULTS/material_refinement"
90
+
91
+ log "step 3/5: material refinement (500-iter optimization)"
92
+ python3 step_3_material_refinement.py \
93
+ --mm_param_path "$RESULTS/coarse_reconstruction/mm_param.npz" \
94
+ --uv_albe_path "$RESULTS/coarse_reconstruction/uv_albe.png" \
95
+ --uv_norm_path "$RESULTS/coarse_reconstruction/uv_norm.png" \
96
+ --uv_rnsr_path "$RESULTS/coarse_reconstruction/uv_rnsr.png" \
97
+ --completion_tex_path "$RESULTS/material_refinement/completion.png" \
98
+ -o "$RESULTS/material_refinement"
99
+
100
+ log "step 4/5: makeup extraction"
101
+ python3 step_4_makeup_extraction.py \
102
+ --uv_albe_path "$RESULTS/material_refinement/albe_refine.png" \
103
+ -o "$RESULTS/makeup_extraction"
104
+
105
+ log "step 5/5: render composite"
106
+ python3 step_5_render_texture.py \
107
+ --mm_param_path "$RESULTS/coarse_reconstruction/mm_param.npz" \
108
+ --uv_bare_skin_path "$RESULTS/makeup_extraction/bare_skin.png" \
109
+ --uv_make_alpha_path "$RESULTS/makeup_extraction/make_alpha.png" \
110
+ --uv_make_blend_path "$RESULTS/makeup_extraction/make_blend.png" \
111
+ --uv_diffuse_shading_path "$RESULTS/material_refinement/diff_refine.png" \
112
+ --uv_specular_shading_path "$RESULTS/material_refinement/rnsr_refine.png" \
113
+ --input_aligned_image "$RESULTS/aligned_img.png" \
114
+ --aligned_image_mask_path "$RESULTS/mask_img.png" \
115
+ -o "$RESULTS/render_texture"
116
+ fi
117
+
118
+ # ---------------------------------------------------------------- outputs
119
+ log "collecting outputs"
120
+ cp "$RESULTS/aligned_img.png" "$OUT_DIR/"
121
+ cp "$RESULTS/makeup_extraction/"{bare_skin,make_base,make_alpha,make_blend}.png "$OUT_DIR/"
122
+ cp "$RESULTS/material_refinement/"{albe_refine,norm_refine,diff_refine,rnsr_refine,tex_refine,completion}.png "$OUT_DIR/"
123
+ cp "$RESULTS/render_texture/"{img_compose,img_compose_with_bg}.png "$OUT_DIR/"
124
+
125
+ ELAPSED=$(( $(date +%s) - START ))
126
+ log "done in ${ELAPSED}s. outputs in $OUT_DIR"
127
+ ls -la "$OUT_DIR"