Critique of be10025: I only wrapped _compute_crop_and_mask. The same
inconsistency still applied to every other operation inside the
VideoWorkspace block:
- extract_frames raises RuntimeError on ffmpeg failure
- _gpu_inpaint_{lama,vace} raises on CUDA OOM / model crashes
- frames_to_video raises RuntimeError on encode failure
- attach_audio re-raises after the AAC fallback also fails
All of those still surfaced as raw stack traces in the Gradio UI.
Now the entire VideoWorkspace body is wrapped:
- gr.Error → re-raised as-is (already user-friendly)
- ValueError / RuntimeError → wrapped with str(e) message — these
pipeline modules raise with readable messages by convention
- Anything else (OSError, CUDA OOM, etc.) → wrapped with
type(e).__name__: e prefix so the debug signal isn't lost
Net effect: every error a user can hit now shows up as a styled red
toast with a meaningful message, not a stack trace.
VideoWorkspace.__exit__ still runs on exception (with-statem…
fix: surface mask-validation ValueError as gr.Error in run_pipeline
Critique of 89dfe5f caught an inconsistency: on_preview_crop wraps
_compute_crop_and_mask in try/except so the ValueError messages from
mask_to_bbox / compute_crop_region (e.g. "The drawn area is too small",
"Watermark area is too large", "Frame is too small") show up as clean
status text. run_pipeline did NOT wrap the same call, so clicking
Remove Watermark with a too-small mask surfaced as a raw ValueError
stack trace in the Gradio UI instead of a styled red toast.
Now run_pipeline catches ValueError from _compute_crop_and_mask and
re-raises as gr.Error, matching the on_preview_crop UX.
Also tidy the upload-status notes formatting: when both auto-trim and
fps-cap apply (a >15s 60fps clip), the two warnings are now separated
by a blank line so they don't visually run together.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: cap working fps and surface prewarm failures (critique of b859009)
Critical re-review of the previous round caught two real issues:
1. The 180→240s LaMa duration bump was insufficient and missed VACE
The original gap I tried to address (60fps × 15s exceeds @spaces.GPU
budget) wasn't actually closed:
- LaMa worst case ≈ 270s vs new 240s budget — still overruns.
- VACE has the same problem and I missed it: 60fps × 15s = 900
frames → range(0, 900, 73) = 13 chunks × ~25s = ~325s vs the
300s VACE budget.
Real fix: PROCESS_FPS_MAX = 30. Both extraction and encoding use the
capped value via a working_meta clone (dataclasses.replace), so the
output mp4 duration matches the trimmed input. At 30fps both modes'
worst case fits comfortably:
- LaMa: 30 × 15 × 0.3 = 135s ≪ 240s
- VACE: 7 chunks × 25s = 175s ≪ 300s
on_video_upload now surfaces a "fps will be capped to 30" notice so
the user knows why a 60fps source is processed at 30fps.
2. wait_for_prewarm didn't track failure
If the prewarm thread cra…
fix: list-audit fixes — LaMa duration, multi-stream audio, prewarm wait
Acted on the items from the gap-list audit that were real bugs/risks:
C. LaMa @spaces.GPU duration was tight for high-fps clips
180 s budget vs 60 fps × 15 s = 900 frames × ~0.3 s/frame = ~270 s
worst case. Bumped to 240 s — covers typical 60 fps loads with
~30 s headroom. (VACE stays at 300 s; per-mode budget split intact.)
H. attach_audio dropped extra audio streams
``-map 1:a:0`` only kept the first audio stream — sources with
commentary tracks / alternate languages / 5.1 + stereo dual-mix
silently lost everything but the main mix. Now uses ``-map 1:a``
(preserves all audio streams). Re-encode fallback path applies
AAC@192k to each independently.
N. Prewarm wait happened inside the GPU lease
_get_pipe()'s ``_prewarm_thread.join()`` could block for several
minutes if the user clicked Quality before the 75 GB prewarm
finished — eating into the @spaces.GPU(duration=300) budget and
risking timeout. New wait_for_prewarm() / is_prewarm_done() helpers…
fix: harden upstream-protection so the Space survives upstream deletion
The audit revealed two paths where a runtime fetch could still hit
upstream sources (Wan-AI, lightx2v, github.com/enesmsahin) even though
all the weights are mirrored at
JackIsNotInTheBox/Video_Watermark_Remover_Checkpoints. Both closed:
pipeline/lama.py
- LAMA_MODEL_URL now defaults to the mirror URL directly (was None
unless the Space variable was set). If the Space variable is dropped
for any reason, the prefetch still works — no silent fallback to the
hardcoded GitHub release that simple_lama_inpainting would otherwise
reach for.
pipeline/vace.py
- New VACE_LOCAL_ONLY knob (defaults to True when VACE_PREWARM=1, which
is the default). Forces every from_pretrained / load_lora_weights call
inside _get_pipe() to pass local_files_only=True. Combined with the
prewarm step populating the cache from the mirror, this guarantees no
upstream HF Hub fetch is ever attempted at inference time — even if
the mirror itself goes offline mid-session, the cached files keep
workin…
fix: composite_and_save raises on length mismatch instead of silent truncation
The function previously zipped frame_paths with the inpainted-crops
iterator and would silently stop at whichever ran out first. If the GPU
inpaint generator yielded fewer frames than expected (chunking bug,
model error, etc.), the output mp4 would be shorter than the source
without any signal that frames were lost.
``zip(..., strict=True)`` (Python 3.10+, available on the Space)
escalates that to a ValueError, so any future generator/path mismatch
surfaces loudly instead of corrupting the output.
In the current pipeline both LaMa and VACE always yield n_frames items
matching frame_paths exactly, so this is purely a defence-in-depth
change — no observable behavior shift on the happy path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: reject sub-32px frames in compute_crop_region
The VACE / Wan2.1 VAE downsamples spatially by 32×, so each frame
dimension must be at least 32px. The existing crop logic floored the
target to a multiple of 32 then forced a min of 32 — but on a 30×30
frame, that produced a target larger than the frame, silently
overflowing downstream numpy slices in pipeline.composite.
Now compute_crop_region rejects frames smaller than MIN_FRAME_DIM=32
on either axis with a clean ValueError, surfacing the failure at the
point of detection instead of as a confusing slice-shape mismatch
later in the pipeline.
Reachable in practice via accidentally-tiny inputs (thumbnails uploaded
as videos, etc.).
Found by tests/test_callbacks.py round.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
refactor: round 7 — collapse dual mask-empty check into _get_mask_from_editor
The function now returns None when the mask is either empty (no layers)
*or* fully blank (all-zero pixels). Three call sites had to repeat
``raw_mask is None or raw_mask.max() == 0`` because the function only
covered the first case; that distinction was never used downstream
(both errors surfaced the same "no drawing detected" message).
Pulling the .max() == 0 guard inside the helper turns each callsite
into a single ``if raw_mask is None:`` and matches the docstring's
"no usable mask drawn" semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
refactor: round 6 — drop unused list-wrapper public APIs
pipeline/lama.py
- Delete inpaint_frames_lama (the list-materialising wrapper around
inpaint_frames_lama_stream). Never called anywhere; only the
streaming generator is used end-to-end.
pipeline/vace.py
- Delete inpaint_frames_vace for the same reason — the streaming
inpaint_frames_vace_stream is the only consumer path.
Reduces public API surface area; the streaming functions are now the
sole supported entry points for both modes, matching how each is
actually integrated with composite_and_save.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
app.py
- Extract _compute_crop_and_mask() — runs the
``mask → bbox → crop_region → inpaint_mask`` chain. Previously the same
five lines lived in both on_preview_crop and run_pipeline; both call
sites now reduce to a single tuple-unpacking line.
- Replace the run_pipeline tempfile dance:
fd, p = tempfile.mkstemp(...); os.close(fd); ...
try: ...
except: try: os.unlink(p) except OSError: pass; raise
with Path / missing_ok=True semantics. Also adds a cast to str() at
the gr.Video output for cross-version Gradio compatibility.
- Drop the redundant ``# ── Compute crop + mask ──`` comment block
(the helper docstring is now the single source).
pipeline/composite.py
- composite_and_save: write intermediate PNGs with compress_level=1
instead of Pillow's default 6. The frames are immediately re-encoded
by frames_to_video's libx264/libx265 pass, so PNG compression cost
is pure waste — compress_level=1 finishes ~50-100ms faster per 1080p
frame; the composite phase s…
pipeline/composite.py
- Move ``_composite_and_save`` from app.py here as the public
``composite_and_save``. The compositing module now owns the entire
composite surface area; app.py is one less helper away from being
pure UI/orchestration.
app.py
- All four pipeline imports (composite, crop, lama, vace, video) are
now top-level. The lazy ``from pipeline.X import …`` inside the
@spaces.GPU functions was unnecessary — ZeroGPU forks the GPU worker
process from the parent, so top-level imports are inherited, and none
of these modules trigger heavy work on import (lama and vace defer
their torch-heavy bits to lazy singletons inside _get_model /
_get_pipe).
- Drop the local _composite_and_save shim now that the canonical
version lives in pipeline.composite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pipeline/video.py
- Replace extract_first_frame (file-based) with extract_first_frame_array
that streams the PNG through ffmpeg's stdout pipe and returns an RGB
ndarray directly. Eliminates the mkstemp/open/unlink dance the only
caller (on_video_upload) had to perform — and removes a tempfile
artifact from /tmp on every upload.
app.py
- _card_title(text, step?, top_margin?) helper replaces 6 hand-built
'<div class="card-title">…</div>' gr.HTML calls in the UI block.
- on_video_upload: nested _msg(text) helper builds the (editor, crop,
state, status) 4-tuple for early-error returns. Drops the repetitive
``return gr.update(), gr.update(), None, "…"`` pattern at every guard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pipeline/crop.py
- Add CropRegion.frame_x2 / frame_y2 (exclusive right/bottom edges) and
pil_box (4-tuple for PIL.Image.crop()). The pattern
``(cr.frame_x, cr.frame_y, cr.frame_x + cr.frame_w, cr.frame_y + cr.frame_h)``
was reconstructed in 5+ places; ``cr.pil_box`` is now the single source.
Numpy slices like ``arr[cr.frame_y:cr.frame_y2, cr.frame_x:cr.frame_x2]``
read more cleanly than the previous ``+ cr.frame_h`` arithmetic.
Sites updated to use the new properties:
- pipeline/composite.py: composite_with_alpha numpy slicing (also drops
a redundant trailing .astype(np.uint8) on the already-uint8 result)
- pipeline/lama.py: _load_crop
- pipeline/vace.py: _load_chunk_crops
- pipeline/crop.py: build_inpaint_mask source-mask cropping
- app.py: _create_crop_preview rectangle + mask overlay
static/style.css (new file)
- The 100-line CSS string previously embedded in app.py now lives in its
own file so it gets proper editor syntax highlighting and shows up as
a…
refactor: round 1 — drop dead code, lift constants, simplify dispatch
app.py
- Remove unused HAS_SPACES sentinel.
- Lift UPLOAD_DURATION_S, MAX_UPLOAD_W, MAX_UPLOAD_H, MODE_FAST, MODE_QUALITY,
ALL_MODES to module-level so the same values aren't duplicated between
on_video_upload's local scope, the radio choices, and the dispatch logic.
- Replace _meta_to_dict/_dict_to_meta (35 lines of field-by-field copying)
with dataclasses.asdict + a 1-line _meta_from_state that filters by
fields(VideoMeta). Optional VideoMeta fields now have safe defaults so
Gradio State dicts from older app versions reconstruct cleanly.
- Drop the Gradio-5 dict-format guard in _get_mask_from_editor: dead while
requirements.txt pins gradio<5.0.0; comment notes the rewrite need if
that pin ever lifts.
- Fix dead overwrite of `mask_rgba` in _create_crop_preview.
- Delete the _inpaint_composite_save_gpu dispatch indirection (20 lines)
and inline the if/else into run_pipeline. Mode validation now happens
exactly once, at the top of run_pipeline.
- Drop the unused _total…
pipeline/vace.py
- New inpaint_frames_vace_stream() generator yields one inpainted
ndarray at a time. Memory footprint per chunk (~250 MB) is now
independent of clip length — the previous list-based version held
all 450 input PIL crops + 450 output PIL frames simultaneously
(~1.4 GB peak), risking OOM on ZeroGPU's 16 GB host RAM.
- inpaint_frames_vace() retained as a list-materialising wrapper
around the streaming version, mirroring the lama_stream/lama pattern.
- Emit semantics preserve "later chunk wins" for overlap regions:
each non-final chunk emits its first stride (=73) outputs; the
trailing OVERLAP frames are recomputed by the next chunk with its
larger temporal context.
- progress_fn(chunk_index, total_chunks) callback for per-chunk
progress reporting.
- New prewarm_vace_cache(): kicks off ~75 GB checkpoint download in a
background thread at app startup so the cache is populated before
the first Quality-mode click. Without it, the first VACE run would
at…
feat: snap-to-rectangle button + wire LaMa to LAMA_MODEL_URL mirror
UI
- New "Snap to Rectangle" action: takes the user's rough scribble and
replaces it with a clean filled rectangle covering the same bbox.
Most watermarks are rectangular (corner logos, channel bugs, subtitle
bars), so this gives the same coverage as careful brush-filling but
in one click after a quick drag. Reuses the existing brush — no
custom Gradio component required.
LaMa upstream protection
- pipeline/lama.py: when LAMA_MODEL_URL env var is set (already configured
on the Space), pre-fetch big-lama.pt into torch.hub's checkpoint cache
before SimpleLama() runs. SimpleLama then finds the file pre-existing
and skips its hardcoded GitHub-release URL. Closes the upstream-
protection loop opened by the Video_Watermark_Remover_Checkpoints mirror.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: cap huggingface-hub <0.30 and diffusers <0.35 so gradio 4.44 oauth works
Gradio 4.44.0 (the last 4.x — 5.x broke ImageEditor) imports HfFolder
from huggingface_hub. HfFolder was removed in newer hub versions, so
the app crashed on import.
The compatibility window:
- huggingface-hub >=0.27.0 (diffusers 0.34 floor) <0.30.0 (HfFolder still present)
- diffusers >=0.34.0 (WanVACEPipeline floor) <0.35.0 (so its hub floor stays at 0.27)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: relax huggingface-hub pin so diffusers 0.34+ can install
The previous <0.27.0 cap blocked the resolver because diffusers 0.34+
(which we need for WanVACEPipeline) requires huggingface-hub>=0.27.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires up the previously-stubbed Quality mode with a full WanVACEPipeline
inference path, served from the upstream-protection mirror.
pipeline/vace.py
- WanVACEPipeline loaded from JackIsNotInTheBox/...Checkpoints/vace-14b
(env-overridable via VACE_REPO_ID / VACE_SUBFOLDER)
- Fuses lightx2v rank-64 distill LoRA → 4-step inference
(~7-8x speedup vs 30-step base; falls back to 30-step if LoRA fails)
- FP8 dynamic-activation quantization on H100/H200 (sm_90+) via torchao,
bf16 fallback when unavailable
- Processes long clips in 81-frame chunks with 8-frame overlap; later
chunks overwrite the seam region (larger temporal context wins)
- Crops resized to crop_region.target_{w,h} for VACE, then downsampled
back to original crop dimensions
- model.enable_model_cpu_offload() to keep peak VRAM in budget
app.py
- Splits _inpaint_composite_save_gpu into per-mode functions:
_gpu_inpaint_lama @spaces.GPU(duration=180)
_gpu_inpaint_vace @spaces.GPU(duration=300)
Closes the long-stan…
feat: auto-trim long uploads and add audio re-encode fallback
- extract_frames accepts max_duration_s; pipeline now silently trims
to first 15 s instead of rejecting clips up to 60 s
- on_video_upload shows a trim notice when the uploaded clip exceeds
the processing window
- attach_audio falls back to AAC re-encode when stream-copy fails
(fixes silent mux failure for sources with MP4-incompatible audio
codecs like raw PCM in .mov or Opus in .webm)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: add mask-clear button and reject too-small / too-large watermarks
- Clear Mask button wipes paint layers while keeping the loaded frame
- mask_to_bbox: reject masks under 100 px (filters stray clicks)
- compute_crop_region: reject bboxes covering >50% of frame area
(inpainting models need surrounding context to reconstruct)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chore: update project dependencies in requirements.txt
Jack Wu
f26f977
chore: update project dependencies in requirements.txt
Jack Wu
2fff848
chore: update dependency versions in requirements.txt
Jack Wu
7b6893a
chore: update python version requirement in README and dependencies
Jack Wu
1f081e2
.
Jack Wu
9a85a2a
feat: introduce checkpoint mirroring script, strengthen video validation, and improve pipeline robustness for masking and compositing.
Jack Wu
ebe8a5c
refactor: implement streaming LaMa inpainting, optimize crop aspect ratio matching, and add support for high-frame-rate video presets
