.claude/ holds settings.local.json and a live git worktree (~1.4MB) whose
metadata embeds machine-local absolute paths. It was showing up as untracked
noise in every status; ignoring it is correct, committing it would not be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Recalibrate GPU duration reservations, drop deprecated warnings
Live logs showed all three models finishing (warm) in 18-39s while
every call reserved a blanket 120s of ZeroGPU quota — the hard-coded
120s floor plus generous load_overhead guesses were never tuned
against real GPU timings. ZeroGPU's admission check gates on the
requested duration against remaining quota, so over-reserving causes
premature "quota exceeded" errors well before quota is actually
exhausted.
- Lower the global floor from 120s to 45s.
- Recalibrate each model's load_overhead to ~1.5-2x its measured warm
end-to-end time (TARO 120->35, MMAudio 30->35, HunyuanFoley 90->55,
with extra margin kept for HunyuanFoley's larger cold-disk-read risk).
- Drop the deprecated local_dir_use_symlinks kwarg from hf_hub_download
calls.
- Silence the harmless weight_norm FutureWarning from vendored
HunyuanVideo-Foley DAC VAE loading code.
Replace torchvision.io.read_video with a PyAV-based shim
torchvision>=0.26 (auto-installed alongside torch 2.11.0) removed the
public read_video/write_video API entirely from torchvision.io in OSS
builds, breaking TARO onset extraction with ImportError. av (PyAV) was
already a pinned dependency, so decode frames directly with it instead.
Save output WAVs via soundfile instead of torchaudio.save
torchaudio 2.11 removed its old save backends in favor of torchcodec,
which isn't installed, so torchaudio.save() raised ImportError on the
very last step of every generation. soundfile is already a pinned
dependency and needs no extra backend.
Use bare PyPI torch/torchaudio 2.11.0 instead of +cu128 index build
ZeroGPU's config validator rejects local-version suffixes like
+cu128 (CONFIG_ERROR: "torch version in requirements.txt is not
compatible with ZeroGPU"), even though 2.11.0 is on its supported
list. The plain PyPI torch==2.11.0 wheel is already CUDA-enabled
(530MB, bundles Blackwell-compatible kernels), so the custom
cu128 index isn't needed.
Pin torch to exact cu128 build to match torchaudio
Without an explicit torch pin, pip resolved a torch build that didn't
ship libcudart.so.12, so torchaudio's compiled CUDA extension failed
to load at runtime (RUNTIME_ERROR, not build error).
Bump torch/torchaudio to a Blackwell-compatible version, swap mmcv for mmcv-lite
Root cause of the "CUDA error: no kernel image is available for execution
on the device" failures on every generation: torchaudio==2.5.1+cu124 pinned
torch down to 2.5.1, but HF's own ZeroGPU docs list only 2.8.0-2.11.0 as
supported (https://huggingface.co/docs/hub/spaces-zerogpu#supported-versions)
for their NVIDIA RTX Pro 6000 Blackwell hardware. 2.5.1 predates Blackwell
(sm_120) kernel support entirely, so every single parameter copy onto the
GPU failed identically.
Bumped to torchaudio==2.11.0+cu128 (the top of ZeroGPU's tested range),
verified both torch and torchaudio 2.11.0 wheels exist for cp310 under the
cu128 index. The platform already injects its own `torch<=2.11.0` cap on
the build command line, so no explicit torch pin is needed — it resolves
to match torchaudio's pin.
This orphaned mmcv==2.2.0's --find-links index (only ever published wheels
up through torch2.4.0 — confirmed via direct check, every later torch
directory 404s), which would have ABI-crashed against 2.11.…
Remove pinned spaces==0.48.2 causing a hard build failure
Switching Space Hardware to ZeroGPU triggered a build failure:
"Cannot install spaces==0.48.2 and spaces==0.50.4 because these package
versions have conflicting dependencies." HF's build pipeline injects its
own required `spaces` version directly on the pip install command line
(spaces==0.50.4 currently) — our own explicit pin collided with it and
made the install unsatisfiable outright, regardless of which hardware
tier was selected (the build log showed the same conflict even after
reverting back to cpu-upgrade).
This is very likely also the actual root cause of today's "no real GPU
access" finding (torch.cuda.is_available() came back False from inside
an actual @spaces.GPU call) — spaces==0.48.2 is old enough that it may
never have correctly supported the current ZeroGPU allocation protocol.
Removing our own pin lets the platform's injected version win with no
conflict, on any hardware tier.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add explicit CUDA availability diagnostic inside GPU worker calls
User correctly pushed back on an unverified claim that the app already had
real GPU access via ZeroGPU regardless of the "Space Hardware" tile
selection. HF's own ZeroGPU docs explicitly say @spaces.GPU is a no-op
outside a Space with "ZeroGPU" hardware selected, and this Space currently
has "CPU upgrade" selected instead — which would perfectly explain the
12+ minute "hang" that was actually just very slow CPU-bound diffusion,
not a deadlock.
This contradicts the earlier-observed CUDA OOM errors and real GPU
allocation during prior testing this session, so rather than guess further,
print hard facts: torch.cuda.is_available(), device name, and device count
from inside an actual @spaces.GPU-decorated call. Next generation attempt
will show definitively whether real GPU access is happening right now.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Force local_files_only on open_clip's own hf_hub_download reference
Live repro crashed the previous fix's own debug line: AttributeError,
huggingface_hub.constants has no is_offline_mode() on the deployed version —
meaning the whole HF_HUB_OFFLINE/is_offline_mode() approach was built against
the wrong version's internals, explaining why neither prior attempt (78108d3,
035533f) actually stopped the live re-download despite being individually
verified against huggingface_hub 1.23.0 in isolation.
The traceback also showed the @spaces.GPU call running through
anyio.to_thread.run_sync — a worker thread in the same process, not the
separate process the code's own docstrings had suggested — so process
isolation wasn't the blocker either.
Rather than keep chasing which offline-mode attribute the actually-deployed
huggingface_hub version honors, sidestep it: open_clip.pretrained binds its
own local `hf_hub_download` name at import time
(`from huggingface_hub import hf_hub_download`), so monkeypatch that specific
module-level reference to force local_files_onl…
Move offline-mode patch inside each GPU function's own body
Live repro (direct HTTP call to /gradio_api/queue/join for MMAudio's
_run_mmaudio, bypassing the browser) showed the previous fix (78108d3,
patching inside _catch_oom's wrapper) still didn't stop the live ~4GB
open_clip re-download. _catch_oom's wrapper sits *outside* the raw
function — it's the code that dispatches the call, not necessarily the
code that runs inside the actual ZeroGPU worker. This codebase's own
docstrings already documented that @spaces.GPU functions run in a
genuinely separate "GPU worker process" (kwargs are silently dropped,
paths must exist in the worker's own filesystem), which points at the
same boundary.
Added _ensure_offline_in_worker() (with a diagnostic print of pid +
the resulting is_offline_mode()) and call it as literally the first
line of all six raw GPU function bodies (_taro_gpu_infer,
_mmaudio_gpu_infer, _hunyuan_gpu_infer, and the three _regen_*_gpu
variants), so it runs wherever each function's own bytecode actually
executes, regardless of how @sp…
Re-assert offline mode inside the ZeroGPU worker itself
The previous fix (1cb4beb) patched huggingface_hub.constants.HF_HUB_OFFLINE
in the main process at startup, verified correct in isolation — but a live
diagnostic (direct HTTP call to /gradio_api/queue/join for MMAudio's
_run_mmaudio, bypassing the browser entirely to reproduce the "Connection
errored out" report) showed open_clip's CLIP loader still re-downloading its
~4GB checkpoint live inside the actual GPU call, confirming the worker
process ZeroGPU dispatches @spaces.GPU calls to doesn't reliably inherit
that patch from the main process.
Fix: re-set both os.environ["HF_HUB_OFFLINE"] and the huggingface_hub
constant at the top of _catch_oom's wrapper, which already wraps all six
@spaces.GPU functions as the innermost decorator — so it executes inside
whatever process actually runs the GPU call, at call time, regardless of
ZeroGPU's fork/spawn timing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Root-caused the "connection errored out" MMAudio failure via server logs:
_mmaudio_gpu_infer started fine, then open_clip's create_model_from_pretrained
('hf-hub:apple/DFN5B-CLIP-ViT-H-14-384', ...) re-downloaded the full ~4GB
checkpoint mid-GPU-call instead of using the encoder cache pre-populated at
startup, eating enough time/bandwidth inside the billed GPU call to drop the
connection.
Root cause: os.environ["HF_HUB_OFFLINE"] = "1" was set *after* huggingface_hub
was already imported (line ~32). huggingface_hub.constants.HF_HUB_OFFLINE is
computed once from os.environ at that import, so setting the env var later
never actually enables offline mode for the rest of the process — every
from_pretrained()/hf-hub: call was silently running online the whole time.
With the mirror-cache trick (_populate_hf_cache_from_mirror) writing a fake
'mirror-snapshot' commit hash rather than the real one, an online resolution
always treats it as a cache miss and re-downloads from the real upstream rep…
Fix spurious "Too many arguments provided for the endpoint" warnings
Root-caused via the actual gradio 5.25.0 source (matching the deployed
version exactly): Blocks.get_api_info() never populates the
unnamed_endpoints dict (confirmed still true in 5.50.0 and 6.20.0 too
— named_endpoints gets every dependency, unnamed_endpoints is just
never written to). But the JS client dispatches every internal
click/change/upload event by numeric fn_index and looks it up
specifically in unnamed_endpoints, so endpoint_info is always
undefined for these, parameters falls back to [], and any non-empty
data array trips a false-positive "too many arguments" warning. This
never actually broke anything (map_data_to_params just falls back to
the raw args on the empty-parameters path), but it fired on every
video upload and every Generate click in both live repro sessions.
Fix: monkeypatch get_api_info to mirror each already-computed
named_endpoints entry into unnamed_endpoints under its numeric
fn_index, since every dependency already gets an auto-generated
api_name regardl…
Fix Generate click never reaching server: watchdog broke the click event
Confirmed via server logs (hf spaces logs) that after the previous fix
(8ed76c6), clicking Generate produced zero server-side activity across
all three tabs — none of _run_taro/_run_mmaudio/_run_hunyuan's print
statements ever appeared, meaning the click never even reached the
backend. Root cause: combining js= and a real server fn= on the same
.click() event (as 8ed76c6 did) is a combination that breaks the click
from firing at all, contrary to what the docs implied.
Fix: stop using Gradio's js= event-chaining for the watchdog entirely.
Arm it instead via a plain native document click listener (event
delegation, so it works regardless of when Svelte creates the button
DOM). The .click() chains are back to their pre-8ed76c6 form, which
was already confirmed working for triggering generation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Stop Generate from going silent: merge watchdog-arm into the click event
Two prior attempts (removing then reworking the standalone fn=None
.then() step) didn't fix it — Generate still never reached the backend
after the button changed state, meaning that whole extra chain link was
the problem, not its inputs value.
Verified in Gradio's own source (events.py): js "runs before fn" on the
SAME event when both are given — it isn't a separate chain link. Moved
the watchdog-arm js= directly onto the existing .click(fn=_btn_generating)
call instead of chaining a standalone fn=None step after it. This
removes the untested fn=None/non-queued link from the chain entirely
rather than trying to fix its behavior.
Fix Generate click going silent after adding the watchdog-arm step
Live logs showed zero server-side activity after a Generate click —
not even the CPU-preprocessing print statements that normally fire
first — meaning the request never reached the backend at all.
The newly added watchdog-arming step was the only fn=None (JS-only,
unqueued per blocks.py's `queue = False if fn is None else queue`)
dependency in this app using inputs=None; every other fn=None step
(slot visibility) uses a real inputs=[...] list and is confirmed
working. inputs=None combined with fn=None is an untested combination
and the most likely place for the chain to break before ever reaching
the real generate call. Switched to inputs=[] to match the known-good
pattern.
Fix infinite retry loop in video upload transcode
Live logs (pulled via `hf spaces logs`) showed 39 repeated failures
alternating between "SDR_Render__h264.mp4" and "SDR_Render__gr_conv.mp4"
"No such file or directory" for a single upload — an infinite loop
between _transcode_for_browser and Gradio's own internal conversion,
each re-triggering on the other's (since-cleaned-up) output via the
cross-tab _sync propagation.
_transcode_for_browser now short-circuits to a no-op gr.update() when
its input is already one of these generated filenames, or no longer
exists on disk — breaking the cycle regardless of the exact re-trigger
mechanism, since there's no longer a path for a re-triggered handler to
chase.
Add client-side watchdog so a crashed generation can't wedge Generate
The button-reset step (_btn_ready) only fires via Gradio's .then()
chain, which requires the preceding step's SSE stream to report SOME
terminal status. A dropped connection (container crash-restart, HTTP/2
protocol error — both seen in this app's logs) delivers no status at
all, so the chain stalls forever and Generate stays disabled with no
way to retry.
Arm a client-side timeout (10 min, well past any normal generation)
right after the button disables, that force-resets it if still stuck.
Token-gated per click so a stale watchdog can't clobber a newer,
still-in-flight generation.
Prevent GPU OOM from crashing the whole Space container
Root cause of "every error triggers a full rebuild": none of the six
@spaces.GPU functions caught CUDA OOM, so an out-of-memory error could
escape as an unrecoverable worker fault instead of a clean exception —
forcing HF to restart the whole container (wiping /tmp, re-downloading
~30GB of checkpoints) instead of just failing one request.
- _catch_oom wraps all 6 @spaces.GPU functions: catches
torch.cuda.OutOfMemoryError, clears the cache, and re-raises as
gr.Error so it renders as a clean UI error instead of a fatal fault.
- Enable HunyuanVideo-Foley's existing (but previously unused)
enable_offload=True path, which swaps SigLIP2/CLAP/Synchformer/DAC-VAE
in and out instead of holding ~14GB of static weights resident for
the whole call — the swap logic was already fully wired in the
vendored model_utils.py/feature_utils.py, just never turned on.
- Add del + empty_cache for MMAudio's net/feature_utils (~9.5GB) at the
end of _mmaudio_gpu_infer and _regen_mmaudio_gpu, which…
Replace server-side slot-visibility updates with client-side JS
Gradio 5.25's client rejects gr.Group visibility outputs with "Too many
arguments provided for the endpoint" (500 on /queue/join), even when
isolated in their own .then() step — confirmed via fn_index tracing
that this wasn't session staleness but a real, reproducible bug tied
specifically to Group components as event outputs.
Toggling display on the slot Group's DOM element (by elem_id) via a
fn=None/js= event needs no server round-trip, so there's nothing left
for that validation to reject.
Disable Generate button and show busy label during generation
Prevents double-clicking Generate from firing overlapping requests on
the same slot, and gives clearer feedback that a job is in flight.
Wired via .then() (not .success()) on all three tabs so the button
re-enables even if the generate call raises.
Patch Gradio's video-conversion race that kills the generation connection
convert_video_to_playable_mp4() writes ffmpeg output to
Path(video_path).with_suffix(".mp4"), a no-op rename when the upload
is already *.mp4 (e.g. HEVC-in-mp4). It overwrites that file in place
while the browser is still streaming it for the preview widget, which
corrupts the response and — since HF Spaces multiplexes a tab over one
HTTP/2 connection — takes the whole connection down, including the
live generation job's SSE queue. Convert a copy instead so the file
being served is never mutated mid-stream.
Fix ffmpeg stream-copy crash in strip_audio_from_video
Falls back to re-encoding (libx264) when the source codec can't be
stream-copied into the mp4 container (e.g. VP9/webm, ProRes/mov),
instead of letting ffmpeg.Error crash the request.
Bump spaces==0.48.1 -> 0.48.2 to resolve build conflict
app.py: replace direct upstream snapshot_download() for siglip2/clap/clip/audioldm2/bigvgan with downloads from JackIsNotInTheBox/Generate_Audio_for_Video_Checkpoints/encoders/<orig>, then symlink into HF cache under the upstream repo IDs so existing from_pretrained() calls resolve from the mirror. Set HF_HUB_OFFLINE=1 after the parallel download phase to force any later HF Hub call (CLAP, AudioLDM2, MMAudio open_clip 'hf-hub:apple/...') to be cache-only.
MMAudio/mmaudio/utils/download_utils.py: hardcoded huggingface.co/hkchengrex/MMAudio + github.com/hkchengrex/MMAudio/releases URLs repointed at upstream/MMAudio/{weights,ext_weights}/ in the mirror. Dormant under normal startup, live if MMAudio's download_model_if_needed() is ever invoked.
Result: full upstream-protection mode. Space pulls all 11 main weights + 5 encoder/vocoder repos from the mirror with no upstream calls at runtime.
- _save_wav: always applies _to_stereo() so every output WAV is 2-channel
- _resample_to_target: only treats (2,T) as stereo; (1,T) preserved as-is
- regen_taro/mmaudio/hunyuan_segment: apply _to_stereo() before _splice_and_save
- media_utils.py: remove pan=stereo|c0=c0|c1=c0 filter that was routing only
left channel to both outputs once input became true stereo
Fix xregen duration truncation: use full segment span, not model window
_xregen_clip_window was centering target_window_s on the segment midpoint,
returning clip_dur=target_window_s even when the original segment was longer.
_build_segments then produced only 1 sub-seg, generating e.g. 8s of MMAudio
audio for a 15s segment — causing _stitch_wavs to truncate the total output.
Fix: _xregen_clip_window now always returns the full original segment span
(seg_start..seg_end clamped to video bounds). _build_segments then splits
it into as many model-sized sub-windows as needed, exactly like initial
generation — so the stitched regen wav always covers the full segment.
Also reverts the incorrect silence-padding workaround from the prior commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix xregen truncating total audio duration when clip < segment window
When an xregen model generates a shorter clip than the original segment
window (e.g. MMAudio 8s on a HunyuanFoley 15s segment), _stitch_wavs
trims the last segment's wav expecting it to cover the full window.
A short wav gets min-clipped, making the final stitched audio shorter
than total_dur_s.
Fix: in _xregen_splice, after prepending leading silence to align to
seg_start, append trailing silence to pad the wav to the full original
segment duration (seg_end - seg_start). _stitch_wavs then trims it
correctly and the output is always exactly total_dur_s long.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All seg wavs are now guaranteed (2,T) by _save_seg_wavs/_to_stereo.
Remove the silent fallbacks that were masking shape bugs:
- _load_seg_wavs: assert (2,T) instead of silently squeezing (1,T)
- _splice_and_save: assert new_wav is (2,T) on entry; remove
_normalize_channel_layout call (function deleted)
- _resample_to_slot_sr: always call _to_stereo(); drop slot_wav_ref
channel-matching logic and defensive squeezes
Any shape violation now raises AssertionError immediately with the
offending shape, instead of silently producing wrong-shaped output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Normalize all seg wavs to stereo (2,T) at save time
- _to_stereo(): new helper that squeezes (1,T)→(T,) then duplicates
mono (T,)→(2,T). Handles all three model outputs uniformly.
- _save_seg_wavs(): applies _to_stereo() before np.save so every .npy
on disk is always (2,T). TARO and HunyuanFoley mono gets duplicated
to fake stereo; MMAudio's genuine stereo is preserved as-is.
- Eliminates the root cause of all channel-shape mismatches in _cf_join:
on-disk format is now uniformly stereo regardless of source model.
_load_seg_wavs squeeze and _normalize_channel_layout remain as
defensive fallbacks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _normalize_channel_layout(): new helper called in _splice_and_save that
upcasts all mono (T,) segments to stereo (2,T) when any segment in the
slot is stereo. Previously mono-wins: a TARO regen on an MMAudio slot
would average MMAudio's stereo segments down to mono.
- _resample_to_slot_sr(): stereo-wins instead of slot-wins. If the new
wav is stereo and the slot ref is mono, keep the new wav stereo and let
_normalize_channel_layout upcast the existing segments. Defensively
squeezes (1,T) to (T,) before any channel decision.
- _post_process_samples / initial generation: seg_wavs are saved as-is
per model (TARO mono, MMAudio stereo, HunyuanFoley mono after squeeze).
Cross-model regens now use stereo-wins so MMAudio stereo is never lost.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix (1,T) shape crash in _cf_join; bump HunyuanFoley secs_per_step to 0.5
- _load_seg_wavs: squeeze (1, T) → (T,) on load. HunyuanFoley's DAC
decoder emits (batch, 1, time); audio_batch[0] = (1, T). When these
single-channel arrays are saved as npy and later loaded alongside
true stereo (2, T) from MMAudio, _cf_join tries to concatenate
(2, N) with (1, M) along axis=1 → ValueError. The squeeze at load
time normalises all single-channel wavs to plain mono (T,) so
_resample_to_slot_sr's channel-matching logic works correctly for
cross-model regens and stitching.
- HunyuanFoley secs_per_step: 0.35 → 0.50. Measured wall time from
logs was 0.607 s/step (includes VAE decode + feature extraction per
segment, not just the denoising loop). More accurate estimate avoids
under-reserving ZeroGPU duration for multi-segment videos.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Free GPU memory between HunyuanFoley segments to prevent OOM
After each segment's denoise_process, explicitly del audio_batch and
visual_feats then call torch.cuda.empty_cache(). The 15-s audio latent
tensor is several GB; without explicit deletion PyTorch holds the CUDA
allocation until GC runs, causing OOM when the second segment allocates
its own latent. This is why seg 1 completed successfully but seg 2 failed
silently (ZeroGPU kills worker on OOM with no Python traceback).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Revert to single multi-seg GPU call; bump hunyuan load_overhead to 90s
- Raise HunyuanFoley load_overhead 55→90 s to account for cold-disk 10 GB
weight load (~73 s measured) plus aux model init (~8 s)
- Raise _clamp_duration floor 60→120 s — Pro ZeroGPU users get 300 s/call,
so 120 s floor safely covers worst-case cold-start without wasting budget
- Replace per-segment GPU call architecture (_hunyuan_gpu_infer_one_seg ×N)
with a single multi-segment @spaces.GPU call (_hunyuan_gpu_infer) that
loads the model once and loops over all segments — avoids reloading the
10 GB weights N times which would exceed the Pro time budget entirely
- Duration estimate now uses _estimate_gpu_duration("hunyuan", num_samples,
num_steps) which scales with actual work (segments × steps) + 90 s overhead
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HunyuanFoley initial gen: one GPU call per segment
ZeroGPU free tier caps GPU windows at 60s regardless of what is requested.
With 2 segments, model load (~19s) + seg1 (~13s) + seg2 (~13s) = ~45s, but
the Transformers cache migration on first call pushes it over 60s, causing
the worker to be silently killed mid-segment-2 with no error message.
Fix: replace the single multi-segment GPU call with one GPU call per segment.
Each call: model load + 1 segment inference = ~32s, comfortably under 60s.
Text features are saved to disk after the first segment and reloaded by
subsequent segments to avoid re-running CLAP/SigLIP extraction each time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HunyuanFoley: pre-download SigLIP2, use local_files_only=True
SigLIP2 (google/siglip2-base-patch16-512, ~1.5 GB) was being downloaded
from the HF network inside every ZeroGPU GPU worker call, consuming ~4s of
GPU budget and risking timeouts/rate-limits.
Fixes:
- Add _dl_siglip2() startup download alongside other model downloads
- Add to parallel download pool (max_workers 7→8)
- Add local_files_only=True to both from_pretrained call sites in
model_utils.py (class-based path and load_model function path)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix MMAudio: load BigVGAN from local snapshot dir, not HF network
BigVGANv2.from_pretrained('nvidia/bigvgan_v2_44khz_128band_512x') was hitting
the HF network inside the ZeroGPU worker because AutoEncoderModule hardcoded
the repo ID string for 44k mode, ignoring the vocoder_ckpt_path argument.
Fixes:
- _dl_bigvgan() now returns the local snapshot dir from snapshot_download()
- bigvgan_local_dir captured at startup alongside other checkpoint paths
- _load_mmaudio_models passes bigvgan_local_dir as bigvgan_vocoder_ckpt
- AutoEncoderModule (44k path) uses vocoder_ckpt_path as the from_pretrained
source when it points to a local directory, falling back to the repo ID
string only when no local path is provided
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HunyuanFoley: save text_feats to disk inside GPU worker
ZeroGPU forbids CUDA tensor deserialization in the main process. The previous
fix resolved the ModuleNotFoundError but text_feats contains CUDA tensors;
unpickling them in main triggers torch.cuda._lazy_init() which ZeroGPU blocks.
Fix: save text_feats via torch.save() inside the GPU worker, return the file
path string instead. Main process receives only numpy arrays + a string path.
Update _hunyuan_extras to use the pre-saved path directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HunyuanFoley xregen: register model lib sys.path in main process
ZeroGPU workers add model library dirs to sys.path via _ensure_syspath,
but this only affects the worker subprocess. When the worker pickles its
return value (which contains hunyuanvideo_foley objects like text_feats),
_ForkingPickler.loads() in the main process fails with:
ModuleNotFoundError: No module named 'hunyuanvideo_foley'
Fix: call _ensure_syspath for all three model libraries at module level
(immediately after the function definition) so the main process has all
model paths in sys.path before it ever deserialises a worker result.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix HunyuanFoley xregen AttributeError: return torch.device from _get_device_and_dtype
denoise_process calls torch.autocast(device_type=device.type, ...) which
requires a torch.device object. _get_device_and_dtype was returning a plain
string 'cuda', causing AttributeError: 'str' object has no attribute 'type'.
All other usages (.to(), map_location=, device=) work with torch.device too.
Add try/except traceback logging to _hunyuan_gpu_infer to surface AttributeError
TARO loads CAVP extractor, onset model, MMDiT, AudioLDM2 VAE, and BigVGAN
vocoder before inference. This took >60s total, exhausting the GPU budget
before _load_taro_models could complete. 120s overhead + inference time
gives enough headroom for all model loading + feature extraction.
Add granular step-by-step logging in _taro_gpu_infer to find exact GPU abort point
_build_regen_pending_html was using segments[i+1][0] as each segment's
right edge, while _build_waveform_html uses the contact-edge midpoint
(segments[i][1] + segments[i+1][0]) / 2. Align both to the same formula
so segment boundaries don't shift when the pending placeholder is shown.
Fix MMAudio TypeError: cast num_steps to int in _mmaudio_gpu_infer
FlowMatching(num_steps=num_steps) was receiving a string from Gradio,
causing 'can only concatenate str (not int) to str' in torch.linspace.
Add int(num_steps) cast alongside the existing num_samples/crossfade_s casts.
Fix TARO GPU worker crash: use local_files_only=True for audioldm2 VAE/vocoder
from_pretrained('cvssp/audioldm2') was hitting the HF network inside the
ZeroGPU worker to resolve cache, timing out and crashing. local_files_only=True
uses the pre-downloaded cache directly with no network calls.
Add xregen error logging: try/except in _xregen_dispatch, logging in xregen_mmaudio._run and _mmaudio_gpu_infer to surface GPU worker crashes
When xregen_mmaudio/xregen_hunyuan call _mmaudio_gpu_infer / _hunyuan_gpu_infer
with clip_start_s and clip_dur_s as positional args (positions 12/13 and 15/16
respectively), ZeroGPU forwards all args to the duration fn before running the
GPU fn. The duration fns only had 9/10 explicit params + **_kwargs — but **_kwargs
only absorbs *keyword* args, not extra positionals. This caused:
TypeError: _mmaudio_duration() takes from 9 to 10 positional arguments but 13 given
which aborted the GPU task before inference even started, causing the Video slot
to show the red Gradio 'Error' component.
Fix: add silent_video, segments_json, clip_start_s, clip_dur_s (and total_dur_s
for hunyuan) to the duration fn signatures with safe defaults. These params are
ignored in the duration calculation — they only exist to match the GPU fn's full
positional signature so ZeroGPU's internal dispatch doesn't blow up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
