Pre-extracted tmp clips (/tmp/xregen_*_clip.mp4) created in the caller
process are invisible to the ZeroGPU GPU worker (separate process, fresh
/tmp). Fix: pass source_video + clip_start_s/clip_dur_s as positional
args; GPU fns extract the xregen clip internally before sub-segment clips.
Also convert all remaining silent_video/segments_json/total_dur_s kwargs
to positional args in gpu_infer calls (kwargs silently dropped by ZeroGPU).
Fix ZeroGPU kwarg stripping: make silent_video/segments_json positional
ZeroGPU only forwards positional args to the GPU worker process — kwargs
are silently dropped, arriving as None and causing json.loads(None) crash.
Make silent_video, segments_json (MMAudio) and silent_video, segments_json,
total_dur_s (HunyuanFoley) required positional params in the gpu_infer
functions so ZeroGPU passes them through correctly.
Fix 'Too many arguments': use **_kwargs in duration fns for extra params
Gradio introspects the duration callable's signature to determine the
expected input count for @spaces.GPU endpoints. Adding silent_video,
segments_json, total_dur_s as explicit params to _mmaudio_duration and
_hunyuan_duration caused Gradio to register 11/13 expected inputs vs the
9/10 actually wired in .click(inputs=[...]), triggering client-side
'Too many arguments' on every page load.
Fix: absorb the extra kwargs with **_kwargs in both duration functions
so Gradio sees only the 9/10 positional params matching the UI inputs.
Fix regen/xregen errors when silent_video tmp file is missing
Add _resolve_silent_video(meta) helper that falls back to re-stripping
the original Gradio upload (source_video) when silent_video has been
evicted from tmp or the Space restarted since generation.
- Store source_video (original video_file upload path) in seg_meta via
_build_seg_meta and _post_process_samples for all three models.
- Replace all meta[silent_video] direct reads in regen/xregen paths
with _resolve_silent_video(meta) — covers _splice_and_save,
regen_taro/mmaudio/hunyuan, and xregen_taro/mmaudio/hunyuan.
Fix waveform crossfade hatch + click hit-detection to use contact edges
- Crossfade hatch was centered on segments[i+1][0] (old algo edge);
now centred on contactEdges[i] = (seg[i][1]+seg[i+1][0])/2,
which is where _stitch_wavs actually applies the blend.
- Click hit-detection used segments[idx+1][0] as boundary; now uses
contactEdges[idx] to match the visual coloring exactly.
Fix ZeroGPU isolation for MMAudio/HunyuanFoley regen + waveform contact-edge fix
- Pass silent_video, segments_json, total_dur_s as explicit params to
_mmaudio_gpu_infer and _hunyuan_gpu_infer; extract segment clips inside
the GPU fn (ffmpeg is CPU-safe inside GPU window). Removes _ctx_store/
_ctx_load for both models — root cause of xregen GPU task aborts.
- xregen_mmaudio/xregen_hunyuan pass params directly without _ctx_store.
- Fix waveform color boundaries: use contact edges (seg[i][1]+seg[i+1][0])/2
instead of seg[i+1][0]+crossfade/2 (wrong with equal-spacing algo).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix NameError: restore import html as _html used in _build_waveform_html
- Remove unused ; move to top-level
- Add _resolve_seed() helper to unify inconsistent seed handling across
TARO/MMAudio/HunyuanFoley (was: 3 different patterns; now: one call)
- Remove redundant torch.device() wrapping in _hunyuan_gpu_infer and
_regen_hunyuan_gpu (device is already a string from _get_device_and_dtype)
- Fix all unregistered mkdtemp() calls in regen GPU functions so temp dirs
are tracked and cleaned up (prevents /tmp accumulation on long-running Spaces)
- Fix misleading comments in _xregen_splice offset alignment logic
xregen: use target model's optimal window centered on segment midpoint
Each xregen_* now:
1. Computes optimal clip window centered on segment midpoint (clamped to video)
2. Runs _build_segments on that clip with target model's window size
3. Calls full generation GPU pipeline (same path as initial generation)
4. Stitches sub-segments with _stitch_wavs + contact-edge trimming
5. Returns (wav, sr, clip_start_s) so _xregen_splice can align to original grid
Handles all cases: target window > span (single inference), target window <
span (multiple sub-segments), video shorter than target window (clamped).
Remove pre-truncation from all inference paths — stitch_wavs owns trimming
All generate_* and regen GPU functions now return full window audio.
_stitch_wavs is the single place that trims to contact-edge windows
before crossfade-join, ensuring regen respects the same segment timing
as the original generation.
Fix segment bleed: equal-spaced windows + contact-edge trimming
_build_segments now places n equally-spaced full-window segments so every
overlap is identical and >= crossfade_s, eliminating the unequal last-
boundary overlap that caused raw bleed.
_stitch_wavs now accepts segments list and trims each generated wav to its
contact-edge window (midpoint of overlap +/- half_cf) before crossfade-join,
so the crossfade zone is always exactly crossfade_s wide at every boundary.
_regen_mmaudio_gpu and _regen_hunyuan_gpu now call _extract_segment_clip
internally (pure ffmpeg, safe within GPU window) instead of relying on
_ctx_store/_ctx_load cross-process context passing.
Removes redundant _extract_segment_clip + _ctx_store calls from CPU
wrappers: regen_mmaudio_segment, regen_hunyuan_segment, xregen_mmaudio,
xregen_hunyuan.
fix: remove ctx_key from all function signatures — use fn-name-keyed global dict
ctx_key as a function argument exposed it to Gradio's API endpoint discovery,
causing 'Too many arguments provided for the endpoint' errors and GPU task aborts.
Fix: remove ctx_key from all @spaces.GPU function signatures and their duration
callables. Store/retrieve context using _ctx_store(fn_name, data) /
_ctx_load(fn_name) — a global dict keyed by function name. This is safe because
ZeroGPU is synchronous (wrapper blocks until GPU fn returns), so only one call
per GPU function is in-flight at a time within a single process.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: use UUID-keyed global dict + ctx_key param for ZeroGPU context passing
ctx_json arg was being set to None by ZeroGPU because the duration callable
signature didn't include it — ZeroGPU validates/forwards args against the
duration fn signature and silently drops any extra params not present there.
Fix: add ctx_key='' to ALL duration callables (not just GPU fns) so the
param survives ZeroGPU's arg-forwarding pipeline. Use a UUID-keyed global
dict (_GPU_CTX) instead of JSON-encoding the full context — the UUID is a
tiny hex string that round-trips safely through any arg marshalling, and the
global dict is readable from any thread (unlike threading.local).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: pass CPU context as ctx_json argument to @spaces.GPU functions
ZeroGPU runs GPU functions on its own worker thread pool — thread-local
storage and per-thread dicts both fail because the writer and reader are
on different threads with different thread IDs.
The only reliable approach: pass context as a JSON string argument.
ZeroGPU forwards all arguments to the GPU worker unchanged.
Changes:
- Add ctx_json='{}' parameter to all 6 @spaces.GPU functions
(_taro_gpu_infer, _mmaudio_gpu_infer, _hunyuan_gpu_infer,
_regen_taro_gpu, _regen_mmaudio_gpu, _regen_hunyuan_gpu)
- Each wrapper serialises its pre-computed data to json.dumps({...})
and passes it as the last positional argument
- GPU functions parse with json.loads(ctx_json)
- TARO/HunyuanFoley regen: numpy/tensor features loaded directly from
disk paths already stored in seg_meta_json — no pre-serialisation needed
- Remove dead _preload_taro_regen_ctx, _preload_hunyuan_regen_ctx helpers
- Remove _CTX/_ctx_set/_ctx_get infrastructure (replaced by arg passing)
Co-Authored-By: Claude Sonnet…
fix: replace threading.local with caller-thread-id dict for ZeroGPU context passing
ZeroGPU dispatches @spaces.GPU functions on its own worker thread pool, not
the Gradio handler thread. threading.local() values are invisible across
threads, causing AttributeError when the GPU worker reads the context.
Replace _tl (threading.local) with _CTX (dict keyed by caller thread ID)
+ _ctx_set/_ctx_get helpers. The caller writes context under (tid, key)
before invoking the GPU function; the GPU worker reads using the same tid
(ZeroGPU runs synchronously on behalf of the caller, so no concurrent write
occurs). Entries are popped on read to prevent unbounded growth.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dropping to 20s caused MMAudio regen to request only 36s, which isn't
enough when open_clip downloads inside the GPU window on cold workers.
The 60s floor was the working value in 813c771 and needs to stay until
we confirm the snapshot_download pre-cache is actually effective.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix: pre-download MMAudio CLIP model at startup to avoid GPU window download
apple/DFN5B-CLIP-ViT-H-14-384 (3.95GB) was being downloaded by open_clip
inside the ZeroGPU GPU window on cold workers, consuming ~5-10s of the
allocated budget before inference started. Pre-download via snapshot_download
at startup so it reads from cache inside the GPU window, same pattern as
the existing CLAP pre-download. Reverts MMAUDIO_LOAD_OVERHEAD back to 30s.
Also adds _regenInFlight per-slot guard to prevent queuing multiple regen
jobs from rapid re-clicks on the same slot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ZeroGPU optimizations, refactoring, and regen UX improvements
ZeroGPU optimizations:
- Pre-load TARO CAVP/onset features on CPU via _preload_taro_regen_ctx()
- Pre-load HunyuanFoley text_feats on CPU via _preload_hunyuan_regen_ctx()
- Move FlowMatching creation outside per-segment loop in MMAudio
- Replace fallback video extraction in regen GPU fns with asserts
- Tighten TARO_SECS_PER_STEP 0.05→0.025 (measured 0.023s/step)
- Drop regen duration floor 60s→20s (cold-start spin-up not in timer)
Refactoring:
- Extract _post_process_samples() shared post-processing for all 3 models
- Unify mux_video_audio() to handle HunyuanFoley internally via model= param
- Extract _preload_taro_regen_ctx / _preload_hunyuan_regen_ctx helpers
- Make _resample_to_target() accept dst_sr; _resample_to_slot_sr delegates to it
Regen UX:
- Flash red border on waveform for 3s when regen aborts
- Show 'GPU cold-start — segment unchanged, try again' instead of misleading
'Quota exceeded' for GPU task aborted errors
- Status bar and seg label both auto-clear after 8s
Co-A…
Fix xregen_mmaudio abort: raise duration floor and load overhead
MMAudio regen was aborting (GPU task aborted, duration=10) because
the ZeroGPU allocation of 30s was too tight when open_clip model
(3.95GB) needed to download on cold start inside the GPU window.
- MMAUDIO_LOAD_OVERHEAD: 15 → 30s (covers cold-start download)
- Regen duration floor: 30 → 60s (matches initial gen minimum,
prevents abort when any model needs a cold-start download)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unify all models to 48kHz; remove all FlashSR traces
FlashSR and its transitive imports (torchaudio, FastAudioSR) trigger
torch.cuda.is_available() during module import, which violates
ZeroGPUs stateless-GPU rule and aborts all subsequent GPU tasks.
Replace _apply_flashsr with torchaudio.functional.resample (sinc,
CPU-only, no CUDA risk). Output is still 48kHz. Remove FlashSR from
requirements.txt and clean up unused _FLASHSR_MODEL/_FLASHSR_LOCK.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix HunyuanFoley CLAP tokenizer overflow for long prompts
CLAP text encoder has a 512-token max but encode_text_feat() called
the tokenizer without truncation=True/max_length, causing a tensor
shape mismatch when prompts exceed 512 tokens (523 > 512 error).
Added truncation=True, max_length=512 to the tokenizer call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix FlashSR CUDA init: bypass FASR class, load SynthesizerTrn on CPU
FASR.__init__ calls torch.cuda.is_available() and .to(device),
which initializes CUDA in the main process and violates ZeroGPU
stateless-GPU rule — aborting all subsequent GPU tasks.
Now we load SynthesizerTrn directly on CPU, replicating the same
hyperparams and normalization that FASR uses, without touching CUDA.
This allows FlashSR to run safely in the CPU post-processing step
outside @spaces.GPU, saving GPU quota per segment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Optimize ZeroGPU: move FlashSR to CPU, fix regen 16kHz bug
- Move FlashSR upsampling (16kHz→48kHz) from inside @spaces.GPU
to CPU wrappers — saves ~1-2s GPU quota per segment since
FlashSR is CPU-only and doesn not need the GPU allocation
- Fix bug: regen_taro_segment and xregen_taro were returning
raw 16kHz wav without applying FlashSR (initial gen applied
it inside the GPU loop but regen never did)
- Reduce TARO regen duration estimate when CAVP/onset feature
cache exists — requests 5s overhead instead of 15s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- FlashSR upsampling (16kHz→48kHz) now runs inside _taro_gpu_infer
under @spaces.GPU to comply with ZeroGPU CUDA-init rules
- Remove stale _apply_flashsr calls from xregen_taro and regen_taro_segment
(FlashSR already applied per-segment inside _taro_infer_segment)
- Add api_name to queue/join fetch body to fix xregen "Too many arguments"
routing issue in Gradio 5
- Extract _build_seg_meta, _cpu_preprocess, _save_wav, _log_inference_timing
helpers; replace inline stitch with _stitch_wavs
- Cap crossfade slider max at 4s + safety clamp in _build_segments
- Add diagonal hatch waveform indicator for crossfade overlap zones
- Remove HF Token settings accordion (ZeroGPU attribution via JWT)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: load FlashSR on CPU only to avoid ZeroGPU CUDA init violation
ZeroGPU forbids CUDA initialization outside @spaces.GPU decorated
functions. FlashSR was calling torch.cuda.is_available() and .cuda()
in the main process, which poisoned subsequent @spaces.GPU calls
(causing 'GPU task aborted' on xregen). FlashSR is tiny (1.72MB) and
fast enough on CPU — no GPU needed for the upsampling step.
remove: HF token settings section and JS fallback
ZeroGPU quota attribution works automatically via the HF JWT session
(confirmed in HAR — x-zerogpu-token/uuid headers present on all regen
calls). Manual token input is redundant.
feat: add FlashSR post-processing to upsample TARO 16kHz → 48kHz
All three models now output at 48kHz (TARO via FlashSR, MMAudio at
44.1kHz natively resampled, HunyuanFoley at 48kHz natively).
FlashSR is applied after generation and after each regen/xregen on
TARO outputs. Console logs confirm each upsampling step with duration
and sample rate. Falls back to sinc resampling if FlashSR errors.
fix: remove CF label from crossfade hatch indicator
Color edge was at seg[i+1].start (crossfade begins) but should be at
seg[i+1].start + crossfade/2 — the 50/50 blend point, which matches
where the cut would be with no crossfade and where listeners perceive
the transition.
fix: centre crossfade hatch on color boundary, not after it
Hatch was drawn at [seg[i+1].start, seg[i].end] which sat entirely inside
seg i+1's color block — visually after the edge. The color boundary IS at
seg[i+1].start, so hatch is now centred there: half crossfade on each side.
fix: correct crossfade indicator position on waveform
Overlap zone is [seg[i+1].start, seg[i].end] — both halves occupy the
same time range, so the hatch must be drawn there (end of seg i), not
at the start of seg i+1. Previously appeared only 'after c' because
overlapStart was used instead of the correct seg[i].end boundary.
feat: add crossfade overlap indicators to waveform display
Draws diagonal hatch pattern with 'CF' label over overlap zones between
adjacent segments so users can visually identify where crossfade blending
occurs. Helps correlate audio artifacts with crossfade regions during testing.
fix: cap crossfade slider max to 4s and clamp in _build_segments
Crossfade duration max was 8s — equal to MMAudio's window (8.0s) and
nearly equal to TARO's (8.192s), which would produce step_s <= 0 and
degenerate segments. With cross-model regen the crossfade must be safe
for the smallest window. Cap sliders to 4s and add a safety clamp in
_build_segments (crossfade <= half window) to guard against stale metadata.
refactor: extract shared helpers to reduce technical debt
