- Replace DOM event dispatch (which Svelte ignores) with direct
POST to /gradio_api/queue/join using the handler's api_name
- Remove hidden trigger textboxes and relay pattern (no longer needed)
- Embed state_json in waveform HTML data-state attribute so JS reads it
- Add elem_id to all input components so JS can read their DOM values
- Register per-slot regen handlers via gr.Button.click() with api_name
so they get stable fn_index entries in gradio_config.dependencies
- Simplify _make_output_slots to just (grps, vids, waveforms)
- Update _splice_and_save to embed updated state in returned waveform HTML
Fix fireRegen: use correct prototype for textarea native setter
Gradio 5.25 SSR is experimental and its client-side argument count
validation (map_data_to_params) rejects .change() calls before they
reach Python, regardless of whether input/output counts are correct.
Disabling SSR (ssr_mode=False in launch()) bypasses this validation
entirely and lets .change() events reach the Python handlers normally.
Also adds startup prints to confirm logging is working and all 8
per-slot regen handlers are registered at app startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs causing regeneration to never reach Python:
1. clear-then-set fired TWO .change() events. The first (empty string)
caused the generator to hit a bare `return` — but Python generators
with a bare return raise StopIteration immediately, which Gradio 5
SSR interprets as an SSE stream error. This errored state blocked
the second (real value) call from completing.
Fix: remove the clear step entirely; use a timestamp suffix in the
trigger value so repeat clicks always produce a unique string and
React's change detection always fires.
2. Early-exit paths used `return tuple` inside a generator function.
In Python, any function containing `yield` is always a generator —
`return value` inside a generator is silently dropped (only
StopIteration is raised). Gradio 5 SSR then sees an empty stream.
Fix: change all early-exit paths to `yield ...; return` so Gradio
receives a proper final update before the stream closes.
Trigger format is now: "slot_id|s…
Fix regen: embed state in trigger value, remove seg_state from inputs
- Replace visible=False textboxes with CSS-hidden (elem_classes=['wf-hidden-input'])
so Gradio 5 SSR renders them to the DOM (visible=False omits them entirely,
making getElementById return null and fireRegen silently fail)
- Remove regen trigger from regen handler outputs — having a .change() component
as its own output causes SSR argument count mismatch
- Drop 4th gr.update() from all regen yield statements to match 3-output list
- Add .wf-hidden-input CSS class: position absolute off-screen, opacity 0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix fireRegen: clear+set pattern for Gradio 5 React .change()
In Gradio 5 SSR mode, including gr.Group updates in the same outputs list
as gr.Video/gr.HTML/gr.State causes argument count mismatches. Fix by:
- Removing gr.Group from btn.click() outputs (now: vids+waves+states = 24)
- Adding .then(fn=_update_slot_visibility) after click for group visibility
- Applies to all three tabs: TARO, MMAudio, HunyuanFoley
Fix waveform: popup via postMessage, regen, playhead, Too many args
Gradio's gr.HTML updates via innerHTML which strips/ignores <script>
tags. Moving canvas + Web Audio API logic into an iframe srcdoc ensures
scripts always execute on every update. The iframe accesses window.parent
directly (same-origin sandbox) to trigger the Gradio regen hidden input.
Playhead sync polls window.parent for the video element every 50ms.
Replaces the three separate per-tab gr.Video components with a single
shared gr.Video above the tabs backed by a gr.State. All three model
tabs now read from the shared state, so switching tabs no longer resets
the uploaded video. User must explicitly clear it with the X button.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both models were passing video_file (with original audio) to ffmpeg
when slicing segments and muxing the final output. Added strip_audio_from_video
call at the start of both functions, matching TARO's existing approach.
Calibrate GPU duration constants from measured H200 timings
Wraps the segment inference loop in each generate_* function with
time.perf_counter() and prints actual wall-clock time, steps/segs,
and measured s/step alongside the current constant after every run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Lower GPU duration estimates and add 300s hard cap
SECS_PER_STEP estimates were calibrated for a slower GPU. H200 is much faster:
TARO: 2.5 → 0.8 s/step
MMAudio: 2.5 → 0.8 s/step
HunyuanFoley: 5.0 → 2.0 s/step
Load overheads trimmed similarly (TARO 30→20, MMAudio 20→15, Hunyuan 30→20).
Add GPU_DURATION_CAP=300s — a single call can never reserve more than 5 min
regardless of video length or sample count, preventing quota exhaustion on
long videos. The diagnostic print now shows both the raw calc and capped value.
56s video / 1 sample / 25 steps: was 592s reserved, now 200s.
Fix dynamic duration arg order mismatch + leftover torchaudio import
Duration callables were receiving args in .click() positional order but
signatures had num_samples before crossfade_s/crossfade_db, causing
crossfade values to be interpreted as num_samples (inflating reservation).
Fix: reorder _mmaudio_duration, generate_mmaudio, _hunyuan_duration,
generate_hunyuan signatures to match .click() input order exactly:
...num_steps, crossfade_s, crossfade_db, num_samples
Update _run_mmaudio and _run_hunyuan wrappers to pass args positionally.
Move crossfade_s/crossfade_db casts to top of each generate function,
remove duplicate casts that were further down. Remove stray torchaudio
import inside generate_mmaudio (already top-level). Add diagnostic prints
to all three duration callables to log actual reservation in logs.
Standardise crossfade defaults to 2s/3dB across all three models
MMAudio crossfade_s was 1.0 in function signature, duration callable, and
UI slider default — align to 2.0 to match TARO and HunyuanFoley.
All model inference defaults (CFG, steps, mode) already matched paper
recommendations: TARO 8.0/25/SDE, MMAudio 4.5/25, HunyuanFoley 4.5/50.
Dynamic GPU duration: reserve only what each run actually needs
Replace fixed @spaces.GPU(duration=600) on all three models with per-call
duration callables (_taro_duration, _mmaudio_duration, _hunyuan_duration).
Each callable runs before GPU allocation (no GPU cost) and computes:
num_samples x num_segments x num_steps x secs_per_step + load_overhead
num_segments is derived from the actual video duration + crossfade setting,
so a 5s clip with 1 sample reserves ~60s instead of 600s. Worst-case long
videos with many samples still get the time they need.
Per-step estimates: TARO=2.5s, MMAudio=2.5s, HunyuanFoley=5.0s.
Load overheads: TARO=30s, MMAudio=20s, HunyuanFoley=30s. Floor is 60s.
Cleanup: remove duplication and dead code across app.py
- Remove [DIAG] print left over from debugging
- Move torchaudio to top-level import; drop per-function import torchaudio
inside generate_mmaudio and generate_hunyuan
- Replace sf.write in generate_taro with torchaudio.save (consistent with
other two models); drop soundfile import
- Remove _taro_build_segments wrapper — was a one-liner; call _build_segments
with TARO_MODEL_DUR directly at both call sites
- Drop MMA_CF_S/MMA_CF_DB and CF_S/CF_DB local aliases in generate_mmaudio
and generate_hunyuan — cast crossfade_s/crossfade_db once at function top
- Extract _make_output_slots() — builds the 8-slot video+audio output column;
replaces identical 7-line loop duplicated across all 3 tabs
- Extract _unpack_outputs(flat, n) — turns _pad_outputs list into Gradio
update lists; replaces identical 4-line block in all 3 _run_* functions
Unify mono and stereo crossfade into a single _cf_join function
Replace separate _crossfade_join (mono/TARO) and _cf_join_stereo (stereo/
MMAudio+HunyuanFoley) with one _cf_join that handles both shapes via a.ndim
check: stereo (C,T) uses axis=1 slicing, mono (T,) uses 1D indexing.
Update _stitch_wavs to accept an sr parameter and call _cf_join, and update
its call site in generate_taro to pass TARO_SR explicitly.
Refactor: consolidate segmentation and crossfade into shared helpers
- Add _build_segments(total, window_s, crossfade_s) as universal segmentation
helper used by all three models (TARO via thin wrapper, MMAudio and
HunyuanFoley directly with their respective window constants)
- Add _cf_join_stereo(a, b, cf_s, db, sr) as shared equal-power crossfade for
stereo (C, T) arrays; MMAudio and HunyuanFoley both call this instead of
duplicating the same inline closure
- Remove duplicate _mma_build_segments and inline _cf_join closures from
generate_mmaudio and generate_hunyuan
- _taro_build_segments now delegates to _build_segments; TARO keeps its own
mono _crossfade_join/_stitch_wavs since it outputs 1D not (C,T)
Add crossfade Duration/Boost controls to MMAudio and HunyuanFoley tabs
- Add Crossfade Duration (s) slider and Crossfade Boost (dB) textbox to both
MMAudio and HunyuanFoley UI tabs, matching TARO existing controls exactly
- Update generate_mmaudio() and generate_hunyuan() signatures to accept
crossfade_s and crossfade_db parameters (same defaults as before)
- Replace hardcoded MMA_CF_S/MMA_CF_DB and CF_S/CF_DB with user-supplied values
passed through _run_mmaudio and _run_hunyuan wrappers
- All 3 models now share identical crossfade UI and equal-power implementation
Fix crossfade: use equal-power fade envelopes in all 3 models
All three models had flat summation (both segments at full volume
simultaneously) during the overlap region, causing loud bumps at
segment joints. Replace with equal-power crossfade using cos/sin
fade-out/fade-in envelopes, which maintains constant perceived
loudness through the transition. Applies to TARO, MMAudio, and
HunyuanFoley.
Fix audio WAV header: ensure contiguous memory layout before torchaudio.save
numpy array slicing can produce non-contiguous memory; torch.from_numpy
on a non-contiguous array can write a malformed WAV header causing the
browser audio player to report wrong duration. np.ascontiguousarray()
forces a contiguous copy before saving. Reverts unnecessary 44100 Hz
resample.
HunyuanFoley: resample standalone audio output to 44100 Hz
48 kHz WAV files can confuse browser audio players and show wrong
duration (e.g. 0:15 instead of 0:51). Resample the audio-only output
to 44100 Hz before saving. The muxed video audio is unaffected.
load_video(video_file, duration) hard-caps to the duration param, so
generation was silently truncated to 8 s for any longer video. Fix:
segment the input with ffmpeg into overlapping <=8 s clips, run
generate() on each, and crossfade-stitch into a full-length track.
Also remove the Duration slider from the UI — window size is fixed at
8 s (MMAudio's native window) and segmentation handles long videos.
HunyuanFoley: sliding-window segmentation for videos longer than 15 s
The model is hard-limited to 15 s per pass (MAX_VIDEO_DURATION_SECONDS=15
in constants.py, enforced in get_frames_av). For longer videos, slice the
input with ffmpeg into overlapping <=15 s segments, run feature_process +
denoise_process on each, then crossfade-stitch all segment wavs into a
single full-length audio track — same strategy as TARO. Text features are
encoded once from the first segment and reused across all segments.
Fix CLAP from_pretrained daemonic process error in ZeroGPU worker
Monkey-patch multiprocessing.Process.start to run inline before calling
ClapTextModelWithProjection.from_pretrained(), restoring it immediately
after. Prevents transformers from forking a subprocess for safe-tensor
verification, which fails in ZeroGPU daemonic workers.
Pre-download CLAP model at startup to fix daemonic process error in ZeroGPU
ClapTextModelWithProjection.from_pretrained() internally spawns a child
process for safe-tensor conversion, which fails inside ZeroGPU's daemonic
worker. Pre-downloading laion/larger_clap_general via snapshot_download()
at module level ensures weights are cached before GPU function runs.
