BoxOfColors Claude Sonnet 4.6 commited on
Commit
359f37a
·
1 Parent(s): 6446441

Fix (1,T) shape crash in _cf_join; bump HunyuanFoley secs_per_step to 0.5

Browse files

- _load_seg_wavs: squeeze (1, T) → (T,) on load. HunyuanFoley's DAC
decoder emits (batch, 1, time); audio_batch[0] = (1, T). When these
single-channel arrays are saved as npy and later loaded alongside
true stereo (2, T) from MMAudio, _cf_join tries to concatenate
(2, N) with (1, M) along axis=1 → ValueError. The squeeze at load
time normalises all single-channel wavs to plain mono (T,) so
_resample_to_slot_sr's channel-matching logic works correctly for
cross-model regens and stitching.

- HunyuanFoley secs_per_step: 0.35 → 0.50. Measured wall time from
logs was 0.607 s/step (includes VAE decode + feature extraction per
segment, not just the denoising loop). More accurate estimate avoids
under-reserving ZeroGPU duration for multi-segment videos.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +14 -3
app.py CHANGED
@@ -328,8 +328,19 @@ def _save_seg_wavs(wavs: list[np.ndarray], tmp_dir: str, prefix: str) -> list[st
328
 
329
 
330
  def _load_seg_wavs(paths: list[str]) -> list[np.ndarray]:
331
- """Load segment wav arrays from .npy file paths."""
332
- return [np.load(p) for p in paths]
 
 
 
 
 
 
 
 
 
 
 
333
 
334
 
335
  # ------------------------------------------------------------------ #
@@ -552,7 +563,7 @@ MODEL_CONFIGS = {
552
  "hunyuan": {
553
  "window_s": 15.0, # HunyuanFoley max video duration
554
  "sr": 48000,
555
- "secs_per_step": 0.35, # measured 0.328 s/step on H200
556
  "load_overhead": 90, # cold disk: ~73s for 10 GB weights + ~8s aux models
557
  "tab_prefix": "hf",
558
  "label": "HunyuanFoley",
 
328
 
329
 
330
  def _load_seg_wavs(paths: list[str]) -> list[np.ndarray]:
331
+ """Load segment wav arrays from .npy file paths.
332
+
333
+ Normalises (1, T) arrays → (T,) mono so that single-channel output from
334
+ models like HunyuanFoley (DAC decoder emits shape (1, T)) never causes a
335
+ shape mismatch in _cf_join when mixed with true stereo (2, T) arrays.
336
+ """
337
+ wavs = []
338
+ for p in paths:
339
+ w = np.load(p)
340
+ if w.ndim == 2 and w.shape[0] == 1:
341
+ w = w.squeeze(0) # (1, T) → (T,) mono
342
+ wavs.append(w)
343
+ return wavs
344
 
345
 
346
  # ------------------------------------------------------------------ #
 
563
  "hunyuan": {
564
  "window_s": 15.0, # HunyuanFoley max video duration
565
  "sr": 48000,
566
+ "secs_per_step": 0.50, # wall-time ~0.607s/step (incl. VAE decode + feature extraction)
567
  "load_overhead": 90, # cold disk: ~73s for 10 GB weights + ~8s aux models
568
  "tab_prefix": "hf",
569
  "label": "HunyuanFoley",