Spaces:
Running on Zero
Running on Zero
Fix rotary/kvcache OOB with speaker_audio + CUDA/CPU device mix
#13
by multimodalart HF Staff - opened
Two bugs that surface on Blackwell + torch 2.10:
setup_cachewas allocated toseq_len, butdelayed_codes(afterapply_delay_pattern) is 9 frames longer. Generation overruns the rotary table / kv cache on the last few steps — usually invisible with the upstream flash_attn (its kvcache kernel silently clamped), but explodes asvectorized_gather_kernel: index out of boundson Blackwell where the SDPA shim does explicitcos = rotary_cos[positions].remaining_steps[mask] = torch.minimum(remaining_steps[mask], torch.tensor(9))mixes a CUDA tensor with a CPUtorch.tensor(9). torch 2.10 device-asserts on this..clamp(max=9)is equivalent and stays on-device.
Verified end-to-end via dev-mode SSH and an outside smoke test.
multimodalart changed pull request status to merged