Support for RTX 50xx (SM 12.0 / sm_120) + Windows fallback without bitsandbytes/torchaudio
Hi! I’m trying to run HeartMuLa-Studio on Windows 11 with an NVIDIA GeForce RTX 5070 (Compute Capability SM 12.0 / sm_120), and I’m hitting compatibility issues related to GPU architecture support and optional dependencies.
With stable PyTorch builds (e.g., torch 2.4.1 + cu121), PyTorch warns that sm_120 is not supported (it only supports sm_50…sm_90) and the app crashes with:
RuntimeError: CUDA error: no kernel image is available for execution on the device.
This looks like the current prebuilt binaries are not compiled with sm_120 kernels. There are existing PyTorch requests for official sm_120 support (Blackwell / RTX 50xx).
The project enables 4-bit quantization via bitsandbytes. The crash happens inside bitsandbytes during 4-bit quantization (also ending in “no kernel image”), so it likely needs a fallback path for GPUs/Windows setups where bitsandbytes does not support the device yet.
As a workaround I tried installing PyTorch nightly (cu128) to get sm_120 support, but on Windows + Python 3.12 there is no torchaudio wheel available from the nightly cu128 index:
ERROR: No matching distribution found for torchaudio.
That prevents the backend from starting because it fails on import torchaudio. Similar cu128/nightly issues have been reported in the community.
What I’m asking for:
Please add an explicit “fallback” run mode that disables bitsandbytes 4-bit (e.g., an env flag like USE_BNB_4BIT=0 / QUANTIZATION=none / FP16/BF16 path) so the project can still start on new GPUs even if bnb 4-bit isn’t compatible yet.
Please make torchaudio an optional dependency (safe import with try/except and a fallback audio I/O/resampling implementation, or at least allow the server to boot with reduced functionality when torchaudio is unavailable).
Please document the current situation for RTX 50xx (sm_120) on Windows: which PyTorch builds are expected to work, and recommended alternatives (nightly, WSL2/Linux, or build-from-source) until official wheels are available.
I can provide full logs and package versions if that helps. Thanks!