Will not fit on a 4090 GPU?
#6
by
sob3king
- opened
is there a workaround? or is this accurate?
SAM-Audio is fundamentally too large for the 4090 24GB card. Even the "small" model uses 21.7GB just for model weights because the encoder stack (PE-AV, RoBERTa, DAC-VAE) is identical across all sizes - only the separation head varies.
Small model: 21.7GB VRAM (leaves <2GB for inference)
Base model: ~23GB VRAM
Large model: ~45GB+ VRAM
SAM-Audio appears designed for 40GB+ A100/H100 cards?