Running Drama 3 on a 16 GB AMD card (ROCm) – community project
Drama 3 officially needs NVIDIA with 48 GB, so I built a small local app that runs the original FP8 checkpoint on my RX 9070 (16 GB VRAM, 128 GB RAM). The MoE experts stay in system RAM and only the ones the router picks get moved to the GPU. It's slower than realtime (~13 s for 3 s of speech) and only tested on that one card so far.
Code: https://github.com/re133/red-curtain
Not affiliated with Fish Audio, no weights included, you download them from here under the Fish Audio Community License.
This is great, have you tested it with a GGUF version so we could run this on a 16gb vram gpu + 32gb of ram?? And on windows, not linux? Theres a Q2 version that uses like 15gb... surely its doable
16 GB VRAM is handy, but 128 GB RAM is a lot.
I made AiStudio tha supports it at 6GB Vram+16-32Gb Ram (via own converted ggufs) on c++ native engine