Weight size VS VRAM requirements
Hello, I'd be interested to hear about what makes the model's inference require the large amount of VRAM ( 4 x 80GB )?
Weight is about 40GB and logically fits into 48GB of VRAM with enough leftover for a short* context.
Very promising model, great job from the team.
Thanks!
The model requires a huge sequence length for generating videos (44.5K tokens) -- which takes memory.
Also the VAE is a massive memory-hog.
But, we've reduced the requirements, so it is now possible on a single 4090.
if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo team
@ved-genmo great work folks! It would be super helpful if you guys can dedicate a section in Readme about hardware & time requirements to run this.
The model requires a huge sequence length for generating videos (44.5K tokens) -- which takes memory.
Also the VAE is a massive memory-hog.
But, we've reduced the requirements, so it is now possible on a single 4090.
That's awesome, thanks for sharing the details.
44.5k tokens, impressive!
if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo team
Excellent does this mean this will not be able to run on a m2 mac 32gb?
if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo teamExcellent does this mean this will not be able to run on a m2 mac 32gb?
not sure. sorry. It will prolly depend on the the base footprint of the OS.
On the mac question, FYI:
You can utilize 75% of the unified memory on the Mac by default (you can update this setting but not suggested)