No v-ram or sys-rm reduction noticable
It seems to eat more memory with gguf in comfyui on my tests?
does comfyui upcast the weights back to their orignal filesize during inferring by default?
is there a way around that?
No the weights stay in quantized precision but each weight needs to be dequantized on the fly so activations would end up in higher precision. You could try using the advanced node to change dequant type, but tbh if you have enough vram to load the full precision then you should.
well, my system ram is what is actually holding me back. it runs super quick, but only 32 gig of system ram makes comfy swap a concerning amount of data to my ssd drives.
I'm afraid that good amount of inference will degrade my ssd faster than I would like? not that I ever experienced anything like that, just on my intuition.
Ill play around with the advanced node a bit, thank you
Oh interesting. What are your machine specs if you don’t mind sharing?