Out of curiosity how much memory (RAM/VRAM) you needed to quantize this model?
I ran this as on a workstation, 2 Rtx pro 6k BW, Xeon, 1TB ram.
It was really tight and I had to run sequential.
· Sign up or log in to comment
