Why does the distilled version weigh more than the dev version?
#1
by nosok12313 - opened
I mean, the quantization method is identical, so what's the deal? And would running the dev version + distilled LoRA actually reduce peak VRAM?
Also, been dying to know - what's the method/tool for quantizing down to w4a8?
I accidentally changed a value on the quantization, reuploading now. Currently, I only know https://github.com/NidAll/comfyui-mixed-quantizer and https://github.com/Starnodes2024/comfyui-starnodes-modelconverter these two. I've forked my own converter but haven't pushed the w4a8 update yet, it has old int4 mixed with int8 quantization not the new w4a8 https://github.com/tsolful/ComfyUI-INT8-Fast