Could you guys try making imatrix and UD quants of diffusion models?
#1
by mingyi456 - opened
I think I have seen 1 person uploading Flux.1-dev GGUF quants with imatrix here: (https://huggingface.co/Eviation/flux-imatrix). Could you guys test to see if imatrix indeed improves GGUF quantization accuracy for diffusion models? Also, UD quants would be good, because many models fit into 24GB of VRAM with plenty of room to spare at 8 bit, but 16 bit is too much.
Finally, I noticed you guys uploaded quants for Flux.1-dev and schnell as well. Will support for finetuning diffusion models with Unsloth be coming soon?
Hi there we were trying to make it work but it'll take more time we'll update you. Diffusion models should actually already work in Unsloth but it'll need more manual adjustments. We might be supporting diffusion models soon yes