jayn7

/

Z-Image-Turbo-GGUF

like 288

Where and how do I run the quantized model? seriously i have no idea, I'm a beginner.

@Klasta you have ComfyUI, right? If not, download and install the latest release from https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.3.75
Then download example_workflow.json from this repo and open it in ComfyUI, it has instructions for the model files to download.

Now, I have to join @RKAAI in praising this model. It just keeps blowing my mind.

The latest thing is, I decided to try training a LoRA for it in ComfyUI. I only have 4 GB of VRAM and can train SDXL-size models only with a Q5 quant at best, so I totally expected it to never work but there it goes running the training iterations with Q3_K_S with my ancient GPU at the moment and it's hardly any slower than SDXL in doing so! It's only using about half the available VRAM which is weird because it did OOM earlier with a Q4. I changed the text encoder to a smaller quant but I didn't think that would matter because it should unload it in between... I have no idea. Strange and wonderful.

@hum-ma how are you training lora with gguf models ?

@rththr simply load the GGUF and plug into the Train LoRA node.

You could use a normal VAE encode instead of a TAE, and replace the CLIP loader with a regular one if you have a fast CPU. I need MultiGPU to make it run on the GPU because encoding captions on my ancient CPU takes a very long time.

However, it is very fragile. Works only in ComfyUI 0.3.75 and even then it sometimes breaks with a "RuntimeError: Inference tensors do not track version counter." if I change something in the dataset. I haven't been able to get the new training nodes in 0.3.76 to work with Z-Image GGUFs at all.

Maybe it's not even supposed to work because PyTorch gives an error about "only floating point tensors can require gradients" when trying to force a GGUF to load in full non-inference mode..?

@hum-ma thank you for the detailed guide , let me try to fire it up and see if it works or not

Update; Z-Image GGUF LoRA training works again in ComfyUI 0.4.0, which is cool because the new training workflow can accept dataset images in multiple resolutions.

Strangely it still gives the error about inference tensors with a minimal test set of only 4 small images, but not with sets of at least 20 images of different sizes.

Did it work for you at all @rththr ?

@hum-ma yes i was able to make it work , i dont have a powerful gpu and limited by 6 gb vram . To make a lora of a card model it took me roughly 45 minutes for 200 steps . Thanks alot without you it wouldnt have been possible . If you want i can send you my converted lora and the training settings.

@rththr oh, that's nice! I've been so caught up with trying to debug all these issues that I haven't really gotten familiar with the training settings yet.

@hum-ma https://huggingface.co/rththr/test/resolve/main/ComfyUI_trained_lora_kkkok_200_steps_00301_.safetensors, if you wanna try it , trigger word is kkkok , try a simple prompt like " kkkok , a car . "

@rththr looks like even without the trigger word it can change the default family cars into sleek sports cars, and they look good. A reasonable number of steps too, did you have to find a specific combination of settings for the results to be so clean?

@hum-ma i have tried few version of lora , the ones without the ostris lora adapter were blurry even at 300 steps , this one is trained on 4 golden lamborghini images with the lora adapter and the result turned out to be good most of the time time it generates lamborghini i have tried changing the colours and it works pretty well , let me know if you need the workflow for the training .