Convert to ComfyUI format ~

#3
by sunnyboxs - opened

This is a fantastic project. If it could also be converted to ComfyUI format, it would make testing much more convenient for everyone.
Great job!

what do you mean?

This is currently not ComfyUI Compatible.

You can ask ChatGPT to make a script to fix the LORA, ask it "write me a python script for a .safetensor LORA file that adds the prefix "diffusion_model" to every layer"

run the python script you get on the lora and it will now work. Creds to Silvercoin on the Lodestone Discord for figuring out that the problem was simply that the layers in the model was missing the prefix.

If you're using the portable version of python, place this in the python_embeded folder along with the lora from this repo and run:

python.exe convert_h3_lora_for_comfyui.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

It works with the bf16 and int8 (non-pruned) versions only β€” pruned versions are also supported just ignore the errors about adaln_proj.linear.weight shape

import sys
from safetensors.torch import load_file, save_file

def convert(in_path: str, out_path: str, prefix: str = "diffusion_model."):
    sd = load_file(in_path)

    new_sd = {}
    skipped = 0
    for k, v in sd.items():
        if k.startswith(prefix):
            new_key = k  # already prefixed, leave as-is
            skipped += 1
        else:
            new_key = prefix + k
        new_sd[new_key] = v

    save_file(new_sd, out_path)

    print(f"Converted {len(sd)} keys ({skipped} already had the prefix).")
    print("Sample before -> after:")
    for i, k in enumerate(list(sd.keys())[:3]):
        print(f"  {k}  ->  {prefix + k if not k.startswith(prefix) else k}")
    print(f"\nSaved: {out_path}")

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(1)
    convert(sys.argv[1], sys.argv[2])

pruned versions not supported?

pruned versions not supported?

image

I'm getting this so I thought it's not working but I tested and if we ignore the errors the Lora is working on pruned versions too.

It worked well for me after I used this script.

https://huggingface.co/Evados/DiffSynth-Studio-Lora-Wan2.1-ComfyUI/blob/main/minimax_h3_turbo_4step_testv1_comfy.safetensors

import sys
from safetensors.torch import load_file, save_file

input_lora = sys.argv[1]
output_lora = sys.argv[2]

src = input_lora
dst = output_lora

lora = load_file(src)

out = {}

for k, v in lora.items():

    new = k

    if (
        k.startswith("blocks.")
        or k.startswith("token_refiner.")
        or k.startswith("final_layer.")
    ):
        new = "diffusion_model." + k

    out[new] = v

    print(new)

save_file(out, dst)

print("done")

Thank you very much for sharing this LoRA with the community. Your work is greatly appreciated. I had great results with it, and I really appreciate the effort you put into making this available for everyone.

sampler: euler
sheduler: beta
8 steps and lora strength 1.8 res: 1280x720 + RTX Super resolution.
8/8 [06:16<00:00, 47.03s/it] On a RTX 4070ti super 16 gb vram and 32gb ram.

Don't work with the pruned model.

If you're using the portable version of python, place this in the python_embeded folder along with the lora from this repo and run:

python.exe convert_h3_lora_for_comfyui.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

It works with the bf16 and int8 (non-pruned) versions only β€” pruned versions are also supported just ignore the errors about adaln_proj.linear.weight shape

import sys
from safetensors.torch import load_file, save_file

def convert(in_path: str, out_path: str, prefix: str = "diffusion_model."):
    sd = load_file(in_path)

    new_sd = {}
    skipped = 0
    for k, v in sd.items():
        if k.startswith(prefix):
            new_key = k  # already prefixed, leave as-is
            skipped += 1
        else:
            new_key = prefix + k
        new_sd[new_key] = v

    save_file(new_sd, out_path)

    print(f"Converted {len(sd)} keys ({skipped} already had the prefix).")
    print("Sample before -> after:")
    for i, k in enumerate(list(sd.keys())[:3]):
        print(f"  {k}  ->  {prefix + k if not k.startswith(prefix) else k}")
    print(f"\nSaved: {out_path}")

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(1)
    convert(sys.argv[1], sys.argv[2])

im using the nnormal version of comfyui - where do i place this script and what t do i call it - what node in comfyui do i use to load the lora?

Sign up or log in to comment