This is not for Comfyui?
I downloaded it but I'm getting:
[WARNING] lora key not loaded: blocks.0.adaln_proj.linear.lora_A.weight
[WARNING] lora key not loaded: blocks.0.adaln_proj.linear.lora_B.weight
[WARNING] lora key not loaded: blocks.0.attn.out_proj.lora_A.weight
[WARNING] lora key not loaded: blocks.0.attn.out_proj.lora_B.weight
And if not then is there any way to convert it for comfyui?
Nah, this is just an experimental demo, not comyfui compatible yet. Will add comfyui support once the full training finished.
I see, thank you so much.
it works in comfy despite the warning.
Really? Let me check because I did A/B testing and I saw no difference at all.
it works in comfy despite the warning.
Really? Let me check because I did A/B testing and I saw no difference at all.
i guess it doesn't work... just tested it without and I get the same result with the same number of steps (6) and the quality is great. I just never tried to do it with less then 20 steps before. I'm cooked
i guess it doesn't work... just tested it without and I get the same result with the same number of steps (6) and the quality is great. I just never tried to do it with less then 20 steps before. I'm cooked
LOL, it's fine.
It worked well for me after I used this script.
import sys
from safetensors.torch import load_file, save_file
input_lora = sys.argv[1]
output_lora = sys.argv[2]
src = input_lora
dst = output_lora
lora = load_file(src)
out = {}
for k, v in lora.items():
new = k
if (
k.startswith("blocks.")
or k.startswith("token_refiner.")
or k.startswith("final_layer.")
):
new = "diffusion_model." + k
out[new] = v
print(new)
save_file(out, dst)
print("done")
Thank you very much for sharing this LoRA with the community. Your work is greatly appreciated. I had great results with it, and I really appreciate the effort you put into making this available for everyone.
script or not; it somehow increased the time it takes to do 8 steps vs 12 steps without it....
At 4 steps, the image looks quite good, but the audio isn't very interesting.
At 8 steps, the overall result seems quite good for both the image and the audio. However, I think it slightly reduces the amount of motion, although I'm not 100% sure. I'll need to do more testing on my end.
It's also much faster. Normally, 12 steps take me around 14–15 minutes at 1280×720.
With the LoRA, 12 steps take only 8 minutes 26 seconds.
At 8 steps, I get roughly the same quality as 12 steps without the LoRA, but it only takes about 6 minutes.
However, it doesn't work with the pruned model.
I've been using a LoRA strength between 1.6 and 2.0.