【 MiniMaxH3TurboLoRA 】Do you need a Pruned LoRA ~

#11
by sunnyboxs - opened

I am using the author's dedicated Turbo LoRA loader, but when using Pruned_int8_convrot,
the console reports numerous errors, and no progress is displayed during sampling

image

image

MiniMaxH3TurboLoRA:
Could you please make the 4-step Turbo LoRA loader compatible with Pruned_int8_convrot models? Thanks!

https://github.com/Larryvrh/ComfyUI-MiniMax-H3-Turbo

image

same issue here

Node updated for pruned base.

Node updated for pruned base.

I pulled master, but still getting errors on run with pruned base.

[ERROR] ERROR lora diffusion_model.blocks.47.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.48.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.final_layer.adaln_proj.linear.weight shape '[10752, 8]' is invalid for input of size 28901376
[ERROR] ERROR lora diffusion_model.blocks.0.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.1.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.2.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.3.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384

Node updated for pruned base.

I pulled master, but still getting errors on run with pruned base.

[ERROR] ERROR lora diffusion_model.blocks.47.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.48.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.final_layer.adaln_proj.linear.weight shape '[10752, 8]' is invalid for input of size 28901376
[ERROR] ERROR lora diffusion_model.blocks.0.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.1.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.2.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384
[ERROR] ERROR lora diffusion_model.blocks.3.adaln_proj.linear.weight shape '[96768, 8]' is invalid for input of size 260112384

Can you give me the full name of your base? I will check what's going on here.

minimax_h3_fl2va_pruned_int8_convrot.safetensors
sha256sum: e889202c41dafb67b10d67b97f0d8541508036a6090af23425a5c2615d03c47a

To be clear: It does work with this pruned base and your ckpt500, it just spams the console with these errors. The resulting video at 8 steps looks really good, 4 steps looks good but has some strange actions.

minimax_h3_ref2va_pruned_int8_convrot.safetensors

image

image

After using MiniMaxH3TurboSampler, the sampling progress display in the console disappeared; I hope this can be fixed.

Normal workflow, int8convrot pruned base, 20 steps - 230 seconds

w/ minimax_h3_turbo_4step_ckpt500, int8convrot pruned base, 8 steps - 116s

Still getting lora diffusio_model_blocks errors, but seems to work regardless.

Hey larryvrh, fantastic work on this node!

Just wanted to share a small fix in case anyone else runs into a RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (happened to me when running with VRAM offloading flags).

In comfyui-minimax-h3-turbo/init.py around line 145, input tensor x was on cuda:0 while a and b were sitting on cpu.

Changing line 145 to explicitly match x.device:
code Python

a_gpu = a.to(x.device, x.dtype)
b_gpu = b.to(x.device, x.dtype)
st_gpu = st.to(x.device, x.dtype)
x = x + (b_gpu @ (a_gpu @ st_gpu.T)).T

Fixed the device mismatch crash completely for my setup on the pruned base! Thanks again for putting this together!

Normal workflow, int8convrot pruned base, 20 steps - 230 seconds

w/ minimax_h3_turbo_4step_ckpt500, int8convrot pruned base, 8 steps - 116s

Still getting lora diffusio_model_blocks errors, but seems to work regardless.

I can confirm I get the same invalid shape error printed onto my console running the pruned int8_convrot model but the LoRA works despite that! Thanks for sharing!

minimax_h3_ref2va_pruned_int8_convrot.safetensors

image

image

After using MiniMaxH3TurboSampler, the sampling progress display in the console disappeared; I hope this can be fixed.

I see same problem, lost progress bar for it/s. it is Minimax-H3 Turbo Sampler (4 steps) making bar disapear.

minimax_h3_ref2va_pruned_int8_convrot.safetensors

image

image

After using MiniMaxH3TurboSampler, the sampling progress display in the console disappeared; I hope this can be fixed.

can confirm, same problem.

minimax_h3_ref2va_pruned_int8_convrot.safetensors

image

image

After using MiniMaxH3TurboSampler, the sampling progress display in the console disappeared; I hope this can be fixed.

can confirm, same problem.

should be fixed in the new version node, try upgrade

it's fixed now, thanks

Sign up or log in to comment