MiniMax H3 Acc LoRas support?
Are these LoRas compatible with ComfyUI? https://huggingface.co/alibaba-pai/MiniMax-H3-Acc-LoRAs
It seems to be involving .py files too 🤔
kijai so fast
uploaded already
Simply loading the lora gives error:
[ERROR] !!! Exception during processing !!! The size of tensor a (96) must match the size of tensor b (3072) at non-singleton dimension 0
[ERROR] Traceback (most recent call last):
File "/home/x/comfy/ComfyUI/execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/home/x/comfy/ComfyUI/execution.py", line 306, in process_inputs
result = f(**inputs)
File "/home/x/comfy/ComfyUI/comfy_api/internal/init.py", line 149, in wrapped_func
return method(locked_class, **inputs)
File "/home/x/comfy/ComfyUI/comfy_api/latest/_io.py", line 1990, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy_extras/nodes_custom_sampler.py", line 1055, in execute
samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1335, in sample
output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, latent_shapes=latent_shapes)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 114, in execute
return self.wrappers[self.idx](self, *args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/custom_nodes/h3-optimizations/h3_optimizations/runtime/context.py", line 189, in wrapper
return executor(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 106, in call
return new_executor.execute(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 114, in execute
return self.wrappers[self.idx](self, *args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/custom_nodes/comfyui-kjnodes/nodes/preview_override_node.py", line 685, in call
return executor(noise, latent_image, sampler, sigmas, denoise_mask, new_callback, disable_pbar, seed, latent_shapes=latent_shapes)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 106, in call
return new_executor.execute(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1262, in outer_sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, latent_shapes=latent_shapes)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1237, in inner_sample
samples = executor.execute(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1005, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/k_diffusion/sampling.py", line 205, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 640, in call
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1208, in call
return self.outer_predict_noise(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1215, in outer_predict_noise
).execute(x, timestep, model_options, seed)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 1218, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 620, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 211, in calc_cond_batch
return _calc_cond_batch_outer(model, conds, x_in, timestep, model_options)
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 219, in _calc_cond_batch_outer
return executor.execute(model, conds, x_in, timestep, model_options)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/samplers.py", line 335, in calc_cond_batch
output = model.apply_model(input_x, timestep, **c).chunk(batch_chunks)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/model_base.py", line 205, in apply_model
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...<2 lines>...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.APPLY_MODEL, transformer_options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
).execute(x, t, c_concat, c_crossattn, control, transformer_options, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/model_base.py", line 249, in _apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds)
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/ldm/minimax/model.py", line 540, in forward
out = comfy.patcher_extension.WrapperExecutor.new_class_executor(
...<2 lines>...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.DIFFUSION_MODEL, transformer_options)
).execute(x, timestep, context, transformer_options, minimax_payload=minimax_payload,
denoise_mask=denoise_mask, audio_denoise_mask=audio_denoise_mask, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 114, in execute
return self.wrappers[self.idx](self, *args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/custom_nodes/h3-optimizations/h3_optimizations/runtime/context.py", line 207, in wrapper
return executor(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 106, in call
return new_executor.execute(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/patcher_extension.py", line 113, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/ldm/minimax/model.py", line 726, in _forward
v, a = self.final_layer(h, t_emb, video_seg, audio_seg)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
return forward_call(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/ldm/minimax/model.py", line 315, in forward
return self.video_out(mod(video_seg)), self.audio_out(mod(audio_seg))
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/miniconda3/envs/comfy-env/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in call_impl
return forward_call(*args, **kwargs)
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 1414, in forward
output = self.forward_comfy_cast_weights(
input,
...<2 lines>...
weight_only_quant=weight_only_quant,
)
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 1338, in forward_comfy_cast_weights
with CastBiasWeightContext(
~~~~~~~~~~~~~~~~~~~~~^
self,
^^^^^
...<3 lines>...
want_requant=want_requant,
^^^^^^^^^^^^^^^^^^^^^^^^^^
) as (weight, bias):
^
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 466, in init
self.state = (None, None) if self.slf is None else cast_bias_weight(*args, **kwargs)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 380, in cast_bias_weight
weight, bias = resolve_cast_module_with_vbar(s, dtype, device, bias_dtype, compute_dtype, want_requant)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 327, in resolve_cast_module_with_vbar
weight = post_cast(s, "weight", weight, dtype, prefetch["resident"], update_weight)
File "/home/x/comfy/ComfyUI/comfy/ops.py", line 311, in post_cast
x = lowvram_fn(x)
File "/home/x/comfy/ComfyUI/comfy/model_patcher.py", line 202, in call
return comfy.lora.calculate_weight(patches, weight, self.key, intermediate_dtype=weight.dtype)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/x/comfy/ComfyUI/comfy/lora.py", line 489, in calculate_weight
weight.copy(v[0])
~~~~~~~~~~~~^^^^^^
RuntimeError: The size of tensor a (96) must match the size of tensor b (3072) at non-singleton dimension 0
-----------------#123 [SamplerCustomAdvanced]: 30.60s - vram 6957960704b
didn't work for me red errors 😢
They work fine for me when using the above linked PR.
But I don't think that these are necessarily 'the best' turbo LoRas for H3.
I'm getting better results with minimax_h3_turbo_4step_ckpt600_ema_V4 , but it may be highly use case dependent.
what's the difference between these and this version?
https://huggingface.co/aptech0081/MiniMax-H3-Acc-LoRAs-ComfyUI
RuntimeError: The size of tensor a (96) must match the size of tensor b (3072) at non-singleton dimension 0
btw, how come pruned file size the same with non-pruned? 🤔 is there any benefit of using pruned acc lora?
What's the difference between pruned and no prune?
Very experimental at the moment: got error - The size of tensor a (32) must match the size of tensor b (1024) at non-singleton dimension 1
They work fine for me when using the above linked PR.
But I don't think that these are necessarily 'the best' turbo LoRas for H3.
I'm getting better results with minimax_h3_turbo_4step_ckpt600_ema_V4 , but it may be highly use case dependent.
could you please share with me the best settings for the lora you mentioned? it's working great but i'm still getting some audio issues with it
are these going to work with native comfyUI nodes?
How to used this Lora? It's always give ksampler error.
Still giving the same error even with the new version
I ran it, but in fast-motion, arms and legs got smeared very badly. Like unusable
I ran it, but in fast-motion, arms and legs got smeared very badly. Like unusable
Actually, it's “smeared”—in my opinion, that's a step in the right direction. I have a problem with “powdery” hair and hands.
Still giving the same error even with the new version
I still think we are waiting for Kijai's PR to be merged in Comfy for this
I tested the original “acc turbo lora” file and… I’m sticking with “minimax_h3_ref2v_lightx2v_turbo_4step_v0.1_resized_avg_rank_20_bf16.safetensors.” The “acc” file for the same seed produces anatomical circus.
I tested the original “acc turbo lora” file and… I’m sticking with “minimax_h3_ref2v_lightx2v_turbo_4step_v0.1_resized_avg_rank_20_bf16.safetensors.” The “acc” file for the same seed produces anatomical circus.
what settings do you use with it to get the best results? strength, scheduler, sampler, audio/video shifts?
@necroryona I typically use:
- a turbo lora at .9 - 1.1 strength
- 6 or 8 steps
- shifts: video 12, audio 3
- res_multistep sampler, simple or beta scheduler
but I'm totally not claiming that I know for a fact that those are the best settings - it's what I settled on after trial and error.
The PR landed on master and I quickly tested 3 turbo loras (redownloaded Kijai's - was updated).
Same seed, prompt, settings (res_multistep, simple, 8 steps, 12.0/3.0 shifts).
Model: minimax_h3_ref2va_pruned_int8_convrot.
ACC:
minimax_h3_ref2v_lightx2v_turbo_4step_v0.1_resized_avg_rank_20_bf16:
larryvrh_minimax_h3_turbo_v4_step600_ema:
They may all have their pros and cons, but the 'visual reasoning' appears to be poorer with the ACC lora. Fidelity is high though.
For reference, this is 21 steps without any distil. lora:
I tested the original “acc turbo lora” file and… I’m sticking with “minimax_h3_ref2v_lightx2v_turbo_4step_v0.1_resized_avg_rank_20_bf16.safetensors.” The “acc” file for the same seed produces anatomical circus.
what settings do you use with it to get the best results? strength, scheduler, sampler, audio/video shifts?
My settings:
Model: Minimax_H3_ref2va_int8_convrot.safetensors (unpruned) + Spectrum + SageAttn (32GB)
Lora: minimax_h3_ref2v_lightx2v_turbo_4step_v0.1_resized_avg_rank_20_bf16.safetensors (stength: 1.0)
Sampler: MiniMax-H3 Turbo Sampler (4-step)
Sigmas: 15 steps - Euler
No video/audio shift.
General set: Pytorch 2.13+Cu132, ComfyUI + Nodes => up to date, RAM: 192, RTX5090
is it normal to have these in the logs when i use the ACC lora?
[INFO] Module diffusion_model.final_layer.video_out has resizing Lora - force loading
[INFO] Module diffusion_model.final_layer.audio_out has resizing Lora - force loading
I feel like PDD LorAs have a huge ghosting effect; it's very clear in fast motion. It doesn't need to be very fast either
Examples:
PDD:
Light2x 768p v1 8steps
Larry v4 600 pruned
I mean fast action video test is a bit extreme for any turbo Lora but in PDD lora case, arms & legs just melt 😆
I don't know if I configured it wrong, but there is only 1 config for PDD lora to run iirc