Kijai

/

MiniMax-H3_comfy

like 178

Since the file size is larger than the one from lightx2v, may be something did got changed 🤔

The original is in diffusers format so:

• key renaming like usual
• they use split qkv, we use fused so had to fuse that, this inflates the size a bit which can probably be compensated for with another method, TBD
• they use peft so application is calculated differently, added alpha 16 to match it, but this is mostly a guess since I don't know how it's intended to be applied exactly at this point

Thanks for detail, going to test 🤤

I see no adaln layers, is this meant for pruned only?

I see no adaln layers, is this meant for pruned only?

No, they just didn't train those, so it's neutral for both in that regard.

Hey, will this work on R2V ?

No, the audio in R2V is bad.

Added rank reduced version, retains about ~95% of the original while shrinking size to ~300MB, seems to work fine so far.

The LoRAs seem to be working great at 4 steps finally. The audio is completely destroyed. Have to bump to at least 8 steps to get the audio somewhat OK on the ears 😄

I have the minimax_h3_fl2va_pruned_int8_convrot.safetensors model and the sound is simply destroyed at step 4. At step 8 it's a little better, but still disgusting. Is this Lora not intended for the pruned version?

The LoRAs seem to be working great at 4 steps finally. The audio is completely destroyed. Have to bump to at least 8 steps to get the audio somewhat OK on the ears 😄

Make sure your ComfyUI is up to date, the low step audio issues were only solved yesterday in this commit: https://github.com/Comfy-Org/ComfyUI/commit/bdcb886a4705a03cf40f4a7226de9fc7c059fc90

Hi kijai, will Ref release an accelerated LoRA?

Sorry asking you here but with this lora is it better to use "MiniMax H3 Mem Eff Sage Attention Patch" or "Patch Sage Attention KJ" ?
"Patch Sage Attention KJ" is a bit faster.

The LoRAs seem to be working great at 4 steps finally. The audio is completely destroyed. Have to bump to at least 8 steps to get the audio somewhat OK on the ears 😄

Make sure your ComfyUI is up to date, the low step audio issues were only solved yesterday in this commit: https://github.com/Comfy-Org/ComfyUI/commit/bdcb886a4705a03cf40f4a7226de9fc7c059fc90

Update to nightly ? because i already update to ComfyUI 0.30.2

The LoRAs seem to be working great at 4 steps finally. The audio is completely destroyed. Have to bump to at least 8 steps to get the audio somewhat OK on the ears 😄

Make sure your ComfyUI is up to date, the low step audio issues were only solved yesterday in this commit: https://github.com/Comfy-Org/ComfyUI/commit/bdcb886a4705a03cf40f4a7226de9fc7c059fc90

Updating fixed the issue. Thanks!

Sorry asking you here but with this lora is it better to use "MiniMax H3 Mem Eff Sage Attention Patch" or "Patch Sage Attention KJ" ?
"Patch Sage Attention KJ" is a bit faster.

You're meant to use both, other activates sage and other modifies the model forward to reduce peak VRAM usage. It's purpose is only memory savings, which only translates to more speed when memory starved/lower end systems.

Further if you want to reduce peak VRAM, there are these new experimental nodes to slice the attention and feed forward layers:

This is far larger saving than the mem eff sage, it probably is pointless when using the low VRAM attention node.

The PR for these nodes are merged into master? I haven't seen them, will update comfy when I get home

Dear Kijai, please tell me, in order for Turbo Lora to produce sound and video normally on my "model minimax_h3_fl2va_pruned_int8_convrot.safetensors", I need to update Comfy UI to the stable 30.2 (I already did this and it does not help) or to the nightly version?

Sorry asking you here but with this lora is it better to use "MiniMax H3 Mem Eff Sage Attention Patch" or "Patch Sage Attention KJ" ?
"Patch Sage Attention KJ" is a bit faster.

You're meant to use both, other activates sage and other modifies the model forward to reduce peak VRAM usage. It's purpose is only memory savings, which only translates to more speed when memory starved/lower end systems.

Further if you want to reduce peak VRAM, there are these new experimental nodes to slice the attention and feed forward layers:

This is far larger saving than the mem eff sage, it probably is pointless when using the low VRAM attention node.

Thank you , i think mostly no one is using both, thats good to know.

Dear Kijai, please tell me, in order for Turbo Lora to produce sound and video normally on my "model minimax_h3_fl2va_pruned_int8_convrot.safetensors", I need to update Comfy UI to the stable 30.2 (I already did this and it does not help) or to the nightly version?

It is not in stable yet, just nightly, so you'd either have to update for that or wait for next stable, which should happen within next 24 hours.

Sorry asking you here but with this lora is it better to use "MiniMax H3 Mem Eff Sage Attention Patch" or "Patch Sage Attention KJ" ?
"Patch Sage Attention KJ" is a bit faster.

You're meant to use both, other activates sage and other modifies the model forward to reduce peak VRAM usage. It's purpose is only memory savings, which only translates to more speed when memory starved/lower end systems.

Further if you want to reduce peak VRAM, there are these new experimental nodes to slice the attention and feed forward layers:

This is far larger saving than the mem eff sage, it probably is pointless when using the low VRAM attention node.

Thank you , i think mostly no one is using both, thats good to know.

I made the same mistake.

@Kijai is there any chance of having a brief guide to use these new h3 nodes?

@Kijai is there any chance of having a brief guide to use these new h3 nodes?

Plug them in, memory use goes down in expense of some compute. It's very situational and hard to say when you should use them, besides the obvious hard out of memory errors at inputs that just couldn't fit your VRAM even with offloading.

The default settings cut peak VRAM from 18GB to 11GB for me at 1344x768 for 362 frames. More chunks can bring it lower but it's diminishing returns.

I feel like something is off with the scaling calculation, i can't put my finger on it, but hey i am not the goat so i could just be smoking my socks

I feel like something is off with the scaling calculation, i can't put my finger on it, but hey i am not the goat so i could just be smoking my socks

Their script (which was released after my conversion) uses alpha 8, so probably that. Which would just mean using these at 0.5 is the same as using alpha 8 version at 1.0.

I didn't really get good results at either 0.5 or 1.0, and found something in between to work the best, so I let it be.

so its rank 64 wtih 8 alpha?

Is this how we use it? Please let me know lol

never from the original lora layers looks like 128, so the scale then is not 0.125 its 0.0625, which would be the reason for saying 0.5 should be the same str