Error after loading minimax_h3_fl2va_pruned_fp8_Q8_0
Hi!
I've tried loading minimax_h3_fl2va_pruned_fp8_Q8_0.gguf through your nodes, seeing that Q8 is supported. Unfortunately, I get an error on sampling, after the model is loaded:
ValueError: Input and weight inner dimensions must match, got 768 and 5120
I have run --upgrade ggufand updated requirements too without issues, but it didn't help. I use latest ComyUI, upgraded today. Should I only use Q8_CR quants with your nodes?
The problem is not with you - there is a bug I need to fix before. I expect to upload a fix (I think only the nodes, not the model conversions) EU timezone ~22:00. I was called away from my computer before I could finalize the nodes/models, I apologize for that.
Thank you! Waiting for the fix then.
Also, since you've only uploaded Q8 and Q4 versions, and seeing others leaving out Q5, I wonder, why? It seems like 5 bit quantization would be a middle option between 4 bit (which was provided by ComfyOrg from the start) and 8bit variants. Am I missing something?
I ran out of storage... :D I had to wait for these too upload overnight first, I'll do the rest tonight.
Hi @notafraud - I uploaded some workflows (I am working on adding more) to https://huggingface.co/molbal/MiniMax-H3-GGUF/tree/main/workflows and I cannot reproduce the 'ValueError: Input and weight inner dimensions must match, got 768 and 5120' error. Would you please..
- Make sure the nodes are updated to one of the commits marked with green.
If you installed from ComfyUI Registry (https://registry.comfy.org/publishers/molbal/nodes/comfyui-gguf-reboot) then it is called Version 26.0.0-08-04 there. If you follow git tags, then it is tagged v26-08-04 in Git.
- Would you share your workflow please with me so that I can try to reproduce it?
Thank you!