Re-run quants

#3066
by xtools-at - opened

Following up on the drama around mradermacher/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-i1-GGUF, @RichardErkhov was involved in the discussions too.

Due to an issue in my heretic pipeline, MTP tensors have been dropped in any of my abliterated models, but the config.json still declared them. Models crash on load with certain common setups (old llama.cpp versions, LMStudio and Windows came up), while loading fine in others (which is why I didn't catch on earlier). The error propagates through any fine-tune made from those, and the quants.

I've restored all MTP tensors in the upstream repos, and quants made by community members since don't seem to have the issue as per the discussions.

@RichardErkhov brought up the question on whether or not to re-run the mradermacher quants - from recent feedback I'd suggest YES. If you have the capacity to do so, I'd appreciate if you could have a look at these repos and run the quants again:

primary:

secondary:

if you're still motivated:

I appreciate your help, and apologise for the hassle!

sure, but make sure that the original mtp works so that llamacpp can pick it up automatically . I will queue it in few minutes, but the problem is that I would have to nuke existing repos and the conversations will be gone. As soon as I get green flag from you I will requeue

and since Im not really following the conversations, let me know if I need to know anything

thank you, ready to go :) as far as i've learned you don't need to know/do anything special, llama.cpp should pick up the mtp tensors when they're there, and bundle them in the quant

they seem to be all there, thanks again and sorry for the inconvenience!

xtools-at changed discussion status to closed

Sign up or log in to comment