mradermacher

/

model_requests

like 181

It's queued, but not sure if it will quant as failed recently

You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#SIQ-1-35B-GGUF for quants to appear.

I assume quanting has failed again. Do you have any idea why @AlexWortega ?

llama_model_load: error loading model: missing tensor 'blk.40.attn_norm.weight'

This may have been be fixed.

There's a project for grafting in MTP layers from the base model for GGUF encapsulated models. They seem to suggest a similar approach in their fix. It requires additional work with each GGUF quant. Instead, they can try re-exporting the model while ensuring that MTP weights are loaded. A complete safetensors model should help facilitate mradermacher team's strival for producing GGUF quants.

That's being said, I have conjured a small script for grafting MTP layers directly into the safetensors instead, which I used for the MTP restored models including the experimental https://huggingface.co/MuXodious/MiniCPM-V-4.6-Thinking-heretic-MTP. ⛱️

1. maybe because of mtp
2. maybe because of I disabled Vl mldule