please add emb4 and packed
#1
by shivshankar - opened
Yes, it looks like they released updated weights for the three released models. I will make a new native q8 gguf from the packed weights and will release a bf16 version as well.
I'm not going to make a emb4 embedded gguf as it currently is not a supported format with audio.cpp. Plus the question is do we also package the student_w4 decoder for something like this? Again not something that is implemented by current S3 decoder in audio.cpp. Feel free to submit a patch and host it yourself.
I did some testing with a hybrid q4/q8 gguf and saw about 15% improvement on CPU (not using student_w4 decoder/etc).
I have merged a new q4, q8, and bf16 with audio.cpp. I will continue to test different formats to see what we can do with this.
dignome changed discussion status to closed