https://huggingface.co/LLM360/K2-V2-Instruct
https://huggingface.co/LLM360/K2-V2-Instruct
I think this is a fairly interesting new 70B-model.
Well, it is "LlamaForCausalLM" architecture, buuut (to quote its authors)...
K2-V2 is our most capable fully open model to date, and one of the strongest open-weight models in its class. It uses a 70B-parameter dense transformer architecture and represents the latest advancement in the LLM360 model family.
...so, I think it would be very nice if you'll quantize it to GGUFs, if that is alright.
Thank you!
We already tried this model on 8th December and it complained that the BPE pre-tokenizer was not recognized. I see that https://huggingface.co/cturan/K2-V2-Instruct-GGUF is just using gpt2 tokenizer with llama-bpe pre-tokenizer. LlamaForCausalLM is often used to implement completely from scratch trained unique models so there is a possibility that if we just use some existing pre-tokenizer our quants might not meet the quality standards of the original model. I will likely manually queue this tomorrow after doing some quant quality testing.
Understood. Thank you.