https://huggingface.co/inceptionai/Jais-2-70B-Chat
latest of one of the high scoring LLMs in arabic/english leaderboards.
please quantize with imatrix (IQ3 will be greatly appreciated!)
The Jais2ForCausalLM architecture is unfortunately not currently supported by llama.cpp nor do I see any users requestion support for it to be implemented or any contributors working on implementing this architecture. Because it this it is unfortunately not possible for use to provide GGUFs for this model.
Please follow our previous discussion about this model under https://huggingface.co/mradermacher/model_requests/discussions/1617 and let me know should llama.cpp support for this model ever be implemented.
I personally loved the Jais 1 family of models and would love to try Jais 2. I will try Jais-2-70B-Chat using vLLM I guess but having to use a custom vLLM fork for that is crazy. I really don't get why they don't upstream support for their model into popular inference engines. Why do they spend a ton of resources to create amazing models just to then not spend the relatively low effort of making regular users able to easily run them.
we hope the model architecture will be supported at some stage.
many thanks, Nicoboss, for trying it.