Absurdly basic quantization

attn_q=q5_k attn_k=q5_k attn_v=q5_k attn_output=q5_k ffn_down_exps=q5_k ffn_up_exps=q5_k ffn_gate_exps=q5_k ffn_down_shexp=q5_k ffn_up_shexp=q5_k ffn_gate_shexp=q5_k attn_norm.weight=f32 attn_q_norm.weight=f32 attn_k_norm.weight=f32 post_attention_norm.weight=f32 post_ffw_norm.weight=f32 ffn_norm.weight=f32 exp_probs_b.bias=f32 ffn_gate_inp.weight=f32

wanted a GGUF early. also you all need a llama.cpp build above 11702 as I understand it to run Kolibri1, though I quantized and ran this model on build 11889.

tested exclusively on a vulkan build of llama-cpp

Downloads last month
69
GGUF
Model size
78B params
Architecture
kolibri1
Hardware compatibility
Log In to add your hardware

5-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for explodingyesyes/Kolibri1-Q5_K_Mixed-GGUF

Quantized
(28)
this model