Error when using in KTransformers
#2
by
orca-zhang
- opened
Exception: can't find blk.0.attn_kv_b.weight in GGUF file!
blk.0.attn_kv_b.weight exists in previous UD-Q2_K_XL
that separates into k_b v_b in UD 2.0
It seems to be a compatibility issue with the framework implementation. I found the modification solution provided by the Contributor here.
https://github.com/kvcache-ai/ktransformers/issues/1195#issuecomment-2831958049
orca-zhang
changed discussion status to
closed