Broken
#1
by Aryanne - opened
open_llama_3b-q4_0-ggjt.bin seems to be broken, doesn't run on koboldcpp, it gives an error about dimensionality of some weight somewhere.
koboldcpp is based on llama.cpp which has hardcoded sizes for the different llama architectures. Meaning 3B isn't in there yet. You could contribute it to koboldcpp or use rustformers/llm which calculates model sizes dynamically.