wen GGUF
#9
by sukkritsharma - opened
I wanted to use this using llama.cpp since my current stack uses nomic-embed-text-v1.5.Q8_0.gguf, wanted to know when are we going to get the GGUF variants
I imagine this will require changes to llama.cpp/gpt4all before we can get GGUFs, I would suggest creating an issue there
zpn changed discussion status to closed
I would make the issue in the llama.cpp main repo: https://github.com/ggerganov/llama.cpp. Support for all modern bert based model will likely be desired (not just embed)