TheBloke

/

wizard-mega-13B-GGML

like 58

https://www.youtube.com/watch?v=6-zYkPo1M68

this is actually the GGML version (my bad) - you should be able to use it with LLama.cpp https://github.com/ggerganov/llama.cpp

@TheStamp with llama.cpp, as in I need both this model and llama.cpp locally? I’m trying to run it in code using langchain instead of openai. Sorry for the newbiness

Then you want llama-cpp-python, which are llama.cpp bindings for Python. Allows you to load GGML files exactly the same as llama.cpp does, but easily accessible from code.

It can then be used either direct from your own Python code, or via an OpenAI-compatible API which you can put LangChain at, or any other client.

https://github.com/abetlen/llama-cpp-python
https://pypi.org/project/llama-cpp-python/

you could use this: https://www.youtube.com/watch?v=aO5ZpFYHa0A&t=1s

Thanks everyone, goal accomplished