How to use the increased context with llama.cpp?
#1
by
avion23
- opened
Is it enough to just increase the context? like this?./main --color -c 16384 --temp 0.2 --repeat_penalty 1.15 -n -1 -i -ins --multiline-input -t 8 -ngl 128 -m ../models/longchat-13b-16k.ggmlv3.q6_K.bin
It's not supported with llama.cpp at this time - please see the README for details of how to use with KoboldCpp, the only client that currently supports it
Oh thanks.
Btw I really appreciate the work you are doing by uploading all these LLM
avion23
changed discussion status to
closed