How to run properly on KoboldCPP?
#8
by
Varkoyote
- opened
Hello! How to properly run this model on KoboldCPP please? For me it's completely broken, no stop sequences, just sending gibberish no matter the settings, it's very bad... thanks!
It is not supported yet in KoboldCPP, it will likely be in the next release. However, llama.cpp has some PRs currently open to fix bugs in the implementation, so I think it is not even working correctly in llama.cpp at the moment (I wonder if this is the source of the repetition problems?).
Anyway, keep an eye on KoboldCPP releases in the coming days.
The frequency of KoboldCpp updates isn't what it used to be. I lost a bet with myself that the support would be added in a day tops. 😭💔