KoboldCpp Parameters/Settings
Hi, first of all I love this model, so thank you!!!
Now, I see you have parameters posted in your model card and you have instructions in the files and versions section as well, but it seems like most of that is for Silly Tavern. I'm only using KoboldCpp. Do you know which settings I should use specifically? I see you have this in the model card section "I recommend running Temperature 1.0-1.25 with 0.1 Top A or 0.01-0.1 Min P, and with 0.8/1.75/2/0 DRY. Also works with lower Temperatures below 1.0. Nothing more needed." I posted a screenshot of my view in KoboldCpp. The model works well, but I'm wondering if it would work even better with the settings more fine tuned. I'm unsure if I only adjust the settings you mentioned and leave everything else as default if that would be ideal, I don't know if the default settings are the same in Kobold as they are in ST and that is what I'm afraid of. Sorry for the long winded post and hopefully I don't sound too dumb! Lol. Thank you in advance.
I apologize in advance if this isn't the proper way to ask for additional information. I don't intend to highjack OP's topic, but since it appears very close to the question I wanted to ask, I suppose it'd be sensible to add it here :
I'm also using Kobold CPP and Kobold Lite as an UI with the intent to write stories using "Story Mode". Will this model perform adequately with "Story Mode", or do I need to somehow include the prompt formatting mentioned ? ([INST]{system}[/INST]{response}[INST]{user's message}[/INST]{response})
Or can I simply use the model in "Story Mode" without worrying about it ? The model seems to understand what it's supposed to do with plain text instructions within "Memory" and "Author's Notes", but I'm curious.
If it's not clear from my line of questioning, I'm quite new and bad at running LLMs locally. Thanks in advance for your understanding.
Hi, first of all I love this model, so thank you!!!
Now, I see you have parameters posted in your model card and you have instructions in the files and versions section as well, but it seems like most of that is for Silly Tavern. I'm only using KoboldCpp. Do you know which settings I should use specifically? I see you have this in the model card section "I recommend running Temperature 1.0-1.25 with 0.1 Top A or 0.01-0.1 Min P, and with 0.8/1.75/2/0 DRY. Also works with lower Temperatures below 1.0. Nothing more needed." I posted a screenshot of my view in KoboldCpp. The model works well, but I'm wondering if it would work even better with the settings more fine tuned. I'm unsure if I only adjust the settings you mentioned and leave everything else as default if that would be ideal, I don't know if the default settings are the same in Kobold as they are in ST and that is what I'm afraid of. Sorry for the long winded post and hopefully I don't sound too dumb! Lol. Thank you in advance.
Use the recommended settings.
I apologize in advance if this isn't the proper way to ask for additional information. I don't intend to highjack OP's topic, but since it appears very close to the question I wanted to ask, I suppose it'd be sensible to add it here :
I'm also using Kobold CPP and Kobold Lite as an UI with the intent to write stories using "Story Mode". Will this model perform adequately with "Story Mode", or do I need to somehow include the prompt formatting mentioned ? ([INST]{system}[/INST]{response}[INST]{user's message}[/INST]{response})
Or can I simply use the model in "Story Mode" without worrying about it ? The model seems to understand what it's supposed to do with plain text instructions within "Memory" and "Author's Notes", but I'm curious.
If it's not clear from my line of questioning, I'm quite new and bad at running LLMs locally. Thanks in advance for your understanding.
I have no clue how Kobold's Story Mode works. But if it works, it works.