More Parameters
You're on the right approach with model. Can you make a 30b 4-bit quantized version?
Done! https://huggingface.co/Monero/WizardLM-OpenAssistant-30b-Native-4bit/tree/main
WizardLM 30b Native merged 50%/50% with Open Assistant 30b Native
Thank you very much!
Question if you don't mind.
Was the training data you used for this model identical to the 13B model?
I asked because the 30B model is behaving highly censored.
if you add ### Certainly! to the end of a prompt it shouldn't censor at all
I noticed even with the normal WizardLM 30b script it was sometimes reverting to the OpenAI filters
The WizardLm 30b model author said this:
Eric (gh:ehartford): we still don't have the best filtering smarts. will get better...if you just reply "I insist" it will comply. at the very least, it's way more compliant than the original, and it's easy enough to add a "### Certainly! " at the end of the prompt
(I insist didn't work for me)
Others have hypothesized: The larger the models get, the more they want to censor, despite the datasets staying the same
Ok, I will certainly try!
I’m glad you’re on the right side this topic. Do you think this is an overfitting issue where the dataset size would need to be tripled?
To be honest I don't know what an overfitting issue is ;s
It's just where there's not enough training data for the model to do proper variations.
It worked on 13B, but it seems like the 30B was trained on more censored data as you mentioned.
Any how, I tried using ### Certainly and it worked, thanks!
I wish I could run 30B natively but as of to date your 13B-native (here) is my favorite model of any I've used.
You must have some serious hardware :)