Post
23
Introducing Bonsai 2 27b GSQ RCO! It applies two newly-popular methods for quants to retain higher accuracy. Bonsai 2 27b GSQ RCO achieves around 6 percent lower perplexity on WikiText-2 compared to Bonsai 2 27b and roughly unchanged benchmark accuracy overall, with small mixed differences. It stays under 7gb, staying small like the original bonsai. Note that this is more of an experiment than a true finished product but the gains we saw are cool! Test it out and let me know what you think!
ProCreations/bonsai-2-27b-gsq-rco-gguf
ProCreations/bonsai-2-27b-gsq-rco-gguf