Q & A
If you have any questions, about the quantization options for the ggufs or the imatrix I will answer them as soon as I can. On a first in first out basis.
Hi there. thanks for the open-source sharing.
I would like to ask what is this model have to do with Rust?
is the optimization only with regarding to imatrix calibration data?
In short the Imatrix was built with thousands of samples of Rust, python, math, and English to map the model weights in order to preserve as much of the Rust/python coding ability through quantization as much as possible. The largest sub-set of the sample data was Rust samples.
Thanks for the reply.
May I ask, if the calibration dataset would be open-source at some point?
or if not. are you using pure safe rust with no unsafe block?
Alright so the rust part of the data sample was pulled from the stack overflow rust data set. And importantly the Imatrix isn't a fine toon just a preservation mechanism. The data set used by Alibaba and Cerberus is gonna effect the output of the model way more then my Imatrix. I built a map using data samples to not crush important weights while quantizing to preserve the models rust ability as much as possible. Long story short the way Imatrix works as long as the sample activates the rust expert and weights the model had it gets mapped. You want unsafe and safe to trigger model weights to preserve them. Imatrix doesn't train it preserves. For this use case of imatrix you want safe,unsafe,incomplete,good and bad examples in the sample to activate weights for both fixing and generating code. I'm not giving out my exact sample set with this repo. This is a model repo not a data-set repo.