🚂 Mr.Chatterbox Chat 2.7B
It's very interesting to see an aristocratic model lol
It's queued!
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#Mr.Chatterbox-2.7B-Chat-GGUF for quants to appear.
is it even compatible with llamacpp? it's the usual bpe tokenizer error when either tokenizer is broken or model is weird
It is supposed to be a Llama architecture model. @tventurella do you have more information?
This post is sponsored by the Grunwald 1410 Foundation
cant think of any llama 2.7b
yeah, but is tokenizer stock ? I dont know the original model, it's not linked and I cant remember any 2.7b llamas
yeah, but is tokenizer stock ?
It may not be which is a shame!
This post is sponsored definitely not endorsed by the Official Monster Raving Loony Party
Hey everyone.
Nobody asked me anything.
There are some 3B parameter Llama (Llama-3.2 family) to my recollection, but not 2.7B specifically.
I think we've all missed this:
"Mr.Chatterbox-2.7B-Chat [...] a language model trained from scratch[...]"
Could it be there is no Chatterbox's base model because...its an original?
Sorry for chirpin' in
EDIT: as an added bonus here's what Deepseek had to say:
"The config you shared shows this is a custom-sized Llama-style model, not a standard Meta release. For example:
hidden_size: 2560
num_hidden_layers: 32
num_attention_heads: 20
vocab_size: 65,664 (which is 65,536 + 128 new tokens, as mentioned in the model card)
These dimensions don't match any official Llama model (like Llama 2 7B or Llama 3 8B). This confirms it's a from-scratch model built using the Llama architecture as a template."
This post was made possible by fried chicken
maybe maybe, that's what we are speculating about =)
This post was made possible by fried chicken
love it. can I have some chicken too ?
Hey everyone, thank you for getting the ball rolling on this! Yes, the model is custom. Base model is https://huggingface.co/tventurella/Mr.Chatterbox-2.7B
@redaihf @RichardErkhov , the tokenizer is trained from scratch - custom, byte-level, BPE tokenizer, the same tokenizer type at gpt-2. It should convert cleanly once llama.cpp knows the tokenizer is gpt.
No, even if the family of tokenizer is same llamacpp might not recognize it. Happens many times, dont think we can fix it really unless your model somehow gets integrated into llamacpp..
I am thinking along the same lines so to speak as @RichardErkhov . Either a more detailed set of configuration files or a new model definition is needed to allow the automated conversion to work correctly.
This post is sponsored by the Trabant Technical Library
I would never find such gold on the ru internet
"back in my days it was good" =(
OK @redaihf @RichardErkhov , made a GGUF version: https://huggingface.co/tventurella/Mr.Chatterbox-2.7B-Chat-GGUF
Congratulations =)