mradermacher

/

model_requests

like 132

We already tied this one in the past:

Wed 07 May 2025 02:49:23 CEST leia    modello-italia-9b 32G     ./modello-italia-9b
https://huggingface.co/sapienzanlp/modello-italia-9b already in hfd.log

I found the root cause why it failed in the past:

modello-italia-9b       WARNING:hf-to-gguf:**************************************************************************************
modello-italia-9b       WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized!
modello-italia-9b       WARNING:hf-to-gguf:**          There are 2 possible reasons for this:
modello-italia-9b       WARNING:hf-to-gguf:**          - the model has not been added to convert_hf_to_gguf_update.py yet
modello-italia-9b       WARNING:hf-to-gguf:**          - the pre-tokenization config has changed upstream
modello-italia-9b       WARNING:hf-to-gguf:**          Check your model files and convert_hf_to_gguf_update.py and update them accordingly.
modello-italia-9b       WARNING:hf-to-gguf:** ref:     https://github.com/ggml-org/llama.cpp/pull/6920
modello-italia-9b       WARNING:hf-to-gguf:**
modello-italia-9b       WARNING:hf-to-gguf:** chkhsh:  6e9745994e2aacf364ad9ecbac67a6dceafd25c832a029aa36d26af26783498c
modello-italia-9b       WARNING:hf-to-gguf:**************************************************************************************

If there is a pre-tokenizer supported by llama.cpp that is compatible with this model we could use it instead but especially for non-English models there are sometimes good reasons to swap the tokenizer so I'm not sure if this is something we should do.