How to use chifural/Mhondoro-Tokenizer-2k with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("chifural/Mhondoro-Tokenizer-2k", device_map="auto")
chifural commited on 