How to use chifural/Mhondoro-Tokenizer-base with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("chifural/Mhondoro-Tokenizer-base", dtype="auto")
chifural commited on 