How to use ddh0/simple-tokenizer-5120 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ddh0/simple-tokenizer-5120", dtype="auto")
