How to use naclbit/trin_tokenizer_v3 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("naclbit/trin_tokenizer_v3", dtype="auto")
