Hugging Face's logo

aeonium
/
tokenizer-64k

Transformers
Russian
Model card Files
xet
 Community
How to use from the
Use from the
Transformers library
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("aeonium/tokenizer-64k", dtype="auto")
Quick Links

Tokenizer for future Aeonium v2 models

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using aeonium/tokenizer-64k 1