How to use Xenova/Kimi-K3-tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Xenova/Kimi-K3-tokenizer", device_map="auto")
Standalone tokenizer files for Kimi K3. No custom tokenizer code or trust_remote_code=True is required.
trust_remote_code=True
from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("Xenova/Kimi-K3-tokenizer") inputs = tokenizer("Hello from Kimi K3!")
-
Base model
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Xenova/Kimi-K3-tokenizer", device_map="auto")