How to use sbordt/experiment_4_simple_tokenizer_eos with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("sbordt/experiment_4_simple_tokenizer_eos", dtype="auto")
d1d2e0d
1
{"model_type": "olmo2"}
