Transformers How to use BobJames/open-higgs-tokenizer with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoFeatureExtractor, AutoModel
extractor = AutoFeatureExtractor.from_pretrained("BobJames/open-higgs-tokenizer")
model = AutoModel.from_pretrained("BobJames/open-higgs-tokenizer", device_map="auto") 
