Add tokenizer.json for compatibility with text-embeddings-inference

#11

Text embeddings inference requires a tokenizer.json.

sbhave1996 changed pull request status to closed

Sign up or log in to comment