[Closed] bug
#1
by roolaml - opened
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Can you give me some test case about this bug ?
No tokenizer.json to run this on text-embeddings-inference. Please fix.
@roolaml
This warning appears to be issued by a Hugging Face transformer, but it does not affect the model, so don't worry about it. You can suppress it by following https://github.com/huggingface/transformers/issues/14285.
@huyhandes
I just added tokenizer.json. You can now run this on text-embeddings-inference.
itdainb changed discussion title from bug to [Closed] bug
itdainb changed discussion status to closed