Got empty prediction and predictions with whitespace postfix
Hi, I tried this model, sometimes it works perfect, but sometimes it gives empty predictions or predictions with multiple labels or predictions with a lot of white blank, for example:
{'generated_text': ' '}
{'generated_text': '1 '}
{'generated_text': '1 0'}
{'generated_text': '1 1 '}
One of my document is longer than the 512 limit, but even when it's not, I also get such predictions.
Did you solve this probleam? I use this model with TGI framework, the text-generation-inference framework. But I could just get empty generated text like {'generated_text': ' '}.
Did you solve this probleam? I use this model with TGI framework, the text-generation-inference framework. But I could just get empty generated text like {'generated_text': ' '}.
Now it seems to be known how to solve this problem. Just need to set the parameter "dtype" to bfloat16 when running the TGI container, because T5 model only support bfloat16.