Querying Bloom from hugginterface inference api
Hello, I have used the inference API before to query some models, but where can I find which are the right parameters that I can use for this model in particular?
Thank you!
See this Repo : (https://github.com/Sentdex/BLOOM_Examples/blob/main/BLOOM_api_example.ipynb)
I hope that can help.
Strangely, setting return_full_text = False seems to still return the full text in my testing.
same here! I just wnat the response and I'm not sure how can I exclude the prompt from the return.
Good morning
has anyone managed to integrate the "Integromat" software with Bloom's API?
Thanks in advance
same here! I just wnat the response and I'm not sure how can I exclude the prompt from the return.
I'm guessing you can run something like
prompt = "blablabla"
prompt_length = len(prompt)
resp = infer(prompt)
generation = resp["generated_text"][prompt_length:]
@dp79pn Please open another discussion as it seems unrelated to the current discussion.
Strangely, setting
return_full_text = Falseseems to still return the full text in my testing.
It could be related to another issue: https://huggingface.co/bigscience/bloom/discussions/153#6397907b71eb2455d898e0a4