When I execute
output = client.text_generation(
"The capital of france is",
max_new_tokens=100,
)
print(output)
with model meta-llama/Llama-3.2-3B-Instruct, it throws following error.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[5], line 3
1 # As seen in the LLM section, if we just do decoding, **the model will only stop when it predicts an EOS token**,
2 # and this does not happen here because this is a conversational (chat) model and we didn't apply the chat template it expects.
----> 3 output = client.text_generation(
4 "The capital of france is",
5 max_new_tokens=100,
6 )
8 print(output)
File c:\Python312\Lib\site-packages\huggingface_hub\inference\_client.py:2298, in InferenceClient.text_generation(self, prompt, details, stream, model, adapter_id, best_of, decoder_input_details, do_sample, frequency_penalty, gra…
