Hermes Bot commited on
Commit
f438be9
·
unverified ·
1 Parent(s): 46f8d38

fix: route chat completions to free serverless Inference API

Browse files

By default, recent huggingface_hub versions route InferenceClient.chat_completion()
to router.huggingface.co, which requires paid providers. Explicitly passing
base_url='https://api-inference.huggingface.co' forces it to use the free serverless
inference endpoint.

Files changed (1) hide show
  1. shared/inference_client.py +1 -0
shared/inference_client.py CHANGED
@@ -106,6 +106,7 @@ def _get_client():
106
  return InferenceClient(
107
  model=INFERENCE_MODEL,
108
  token=HF_TOKEN,
 
109
  )
110
  def generate(
111
  project: str,
 
106
  return InferenceClient(
107
  model=INFERENCE_MODEL,
108
  token=HF_TOKEN,
109
+ base_url="https://api-inference.huggingface.co",
110
  )
111
  def generate(
112
  project: str,