apingali Claude Opus 4.7 (1M context) commited on
Commit
8f09671
·
1 Parent(s): 66ce4cf

fix(hf-space): use provider="auto" for InferenceClient (new routing layer)

Browse files

User confirmed:
- All HF Inference Providers enabled (incl. Featherless)
- Access granted to gated google/gemma-2-9b-it
- Featherless AI shown as the provider on the model's playground

Yet the Space call still returns "model_not_supported". Root cause:
the legacy `InferenceClient(model=...)` constructor without provider=
defaults to hf-inference-only routing, which doesn't serve most newer
models even when the user has access through partners.

Modern pattern (HF Inference Providers, intro'd late 2024) requires
`provider="auto"` to opt into automatic partner selection. With this,
the client picks Featherless for Gemma 2 9B because that's the live
provider for the model on the user's account.

Pure config change — no schema/contract changes — 31 tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +13 -1
app.py CHANGED
@@ -302,7 +302,19 @@ def _call_huggingface(system_block: str, user_prompt: str) -> str:
302
  "https://huggingface.co/settings/tokens). Then restart the Space. "
303
  "Until then, pick a different model from the dropdown."
304
  )
305
- client = InferenceClient(model=HF_MODEL_ID, token=token, timeout=120)
 
 
 
 
 
 
 
 
 
 
 
 
306
  try:
307
  resp = client.chat_completion(
308
  messages=[
 
302
  "https://huggingface.co/settings/tokens). Then restart the Space. "
303
  "Until then, pick a different model from the dropdown."
304
  )
305
+ # `provider="auto"` opts into the modern HF Inference Providers
306
+ # routing layer (introduced 2024-Q4), which picks the right partner
307
+ # (featherless-ai / together-ai / hf-inference / etc.) for the model
308
+ # automatically. Without this flag, InferenceClient falls back to
309
+ # the legacy hf-inference-only path, which doesn't serve most newer
310
+ # models and returns a misleading "model not supported" error even
311
+ # when the user has all providers enabled and access to the model.
312
+ client = InferenceClient(
313
+ model=HF_MODEL_ID,
314
+ token=token,
315
+ provider="auto",
316
+ timeout=120,
317
+ )
318
  try:
319
  resp = client.chat_completion(
320
  messages=[