Spaces:
Running on Zero
fix(hf-space): use provider="auto" for InferenceClient (new routing layer)
Browse filesUser confirmed:
- All HF Inference Providers enabled (incl. Featherless)
- Access granted to gated google/gemma-2-9b-it
- Featherless AI shown as the provider on the model's playground
Yet the Space call still returns "model_not_supported". Root cause:
the legacy `InferenceClient(model=...)` constructor without provider=
defaults to hf-inference-only routing, which doesn't serve most newer
models even when the user has access through partners.
Modern pattern (HF Inference Providers, intro'd late 2024) requires
`provider="auto"` to opt into automatic partner selection. With this,
the client picks Featherless for Gemma 2 9B because that's the live
provider for the model on the user's account.
Pure config change — no schema/contract changes — 31 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@@ -302,7 +302,19 @@ def _call_huggingface(system_block: str, user_prompt: str) -> str:
|
|
| 302 |
"https://huggingface.co/settings/tokens). Then restart the Space. "
|
| 303 |
"Until then, pick a different model from the dropdown."
|
| 304 |
)
|
| 305 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 306 |
try:
|
| 307 |
resp = client.chat_completion(
|
| 308 |
messages=[
|
|
|
|
| 302 |
"https://huggingface.co/settings/tokens). Then restart the Space. "
|
| 303 |
"Until then, pick a different model from the dropdown."
|
| 304 |
)
|
| 305 |
+
# `provider="auto"` opts into the modern HF Inference Providers
|
| 306 |
+
# routing layer (introduced 2024-Q4), which picks the right partner
|
| 307 |
+
# (featherless-ai / together-ai / hf-inference / etc.) for the model
|
| 308 |
+
# automatically. Without this flag, InferenceClient falls back to
|
| 309 |
+
# the legacy hf-inference-only path, which doesn't serve most newer
|
| 310 |
+
# models and returns a misleading "model not supported" error even
|
| 311 |
+
# when the user has all providers enabled and access to the model.
|
| 312 |
+
client = InferenceClient(
|
| 313 |
+
model=HF_MODEL_ID,
|
| 314 |
+
token=token,
|
| 315 |
+
provider="auto",
|
| 316 |
+
timeout=120,
|
| 317 |
+
)
|
| 318 |
try:
|
| 319 |
resp = client.chat_completion(
|
| 320 |
messages=[
|