apingali Claude Opus 4.7 (1M context) commited on
Commit
6fc254e
·
1 Parent(s): 17c3a19

fix(hf-space): trust_remote_code for Phi-4-mini + verbose error reporting

Browse files

User reported live: ZeroGPU call surfaced "(Error)" with no actionable
detail. Two fixes:

1. Phi-4-mini-instruct has the `custom_code` tag on HuggingFace Hub —
it ships a custom Phi3-based modeling.py inside the repo. Loading
it via AutoModelForCausalLM.from_pretrained() without
`trust_remote_code=True` raises a generic Error from the
transformers Auto class. Added the flag to both AutoTokenizer and
AutoModel calls.

2. Bumped ZEROGPU_DURATION_SECONDS default from 120 to 300 (the
Pro-tier max) to give Phi-4-mini's first-call cold start (download
~8GB + load to GPU + first inference) breathing room. Subsequent
warm calls use a fraction of that.

3. Error messages now include str(e) (truncated to 400 chars) alongside
the class name. Format went from:
"failed (Error). Try again..."
to:
"failed.
**OSError:** Can't load tokenizer for 'foo/bar'. (...)
Try again..."
Future failures will be diagnosable from the UI without needing
Space log access.

Tests: 31/31 still pass (no contract changes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +25 -7
app.py CHANGED
@@ -189,7 +189,7 @@ ROOT = Path(__file__).parent
189
  ANTHROPIC_MODEL_ID = os.environ.get("MODEL_ID", "claude-opus-4-7")
190
  HF_MODEL_ID = os.environ.get("HF_MODEL_ID", "google/gemma-2-9b-it")
191
  ZEROGPU_MODEL_ID = os.environ.get("ZEROGPU_MODEL_ID", "microsoft/Phi-4-mini-instruct")
192
- ZEROGPU_DURATION_SECONDS = int(os.environ.get("ZEROGPU_DURATION_SECONDS", "120"))
193
  MAX_DESCRIPTION_WORDS = int(os.environ.get("MAX_DESCRIPTION_WORDS", "5000"))
194
  MIN_DESCRIPTION_WORDS = 200
195
 
@@ -307,14 +307,22 @@ _zerogpu_tokenizer = None
307
 
308
  def _load_zerogpu_model():
309
  """Load the model + tokenizer once. Called lazily on first request
310
- so module import stays fast (the model weights are tens of GB)."""
 
 
 
 
311
  global _zerogpu_model, _zerogpu_tokenizer
312
  if _zerogpu_model is not None:
313
  return
314
- _zerogpu_tokenizer = _AutoTokenizer.from_pretrained(ZEROGPU_MODEL_ID)
 
 
 
315
  _zerogpu_model = _AutoModelForCausalLM.from_pretrained(
316
  ZEROGPU_MODEL_ID,
317
  torch_dtype=_torch.bfloat16,
 
318
  device_map="auto",
319
  )
320
 
@@ -627,16 +635,26 @@ def diagnose(
627
  raw = _call_model(SYSTEM_BLOCK, user_prompt, provider)
628
  except Exception as e:
629
  # API timeout / rate limit / auth / server / network failure
630
- # (Anthropic SDK or huggingface_hub InferenceClient).
 
 
 
631
  model_label = {
632
  "anthropic": ANTHROPIC_MODEL_ID,
633
  "huggingface": HF_MODEL_ID,
634
  "zerogpu": ZEROGPU_MODEL_ID,
635
  }.get(provider, provider)
 
 
 
 
 
 
636
  return (
637
- f"⚠ The diagnostic call to {provider} ({model_label}) failed "
638
- f"({type(e).__name__}). Try again in a moment, switch providers in "
639
- f"the dropdown, or shorten your description.",
 
640
  "",
641
  )
642
 
 
189
  ANTHROPIC_MODEL_ID = os.environ.get("MODEL_ID", "claude-opus-4-7")
190
  HF_MODEL_ID = os.environ.get("HF_MODEL_ID", "google/gemma-2-9b-it")
191
  ZEROGPU_MODEL_ID = os.environ.get("ZEROGPU_MODEL_ID", "microsoft/Phi-4-mini-instruct")
192
+ ZEROGPU_DURATION_SECONDS = int(os.environ.get("ZEROGPU_DURATION_SECONDS", "300"))
193
  MAX_DESCRIPTION_WORDS = int(os.environ.get("MAX_DESCRIPTION_WORDS", "5000"))
194
  MIN_DESCRIPTION_WORDS = 200
195
 
 
307
 
308
  def _load_zerogpu_model():
309
  """Load the model + tokenizer once. Called lazily on first request
310
+ so module import stays fast (the model weights are tens of GB).
311
+
312
+ `trust_remote_code=True` is required for models tagged `custom_code`
313
+ on HuggingFace Hub (e.g. Phi-4-mini-instruct uses a custom Phi3-based
314
+ config that ships modeling files in the repo)."""
315
  global _zerogpu_model, _zerogpu_tokenizer
316
  if _zerogpu_model is not None:
317
  return
318
+ _zerogpu_tokenizer = _AutoTokenizer.from_pretrained(
319
+ ZEROGPU_MODEL_ID,
320
+ trust_remote_code=True,
321
+ )
322
  _zerogpu_model = _AutoModelForCausalLM.from_pretrained(
323
  ZEROGPU_MODEL_ID,
324
  torch_dtype=_torch.bfloat16,
325
+ trust_remote_code=True,
326
  device_map="auto",
327
  )
328
 
 
635
  raw = _call_model(SYSTEM_BLOCK, user_prompt, provider)
636
  except Exception as e:
637
  # API timeout / rate limit / auth / server / network failure
638
+ # (Anthropic SDK, huggingface_hub InferenceClient, or
639
+ # transformers/torch on the zerogpu path). Include both the
640
+ # exception class AND its string form so unexpected failures
641
+ # are diagnosable from the UI without server log access.
642
  model_label = {
643
  "anthropic": ANTHROPIC_MODEL_ID,
644
  "huggingface": HF_MODEL_ID,
645
  "zerogpu": ZEROGPU_MODEL_ID,
646
  }.get(provider, provider)
647
+ detail = str(e).strip() or "(no message)"
648
+ # Cap the detail so we don't spill multi-paragraph tracebacks
649
+ # into the UI. 400 chars is enough for a stack-trace summary
650
+ # without flooding the markdown tab.
651
+ if len(detail) > 400:
652
+ detail = detail[:400] + "…"
653
  return (
654
+ f"⚠ The diagnostic call to {provider} ({model_label}) failed.\n\n"
655
+ f"**{type(e).__name__}:** {detail}\n\n"
656
+ f"Try again in a moment, switch providers in the dropdown, "
657
+ f"or shorten your description.",
658
  "",
659
  )
660