BoxOfColors Claude Sonnet 5 commited on
Commit
bcb6e0a
Β·
1 Parent(s): 035533f

Force local_files_only on open_clip's own hf_hub_download reference

Browse files

Live repro crashed the previous fix's own debug line: AttributeError,
huggingface_hub.constants has no is_offline_mode() on the deployed version β€”
meaning the whole HF_HUB_OFFLINE/is_offline_mode() approach was built against
the wrong version's internals, explaining why neither prior attempt (78108d3,
035533f) actually stopped the live re-download despite being individually
verified against huggingface_hub 1.23.0 in isolation.

The traceback also showed the @spaces.GPU call running through
anyio.to_thread.run_sync β€” a worker thread in the same process, not the
separate process the code's own docstrings had suggested β€” so process
isolation wasn't the blocker either.

Rather than keep chasing which offline-mode attribute the actually-deployed
huggingface_hub version honors, sidestep it: open_clip.pretrained binds its
own local `hf_hub_download` name at import time
(`from huggingface_hub import hf_hub_download`), so monkeypatch that specific
module-level reference to force local_files_only=True on every call. That
parameter's handling (skip the network HEAD call, fall back to whatever's
cached locally under the repo) is a much more fundamental, version-stable
code path than the HF_HUB_OFFLINE constant-freezing chain, and is exactly
what made the earlier isolated repro (scenario_b.py) correctly resolve via
the mirror-snapshot cache without hitting the network.

Also made the debug print defensive (getattr with fallback) so a future
version mismatch degrades gracefully instead of crashing the whole call.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +40 -35
app.py CHANGED
@@ -232,46 +232,51 @@ print(f"[startup] main process pid={os.getpid()}")
232
 
233
  # Force any later HF Hub call to be cache-only. With every encoder cache
234
  # pre-populated above from the user's mirror, downstream from_pretrained()
235
- # calls (CLAP, AudioLDM2, MMAudio's open_clip 'hf-hub:...') resolve from
236
- # the populated cache instead of networking out to upstream β€” full
237
- # upstream-protection mode.
238
- #
239
- # Setting only the env var is NOT enough: huggingface_hub.constants.HF_HUB_OFFLINE
240
- # is computed once from os.environ at *import time* (constants.py), and
241
- # huggingface_hub was already imported above (line ~32) before this point, so
242
- # the env var alone never takes effect for the rest of this process. Confirmed
243
- # live: MMAudio's open_clip CLIP loader (create_model_from_pretrained('hf-hub:
244
- # apple/DFN5B-CLIP-ViT-H-14-384', ...)) still resolved "main" over the network,
245
- # didn't match our locally mirrored 'mirror-snapshot' cache entry's fake commit
246
- # hash, and re-downloaded the full ~4GB checkpoint mid-GPU-call β€” verified via
247
- # an isolated repro against the real huggingface_hub package (see scratchpad).
248
- # Patching the already-imported module's constant directly is what actually
249
- # gates the HTTP layer (huggingface_hub/utils/_http.py reads it live via
250
- # constants.is_offline_mode()), which is shared by transformers/diffusers/
251
- # open_clip alike since they all delegate the actual network call to it.
252
  os.environ["HF_HUB_OFFLINE"] = "1"
 
253
  import huggingface_hub.constants as _hf_constants
254
- _hf_constants.HF_HUB_OFFLINE = True
 
 
255
 
256
  def _ensure_offline_in_worker():
257
- """Re-assert offline mode from inside a @spaces.GPU function's own body.
258
-
259
- The module-level patch above (and an earlier attempt inside _catch_oom's
260
- wrapper, which sits *outside* the raw function) both failed to stop
261
- MMAudio's open_clip CLIP loader from re-downloading its ~4GB checkpoint
262
- live during GPU inference β€” confirmed via a direct HTTP repro against the
263
- deployed Space. This codebase's own docstrings already document that
264
- @spaces.GPU functions run in a genuinely separate "GPU worker process"
265
- (see e.g. _mmaudio_gpu_infer: "kwargs are silently dropped" / paths must
266
- "exist in the GPU worker's process"), so any patch applied by code that
267
- merely *wraps* the call β€” rather than running inside the function body
268
- that's actually shipped to and executed by that worker β€” never reaches
269
- it. Calling this as literally the first line of each raw GPU function
270
- guarantees it runs wherever that function's own bytecode runs."""
271
  os.environ["HF_HUB_OFFLINE"] = "1"
272
- _hf_constants.HF_HUB_OFFLINE = True
273
- print(f"[_ensure_offline_in_worker] pid={os.getpid()} HF_HUB_OFFLINE={_hf_constants.HF_HUB_OFFLINE} "
274
- f"is_offline_mode()={_hf_constants.is_offline_mode()}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
275
 
276
  # ================================================================== #
277
  # SHARED CONSTANTS / HELPERS #
 
232
 
233
  # Force any later HF Hub call to be cache-only. With every encoder cache
234
  # pre-populated above from the user's mirror, downstream from_pretrained()
235
+ # calls (CLAP, AudioLDM2) resolve from the populated cache instead of
236
+ # networking out to upstream. Best-effort: the exact attribute name/shape of
237
+ # huggingface_hub's offline-mode internals has proven to vary by installed
238
+ # version (an is_offline_mode() debug print crashed live with AttributeError,
239
+ # meaning the deployed version's constants.py doesn't match what was tested
240
+ # against locally) β€” os.environ is the one thing every version reads at
241
+ # *some* point, so set it and best-effort mirror it onto the constant too,
242
+ # but don't rely on this alone (see the open_clip-specific patch below for
243
+ # the one confirmed, reproduced failure).
 
 
 
 
 
 
 
 
244
  os.environ["HF_HUB_OFFLINE"] = "1"
245
+ import huggingface_hub
246
  import huggingface_hub.constants as _hf_constants
247
+ print(f"[startup] huggingface_hub version={getattr(huggingface_hub, '__version__', '?')}")
248
+ if hasattr(_hf_constants, "HF_HUB_OFFLINE"):
249
+ _hf_constants.HF_HUB_OFFLINE = True
250
 
251
  def _ensure_offline_in_worker():
252
+ """Re-assert offline mode from inside a @spaces.GPU function's own body
253
+ (called as the first line of all six raw GPU functions) in case the
254
+ worker thread anyio dispatches @spaces.GPU calls onto doesn't share the
255
+ same already-mutated module state as the main thread."""
 
 
 
 
 
 
 
 
 
 
256
  os.environ["HF_HUB_OFFLINE"] = "1"
257
+ if hasattr(_hf_constants, "HF_HUB_OFFLINE"):
258
+ _hf_constants.HF_HUB_OFFLINE = True
259
+ print(f"[_ensure_offline_in_worker] pid={os.getpid()} "
260
+ f"HF_HUB_OFFLINE={getattr(_hf_constants, 'HF_HUB_OFFLINE', '?')}")
261
+
262
+ # The one CONFIRMED, reproduced failure: MMAudio's open_clip CLIP loader
263
+ # (create_model_from_pretrained('hf-hub:apple/DFN5B-CLIP-ViT-H-14-384', ...))
264
+ # re-downloads its ~4GB checkpoint live during GPU inference regardless of the
265
+ # HF_HUB_OFFLINE patches above β€” verified via a direct HTTP repro against the
266
+ # deployed Space (queue/join straight to _run_mmaudio, bypassing the browser).
267
+ # open_clip.pretrained calls a *bound* local reference to hf_hub_download
268
+ # (`from huggingface_hub import hf_hub_download` at its own import time), so
269
+ # rather than continue chasing which huggingface_hub-internal offline-mode
270
+ # attribute the deployed version actually honors, force the one parameter
271
+ # that unambiguously controls this regardless of version: local_files_only.
272
+ import open_clip.pretrained as _open_clip_pretrained
273
+ _orig_open_clip_hf_hub_download = _open_clip_pretrained.hf_hub_download
274
+
275
+ def _patched_open_clip_hf_hub_download(*args, **kwargs):
276
+ kwargs["local_files_only"] = True
277
+ return _orig_open_clip_hf_hub_download(*args, **kwargs)
278
+
279
+ _open_clip_pretrained.hf_hub_download = _patched_open_clip_hf_hub_download
280
 
281
  # ================================================================== #
282
  # SHARED CONSTANTS / HELPERS #