Force local_files_only on open_clip's own hf_hub_download reference
Browse filesLive repro crashed the previous fix's own debug line: AttributeError,
huggingface_hub.constants has no is_offline_mode() on the deployed version β
meaning the whole HF_HUB_OFFLINE/is_offline_mode() approach was built against
the wrong version's internals, explaining why neither prior attempt (78108d3,
035533f) actually stopped the live re-download despite being individually
verified against huggingface_hub 1.23.0 in isolation.
The traceback also showed the @spaces.GPU call running through
anyio.to_thread.run_sync β a worker thread in the same process, not the
separate process the code's own docstrings had suggested β so process
isolation wasn't the blocker either.
Rather than keep chasing which offline-mode attribute the actually-deployed
huggingface_hub version honors, sidestep it: open_clip.pretrained binds its
own local `hf_hub_download` name at import time
(`from huggingface_hub import hf_hub_download`), so monkeypatch that specific
module-level reference to force local_files_only=True on every call. That
parameter's handling (skip the network HEAD call, fall back to whatever's
cached locally under the repo) is a much more fundamental, version-stable
code path than the HF_HUB_OFFLINE constant-freezing chain, and is exactly
what made the earlier isolated repro (scenario_b.py) correctly resolve via
the mirror-snapshot cache without hitting the network.
Also made the debug print defensive (getattr with fallback) so a future
version mismatch degrades gracefully instead of crashing the whole call.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
@@ -232,46 +232,51 @@ print(f"[startup] main process pid={os.getpid()}")
|
|
| 232 |
|
| 233 |
# Force any later HF Hub call to be cache-only. With every encoder cache
|
| 234 |
# pre-populated above from the user's mirror, downstream from_pretrained()
|
| 235 |
-
# calls (CLAP, AudioLDM2
|
| 236 |
-
#
|
| 237 |
-
#
|
| 238 |
-
#
|
| 239 |
-
#
|
| 240 |
-
#
|
| 241 |
-
#
|
| 242 |
-
#
|
| 243 |
-
#
|
| 244 |
-
# apple/DFN5B-CLIP-ViT-H-14-384', ...)) still resolved "main" over the network,
|
| 245 |
-
# didn't match our locally mirrored 'mirror-snapshot' cache entry's fake commit
|
| 246 |
-
# hash, and re-downloaded the full ~4GB checkpoint mid-GPU-call β verified via
|
| 247 |
-
# an isolated repro against the real huggingface_hub package (see scratchpad).
|
| 248 |
-
# Patching the already-imported module's constant directly is what actually
|
| 249 |
-
# gates the HTTP layer (huggingface_hub/utils/_http.py reads it live via
|
| 250 |
-
# constants.is_offline_mode()), which is shared by transformers/diffusers/
|
| 251 |
-
# open_clip alike since they all delegate the actual network call to it.
|
| 252 |
os.environ["HF_HUB_OFFLINE"] = "1"
|
|
|
|
| 253 |
import huggingface_hub.constants as _hf_constants
|
| 254 |
-
|
|
|
|
|
|
|
| 255 |
|
| 256 |
def _ensure_offline_in_worker():
|
| 257 |
-
"""Re-assert offline mode from inside a @spaces.GPU function's own body
|
| 258 |
-
|
| 259 |
-
|
| 260 |
-
|
| 261 |
-
MMAudio's open_clip CLIP loader from re-downloading its ~4GB checkpoint
|
| 262 |
-
live during GPU inference β confirmed via a direct HTTP repro against the
|
| 263 |
-
deployed Space. This codebase's own docstrings already document that
|
| 264 |
-
@spaces.GPU functions run in a genuinely separate "GPU worker process"
|
| 265 |
-
(see e.g. _mmaudio_gpu_infer: "kwargs are silently dropped" / paths must
|
| 266 |
-
"exist in the GPU worker's process"), so any patch applied by code that
|
| 267 |
-
merely *wraps* the call β rather than running inside the function body
|
| 268 |
-
that's actually shipped to and executed by that worker β never reaches
|
| 269 |
-
it. Calling this as literally the first line of each raw GPU function
|
| 270 |
-
guarantees it runs wherever that function's own bytecode runs."""
|
| 271 |
os.environ["HF_HUB_OFFLINE"] = "1"
|
| 272 |
-
_hf_constants
|
| 273 |
-
|
| 274 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 275 |
|
| 276 |
# ================================================================== #
|
| 277 |
# SHARED CONSTANTS / HELPERS #
|
|
|
|
| 232 |
|
| 233 |
# Force any later HF Hub call to be cache-only. With every encoder cache
|
| 234 |
# pre-populated above from the user's mirror, downstream from_pretrained()
|
| 235 |
+
# calls (CLAP, AudioLDM2) resolve from the populated cache instead of
|
| 236 |
+
# networking out to upstream. Best-effort: the exact attribute name/shape of
|
| 237 |
+
# huggingface_hub's offline-mode internals has proven to vary by installed
|
| 238 |
+
# version (an is_offline_mode() debug print crashed live with AttributeError,
|
| 239 |
+
# meaning the deployed version's constants.py doesn't match what was tested
|
| 240 |
+
# against locally) β os.environ is the one thing every version reads at
|
| 241 |
+
# *some* point, so set it and best-effort mirror it onto the constant too,
|
| 242 |
+
# but don't rely on this alone (see the open_clip-specific patch below for
|
| 243 |
+
# the one confirmed, reproduced failure).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 244 |
os.environ["HF_HUB_OFFLINE"] = "1"
|
| 245 |
+
import huggingface_hub
|
| 246 |
import huggingface_hub.constants as _hf_constants
|
| 247 |
+
print(f"[startup] huggingface_hub version={getattr(huggingface_hub, '__version__', '?')}")
|
| 248 |
+
if hasattr(_hf_constants, "HF_HUB_OFFLINE"):
|
| 249 |
+
_hf_constants.HF_HUB_OFFLINE = True
|
| 250 |
|
| 251 |
def _ensure_offline_in_worker():
|
| 252 |
+
"""Re-assert offline mode from inside a @spaces.GPU function's own body
|
| 253 |
+
(called as the first line of all six raw GPU functions) in case the
|
| 254 |
+
worker thread anyio dispatches @spaces.GPU calls onto doesn't share the
|
| 255 |
+
same already-mutated module state as the main thread."""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 256 |
os.environ["HF_HUB_OFFLINE"] = "1"
|
| 257 |
+
if hasattr(_hf_constants, "HF_HUB_OFFLINE"):
|
| 258 |
+
_hf_constants.HF_HUB_OFFLINE = True
|
| 259 |
+
print(f"[_ensure_offline_in_worker] pid={os.getpid()} "
|
| 260 |
+
f"HF_HUB_OFFLINE={getattr(_hf_constants, 'HF_HUB_OFFLINE', '?')}")
|
| 261 |
+
|
| 262 |
+
# The one CONFIRMED, reproduced failure: MMAudio's open_clip CLIP loader
|
| 263 |
+
# (create_model_from_pretrained('hf-hub:apple/DFN5B-CLIP-ViT-H-14-384', ...))
|
| 264 |
+
# re-downloads its ~4GB checkpoint live during GPU inference regardless of the
|
| 265 |
+
# HF_HUB_OFFLINE patches above β verified via a direct HTTP repro against the
|
| 266 |
+
# deployed Space (queue/join straight to _run_mmaudio, bypassing the browser).
|
| 267 |
+
# open_clip.pretrained calls a *bound* local reference to hf_hub_download
|
| 268 |
+
# (`from huggingface_hub import hf_hub_download` at its own import time), so
|
| 269 |
+
# rather than continue chasing which huggingface_hub-internal offline-mode
|
| 270 |
+
# attribute the deployed version actually honors, force the one parameter
|
| 271 |
+
# that unambiguously controls this regardless of version: local_files_only.
|
| 272 |
+
import open_clip.pretrained as _open_clip_pretrained
|
| 273 |
+
_orig_open_clip_hf_hub_download = _open_clip_pretrained.hf_hub_download
|
| 274 |
+
|
| 275 |
+
def _patched_open_clip_hf_hub_download(*args, **kwargs):
|
| 276 |
+
kwargs["local_files_only"] = True
|
| 277 |
+
return _orig_open_clip_hf_hub_download(*args, **kwargs)
|
| 278 |
+
|
| 279 |
+
_open_clip_pretrained.hf_hub_download = _patched_open_clip_hf_hub_download
|
| 280 |
|
| 281 |
# ================================================================== #
|
| 282 |
# SHARED CONSTANTS / HELPERS #
|