atakan Claude Opus 5 commited on
Commit
5388c07
·
1 Parent(s): c74703c

fix: Mount the ZeroGPU probe instead of launching a second server

Browse files

Reading spaces 0.51.1 source: the GPU worker is created with
multiprocessing.get_context('fork'), and the child immediately does
torch.unpatch() then torch.init(nvidia_uuid), which sets CUDA_VISIBLE_DEVICES to
the allocated UUID and calls torch.Tensor([0]).cuda(). Our failure is entirely
inside that, before any of our code.

The bisect was already in hand: /api/gpudiag's @spaces.GPU function does nothing
but read env vars, and it failed before its body ran. So the model, the 18.8GB of
packed tensors and the payload are all ruled out -- *any* @spaces.GPU call from
this process fails. That makes it a property of the process being forked.

And the startup log names one: Gradio 6 launches with SSR, "Node proxy ->
Python :7862". _gpu_demo.launch(prevent_thread_lock=True) was putting a Node
subprocess and a server thread into the exact process ZeroGPU forks from.
Mounting instead gives one server in one process with neither, and still counts
as wiring _gpu_probe to a Gradio event handler for startup validation.

This reverses an earlier note that mounting collides with "address already in
use". That note was right about what it saw and wrong about the cause: the
collision came from mounting *and* calling launch(), two servers. Mounting alone
under a single uvicorn.run is one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app_space.py +13 -12
app_space.py CHANGED
@@ -10,10 +10,13 @@ ControlAI, and each of the four below was learned by having the Space fail:
10
  function detected". Hence the hidden probe button.
11
  2. That function must be a module-level `def`. Nested inside a `with
12
  gr.Blocks():` block, the same detection fails.
13
- 3. Do not `gr.mount_gradio_app()` the probe into the FastAPI app and then run
14
- uvicorn on the same port -- Gradio's own server setup collides with it
15
- ("address already in use"). The probe launches on its own port; FastAPI is
16
- the only thing bound to the public one.
 
 
 
17
  4. Never pass the model-holding object as an *argument* to a `@spaces.GPU`
18
  function. ZeroGPU marshals arguments across a process boundary and will try
19
  to share the model's CUDA tensors, failing with `_share_cuda_: only
@@ -214,14 +217,12 @@ with gr.Blocks() as _gpu_demo:
214
 
215
  def main() -> None:
216
  port = int(os.environ.get("PORT", 7860))
217
- # Separate port, non-blocking: see note 3.
218
- _gpu_demo.launch(
219
- server_name="0.0.0.0",
220
- server_port=port + 1,
221
- prevent_thread_lock=True,
222
- share=False,
223
- )
224
- uvicorn.run(app, host="0.0.0.0", port=port, log_level="info")
225
 
226
 
227
  if __name__ == "__main__":
 
10
  function detected". Hence the hidden probe button.
11
  2. That function must be a module-level `def`. Nested inside a `with
12
  gr.Blocks():` block, the same detection fails.
13
+ 3. Mount the probe into the FastAPI app; do not `launch()` it. An earlier note
14
+ said the opposite -- that mounting collided with "address already in use" --
15
+ but that was mounting *and* calling `launch()`, i.e. two servers. Mounting
16
+ alone, with one `uvicorn.run`, is a single server and does not collide.
17
+ Launching it separately is actively harmful now: Gradio 6 launches with SSR,
18
+ which spawns a Node subprocess and a server thread into the very process
19
+ ZeroGPU then `fork()`s its CUDA worker from.
20
  4. Never pass the model-holding object as an *argument* to a `@spaces.GPU`
21
  function. ZeroGPU marshals arguments across a process boundary and will try
22
  to share the model's CUDA tensors, failing with `_share_cuda_: only
 
217
 
218
  def main() -> None:
219
  port = int(os.environ.get("PORT", 7860))
220
+ # Mounted, not launched: one server, one process, no Node SSR subprocess and
221
+ # no extra server thread in the process ZeroGPU forks its CUDA worker from.
222
+ # The mount still counts as wiring _gpu_probe to a Gradio event handler,
223
+ # which is what the platform's startup validation looks for.
224
+ merged = gr.mount_gradio_app(app, _gpu_demo, path="/_gpu")
225
+ uvicorn.run(merged, host="0.0.0.0", port=port, log_level="info")
 
 
226
 
227
 
228
  if __name__ == "__main__":