Spaces:
Running on Zero
fix: Mount the ZeroGPU probe instead of launching a second server
Browse filesReading spaces 0.51.1 source: the GPU worker is created with
multiprocessing.get_context('fork'), and the child immediately does
torch.unpatch() then torch.init(nvidia_uuid), which sets CUDA_VISIBLE_DEVICES to
the allocated UUID and calls torch.Tensor([0]).cuda(). Our failure is entirely
inside that, before any of our code.
The bisect was already in hand: /api/gpudiag's @spaces.GPU function does nothing
but read env vars, and it failed before its body ran. So the model, the 18.8GB of
packed tensors and the payload are all ruled out -- *any* @spaces.GPU call from
this process fails. That makes it a property of the process being forked.
And the startup log names one: Gradio 6 launches with SSR, "Node proxy ->
Python :7862". _gpu_demo.launch(prevent_thread_lock=True) was putting a Node
subprocess and a server thread into the exact process ZeroGPU forks from.
Mounting instead gives one server in one process with neither, and still counts
as wiring _gpu_probe to a Gradio event handler for startup validation.
This reverses an earlier note that mounting collides with "address already in
use". That note was right about what it saw and wrong about the cause: the
collision came from mounting *and* calling launch(), two servers. Mounting alone
under a single uvicorn.run is one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- app_space.py +13 -12
|
@@ -10,10 +10,13 @@ ControlAI, and each of the four below was learned by having the Space fail:
|
|
| 10 |
function detected". Hence the hidden probe button.
|
| 11 |
2. That function must be a module-level `def`. Nested inside a `with
|
| 12 |
gr.Blocks():` block, the same detection fails.
|
| 13 |
-
3.
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
| 17 |
4. Never pass the model-holding object as an *argument* to a `@spaces.GPU`
|
| 18 |
function. ZeroGPU marshals arguments across a process boundary and will try
|
| 19 |
to share the model's CUDA tensors, failing with `_share_cuda_: only
|
|
@@ -214,14 +217,12 @@ with gr.Blocks() as _gpu_demo:
|
|
| 214 |
|
| 215 |
def main() -> None:
|
| 216 |
port = int(os.environ.get("PORT", 7860))
|
| 217 |
-
#
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
)
|
| 224 |
-
uvicorn.run(app, host="0.0.0.0", port=port, log_level="info")
|
| 225 |
|
| 226 |
|
| 227 |
if __name__ == "__main__":
|
|
|
|
| 10 |
function detected". Hence the hidden probe button.
|
| 11 |
2. That function must be a module-level `def`. Nested inside a `with
|
| 12 |
gr.Blocks():` block, the same detection fails.
|
| 13 |
+
3. Mount the probe into the FastAPI app; do not `launch()` it. An earlier note
|
| 14 |
+
said the opposite -- that mounting collided with "address already in use" --
|
| 15 |
+
but that was mounting *and* calling `launch()`, i.e. two servers. Mounting
|
| 16 |
+
alone, with one `uvicorn.run`, is a single server and does not collide.
|
| 17 |
+
Launching it separately is actively harmful now: Gradio 6 launches with SSR,
|
| 18 |
+
which spawns a Node subprocess and a server thread into the very process
|
| 19 |
+
ZeroGPU then `fork()`s its CUDA worker from.
|
| 20 |
4. Never pass the model-holding object as an *argument* to a `@spaces.GPU`
|
| 21 |
function. ZeroGPU marshals arguments across a process boundary and will try
|
| 22 |
to share the model's CUDA tensors, failing with `_share_cuda_: only
|
|
|
|
| 217 |
|
| 218 |
def main() -> None:
|
| 219 |
port = int(os.environ.get("PORT", 7860))
|
| 220 |
+
# Mounted, not launched: one server, one process, no Node SSR subprocess and
|
| 221 |
+
# no extra server thread in the process ZeroGPU forks its CUDA worker from.
|
| 222 |
+
# The mount still counts as wiring _gpu_probe to a Gradio event handler,
|
| 223 |
+
# which is what the platform's startup validation looks for.
|
| 224 |
+
merged = gr.mount_gradio_app(app, _gpu_demo, path="/_gpu")
|
| 225 |
+
uvicorn.run(merged, host="0.0.0.0", port=port, log_level="info")
|
|
|
|
|
|
|
| 226 |
|
| 227 |
|
| 228 |
if __name__ == "__main__":
|