fix: Build the Space's agent at import scope, where ZeroGPU is watching

Third failure, same RuntimeError, but the traceback finally names the line:
engine_torch.py:102, `self.model.to("cuda")`. So it was never device_map
specifically -- .to("cuda") fails too, and the previous commit's reasoning was
only half right.

ZeroGPU patches torch during the import of the Space's entry module, and only
CUDA operations inside that window are intercepted. app.py builds the model in
FastAPI's lifespan, on a ThreadPoolExecutor worker: after import, on another
thread, outside the window. Any CUDA op there reaches the real
torch._C._cuda_init and raises. That is also why the note from the last working
deploy -- "explicit .to('cuda') works" -- did not carry over: back then the model
was built at import, not in a lifespan.

So where the engine is constructed matters as much as how.
app_space.py::_build_agent_at_import builds it during its own import and assigns
app._agent directly, leaving lifespan's get_agent() a no-op. _fetch_index moves
ahead of it, since the agent touches retrieval as it starts.

device_map remains excluded for its own separate reason (caching_allocator_warmup
calls torch.empty(device="cuda") directly), and bitsandbytes with it, so bf16 and
a model small enough to carry in it stand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CLAUDE.md

@@ -193,16 +193,19 @@ mlx_lm` several frames from the real cause.
 cannot express either of the two things that matter: `DynamicCache.crop()` for prefix reuse across
 tool steps, and injecting `</think>` to close an overrunning reasoning block.
 
-**It loads bf16 and moves the model with an explicit `.to("cuda")`. Do not reintroduce `device_map`
-or bitsandbytes.** Both are the obvious thing to reach for and both fail on ZeroGPU with
-`RuntimeError: Low-level CUDA init (torch._C._cuda_init) reached`. ZeroGPU emulates CUDA at startup
-and attaches real hardware only inside a `@spaces.GPU` call; its emulation intercepts `.to("cuda")`,
-but `device_map` routes transformers through `caching_allocator_warmup`, which calls
-`torch.empty(..., device="cuda")` directly and trips the guard. bitsandbytes *requires* `device_map`
-at load, so **4-bit quantisation is unavailable** while the model is loaded at startup rather than
-inside the GPU function. That is the constraint that sets the Space's model: `Qwen/Qwen3-8B` in
-bf16, ~16GB to download, against ~28GB for the 14B run locally — and a Space rebuild re-downloads
-from scratch.
+**It loads bf16 with an explicit `.to("cuda")`, and `app_space.py` constructs it at module import
+scope. Do not reintroduce `device_map` or bitsandbytes, and do not move the construction into a
+lifespan or a request.** All of those fail identically with `RuntimeError: Low-level CUDA init
+(torch._C._cuda_init) reached`. ZeroGPU patches torch during the import of the Space's entry module
+and attaches real hardware only inside a `@spaces.GPU` call; only CUDA operations inside that import
+window are intercepted. `app.py` normally builds the model in FastAPI's `lifespan`, on a
+`ThreadPoolExecutor` worker — after import, on another thread — and `.to("cuda")` there reaches real
+CUDA init and raises. `app_space.py::_build_agent_at_import` builds the agent during import and
+assigns `app._agent`, so `lifespan`'s `get_agent()` is a no-op. `device_map` fails independently, by
+routing transformers through `caching_allocator_warmup` and its direct
+`torch.empty(..., device="cuda")`; bitsandbytes requires `device_map`, so **4-bit quantisation is
+unavailable**, which is what forces a model small enough to carry in bf16: `Qwen/Qwen3-8B`, ~16GB
+against ~28GB for the 14B run locally.
 
 **ZeroGPU platform gotchas, each learned by having the Space fail:**
 - A `@spaces.GPU` function is only detected if wired to a real Gradio event handler. One called

app_space.py

@@ -40,7 +40,8 @@ import gradio as gr
 import spaces
 import uvicorn
 
-from app import app, get_agent
+import app as app_module
+from app import app
 
 
 def _fetch_index() -> None:
@@ -57,6 +58,34 @@ def _fetch_index() -> None:
         print(f"[space] could not fetch the index ({exc}); retrieval disabled")
 
 
+def _build_agent_at_import() -> None:
+    """Construct the agent *now*, while this module is still being imported.
+
+    ZeroGPU installs its CUDA emulation by patching torch during the import of
+    the Space's entry module, and only operations that happen inside that window
+    are intercepted. `app.py` normally builds the model in FastAPI's `lifespan`,
+    on a ThreadPoolExecutor worker -- long after import, on another thread, where
+    the patching does not apply. `.to("cuda")` there reaches the real
+    `torch._C._cuda_init` and raises:
+
+        RuntimeError: Low-level CUDA init (`torch._C._cuda_init`) reached.
+
+    Building here and handing the finished agent to `app.py` keeps the load
+    inside the window. `lifespan` then finds `_agent` already set and its
+    `get_agent()` is a no-op.
+    """
+    from controlai_agent.agent import ControlAgent
+    from controlai_agent.engine_torch import TorchEngine
+
+    print("[space] building agent at import scope (ZeroGPU CUDA window)")
+    app_module._agent = ControlAgent(engine=TorchEngine())
+    print("[space] agent ready")
+
+
+_fetch_index()          # the agent prewarms retrieval, so the index must precede it
+_build_agent_at_import()
+
+
 @spaces.GPU(duration=120)
 def _gpu_probe(text: str) -> str:
     """Satisfies ZeroGPU's startup validation, and warms the model on first use.
@@ -64,8 +93,7 @@ def _gpu_probe(text: str) -> str:
     Takes only a plain string. The agent is reached through the module global --
     see note 4 above; passing it in is what hangs the whole Space.
     """
-    agent = get_agent()
-    return agent.run(text).answer if text else "ready"
+    return app_module.get_agent().run(text).answer if text else "ready"
 
 
 with gr.Blocks() as _gpu_demo:
@@ -76,7 +104,6 @@ with gr.Blocks() as _gpu_demo:
 
 
 def main() -> None:
-    _fetch_index()
     port = int(os.environ.get("PORT", 7860))
     # Separate port, non-blocking: see note 3.
     _gpu_demo.launch(

controlai_agent/engine_torch.py

@@ -20,19 +20,22 @@ It mirrors `LocalEngine`'s design decisions rather than reaching for
     punishes the `[`, `0`, `,` that matrices and JSON are made of.
 
 **Loading is bf16 and moves to the GPU with an explicit `.to("cuda")`. Do not
-reintroduce `device_map` or bitsandbytes here.** Both are the obvious thing to
-reach for and both break on ZeroGPU, which is the only place this file runs:
+reintroduce `device_map` or bitsandbytes, and do not construct this class lazily
+at request time.** All three break on ZeroGPU, which is the only place this file
+runs, and all three fail the same way:
 
     RuntimeError: Low-level CUDA init (`torch._C._cuda_init`) reached. This
     means ZeroGPU's PyTorch CUDA emulation mode did not intercept a CUDA
     operation in your code.
 
-ZeroGPU emulates CUDA at startup and attaches real hardware only inside a
-`@spaces.GPU` call. Its emulation intercepts `.to("cuda")`, but passing
-`device_map` sends transformers through `caching_allocator_warmup`, which calls
-`torch.empty(..., device="cuda")` directly and trips the guard. bitsandbytes in
-turn *requires* `device_map` at load, so 4-bit quantisation is unavailable here
-as long as the model is loaded at startup rather than inside the GPU function.
+ZeroGPU patches torch during the import of the Space's entry module and attaches
+real hardware only inside a `@spaces.GPU` call. Only CUDA operations inside that
+import window are intercepted, so **where this object is constructed matters as
+much as how**: `app_space.py` builds it at module scope for exactly that reason.
+`device_map` fails on top of that, because it routes transformers through
+`caching_allocator_warmup`, which calls `torch.empty(..., device="cuda")`
+directly. bitsandbytes in turn *requires* `device_map`, so 4-bit quantisation is
+unavailable here — which is why the model has to be small enough in bf16.
 
 That is why the model is Qwen3-8B rather than the 14B run locally: bf16 8B is
 ~16GB to download against ~28GB, and a Space rebuild re-downloads from scratch.