Spaces:
Running on Zero
fix: Accept the Space's existing CONTROLAI_BACKEND spelling, and stop guessing
Browse filesThe Space boots with CONTROLAI_BACKEND=pytorch, a variable left over from the
old orchestrator's escape hatch (dfe7123). _make_engine tested for "torch"
exactly, so it did not match, fell through to LocalEngine on a machine with no
MLX, and failed with ModuleNotFoundError: mlx_lm several frames below the actual
cause. app_space.py's os.environ.setdefault could not correct it either, since
the variable was already set.
Three changes, because the silent fallback was the real defect:
- "pytorch" and "cuda" are accepted alongside "torch".
- An unrecognised value now raises instead of quietly selecting MLX. A misspelt
backend should fail at the point of the mistake, naming the variable.
- app_space.py assigns CONTROLAI_BACKEND rather than setdefault-ing it. That
module is the CUDA entry point by definition; a stale Space variable must not
be able to select something else.
The rest of the boot was clean: the build resolved, the private index downloaded
in about three seconds, and the ZeroGPU probe came up on its own port.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- CLAUDE.md +6 -2
- app.py +13 -1
- app_space.py +5 -2
|
@@ -181,9 +181,13 @@ of the serving path and uses `requirements-training.txt`/`requirements-corpus.tx
|
|
| 181 |
|
| 182 |
### Deployment (`app_space.py`, `engine_torch.py`, `requirements-space.txt`)
|
| 183 |
The demo Space (huggingface.co/spaces/atakankahya/ControlAI-Agent) runs Linux/NVIDIA on ZeroGPU,
|
| 184 |
-
where MLX does not exist. `CONTROLAI_BACKEND
|
| 185 |
`TorchEngine` instead of `LocalEngine`; `Embedder` switches on the same variable. That is the whole
|
| 186 |
-
switch — two branches, no orchestrator.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
|
| 188 |
`engine_torch.py` mirrors `engine.py` rather than calling `model.generate`, because `generate`
|
| 189 |
cannot express either of the two things that matter: `DynamicCache.crop()` for prefix reuse across
|
|
|
|
| 181 |
|
| 182 |
### Deployment (`app_space.py`, `engine_torch.py`, `requirements-space.txt`)
|
| 183 |
The demo Space (huggingface.co/spaces/atakankahya/ControlAI-Agent) runs Linux/NVIDIA on ZeroGPU,
|
| 184 |
+
where MLX does not exist. `CONTROLAI_BACKEND` makes `app.py::_make_engine` build
|
| 185 |
`TorchEngine` instead of `LocalEngine`; `Embedder` switches on the same variable. That is the whole
|
| 186 |
+
switch — two branches, no orchestrator. It accepts `torch`, `pytorch` or `cuda`, and **raises on
|
| 187 |
+
anything it does not recognise rather than falling back to MLX**. The deployed Space still carries
|
| 188 |
+
`CONTROLAI_BACKEND=pytorch` as a variable from the old orchestrator; a check for `"torch"` alone
|
| 189 |
+
silently selected MLX on a box with no MLX, and the failure surfaced as `ModuleNotFoundError:
|
| 190 |
+
mlx_lm` several frames from the real cause.
|
| 191 |
|
| 192 |
`engine_torch.py` mirrors `engine.py` rather than calling `model.generate`, because `generate`
|
| 193 |
cannot express either of the two things that matter: `DynamicCache.crop()` for prefix reuse across
|
|
@@ -66,10 +66,22 @@ def _make_engine():
|
|
| 66 |
never needs torch installed. Everything downstream -- the agent loop, the
|
| 67 |
registry, every tool -- is identical either way.
|
| 68 |
"""
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
| 70 |
from controlai_agent.engine_torch import TorchEngine
|
| 71 |
|
| 72 |
return TorchEngine()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
return None # ControlAgent's own default, LocalEngine
|
| 74 |
|
| 75 |
|
|
|
|
| 66 |
never needs torch installed. Everything downstream -- the agent loop, the
|
| 67 |
registry, every tool -- is identical either way.
|
| 68 |
"""
|
| 69 |
+
backend = os.environ.get("CONTROLAI_BACKEND", "mlx").strip().lower()
|
| 70 |
+
# "pytorch" is what the old orchestrator called this and it survives as a
|
| 71 |
+
# variable on the deployed Space; "cuda" is the obvious other guess.
|
| 72 |
+
if backend in ("torch", "pytorch", "cuda"):
|
| 73 |
from controlai_agent.engine_torch import TorchEngine
|
| 74 |
|
| 75 |
return TorchEngine()
|
| 76 |
+
if backend not in ("mlx", ""):
|
| 77 |
+
# Never fall through to MLX because a variable was misspelt. That is
|
| 78 |
+
# exactly what happened on the Space: CONTROLAI_BACKEND=pytorch did not
|
| 79 |
+
# match a check for "torch", so it built LocalEngine on a box with no
|
| 80 |
+
# MLX and died in an import several frames deeper than the real cause.
|
| 81 |
+
raise ValueError(
|
| 82 |
+
f"CONTROLAI_BACKEND={backend!r} is not a known backend "
|
| 83 |
+
f"(expected one of: mlx, torch/pytorch/cuda)"
|
| 84 |
+
)
|
| 85 |
return None # ControlAgent's own default, LocalEngine
|
| 86 |
|
| 87 |
|
|
@@ -29,8 +29,11 @@ from __future__ import annotations
|
|
| 29 |
|
| 30 |
import os
|
| 31 |
|
| 32 |
-
#
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
| 34 |
# Qwen3-14B in 4-bit NF4. Override with CONTROLAI_MODEL_TORCH in Space settings.
|
| 35 |
|
| 36 |
import gradio as gr
|
|
|
|
| 29 |
|
| 30 |
import os
|
| 31 |
|
| 32 |
+
# Assigned, not setdefault: this module is the CUDA entry point by definition,
|
| 33 |
+
# and a stale Space variable must not be able to select something else. One did
|
| 34 |
+
# -- CONTROLAI_BACKEND=pytorch, left over from the old orchestrator -- and the
|
| 35 |
+
# Space booted into the MLX engine and died on `import mlx_lm`.
|
| 36 |
+
os.environ["CONTROLAI_BACKEND"] = "torch"
|
| 37 |
# Qwen3-14B in 4-bit NF4. Override with CONTROLAI_MODEL_TORCH in Space settings.
|
| 38 |
|
| 39 |
import gradio as gr
|