Route ZeroGPU auto backend to Transformers

Co-authored-by: Codex <chatgpt-codex-connector[bot]@users.noreply.github.com>

DEPLOY.md

@@ -5,12 +5,12 @@ The active Hugging Face deployment is a **Gradio ZeroGPU Space**.
 This workflow is intentionally fixed:
 
 1. The Space must stay a Gradio Space, not a Docker Space.
-2. Runtime extraction should use the `llamacpp-gpu` backend when we are targeting the Llama Champion badge.
+2. Runtime extraction should use the Transformers backend on ZeroGPU.
 3. The extraction call must run behind `@spaces.GPU` so Hugging Face allocates ZeroGPU only while the model is needed.
 4. Model files must not be committed to the Space git repo.
-5. When the fine-tuned GGUF model is ready, only replace the `LLAMACPP_*` model variables.
+5. When the fine-tuned model is ready, replace only the model variables for the active lanes.
 
-Do not change this architecture unless the project intentionally gives up ZeroGPU or the llama.cpp backend proves incompatible with ZeroGPU. The intended future model-serving change is inserting the fine-tuned GGUF repository path into the existing `LLAMACPP_*` variables.
+Do not change this architecture unless the project intentionally gives up ZeroGPU. The intended future model-serving change is inserting the fine-tuned Transformers repository into `ZEROGPU_MODEL_ID`, and optionally inserting the fine-tuned GGUF repository into `LLAMACPP_*` for CPU fallback.
 
 ## 1. Space Metadata
 
@@ -34,53 +34,51 @@ ZeroGPU is Gradio-only on Hugging Face. It is not available for Docker Spaces, w
 
 ## 2. Model Serving
 
-The badge-target model path is the official OpenBMB GGUF repo running through llama.cpp:
+The active ZeroGPU model path is the official OpenBMB Transformers repo:
 
 ```text
-LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
-LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
-LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 EXTRACTOR_BACKEND=auto
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
 ```
 
 The backend lives in:
 
 ```text
-src/extraction/llamacpp_gpu.py
+src/extraction/zerogpu_transformers.py
 ```
 
 It uses:
 
 ```python
-llama_cpp.Llama(...)
 @spaces.GPU(duration=120)
+transformers.AutoModelForImageTextToText
 ```
 
-This is a valid hackathon badge option because the submitted app remains a Gradio ZeroGPU Space,
-but the model runtime is `llama.cpp` over GGUF rather than a hosted inference API.
+This is the correct runtime for a ZeroGPU Space because the GPU is allocated only inside the
+decorated worker. A normal app-level `torch.cuda.is_available()` check may be false before the
+worker starts, so `auto` also checks Hugging Face's `ACCELERATOR` runtime variable for values such
+as `zero-a10g`.
 
-The safe fallback backend is the official OpenBMB Transformers model:
+The CPU fallback model path is the official OpenBMB GGUF repo running through llama.cpp:
 
 ```text
-EXTRACTOR_BACKEND=zerogpu
-ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
+LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
 ```
 
-Use the fallback only if `llama-cpp-python` cannot load MiniCPM-V 4.6 on ZeroGPU.
-
 ## 3. Future Fine-Tuned Model
 
 When the fine-tuned model is ready:
 
-1. Convert/quantize the fine-tuned model to GGUF.
-2. Upload the fine-tuned GGUF model and compatible mmproj file to a Hugging Face model repo.
-3. Keep the same Gradio + ZeroGPU + llama.cpp architecture.
+1. Upload the fine-tuned Transformers checkpoint to a Hugging Face model repo.
+2. Optionally convert/quantize the fine-tuned model to GGUF for CPU fallback.
+3. Keep the same Gradio + ZeroGPU/CUDA Transformers + CPU llama.cpp architecture.
 4. Change only these variables:
 
 ```bash
+ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-transformers-repo>
 LLAMACPP_GGUF_REPO=<owner>/<fine-tuned-minicpm-v-gguf-repo>
 LLAMACPP_MODEL_FILE=<fine-tuned-model>.gguf
-LLAMACPP_MMPROJ_FILE=<compatible-mmproj>.gguf
 ```
 
 Do not add model files to the Space git repo. Do not reintroduce Docker or `llama-server` for the ZeroGPU deployment.
@@ -93,8 +91,9 @@ This architecture keeps:
 
 - Free ZeroGPU eligibility.
 - No external hosted inference API calls.
-- A valid `llama.cpp` / GGUF runtime path for the Llama Champion badge, if `llama-cpp-python` is compatible.
-- A clean future swap to a fine-tuned GGUF model by changing only `LLAMACPP_*` variables.
+- The official OpenBMB Transformers runtime on ZeroGPU.
+- A CPU `llama.cpp` / GGUF fallback when the Space is not on ZeroGPU or CUDA.
+- A clean future swap to a fine-tuned model by changing only `ZEROGPU_MODEL_ID` and optional `LLAMACPP_*` variables.
 
 ## 5. Local Development
 

DEPLOYMENT_LOG.md

@@ -1,5 +1,31 @@
 # Deployment Log
 
+## 2026-06-13 — Route ZeroGPU to Transformers, CPU to llama.cpp
+
+Decision: keep `EXTRACTOR_BACKEND=auto`, but make ZeroGPU select the official OpenBMB
+Transformers backend instead of relying on app-level CUDA visibility.
+
+Why:
+- On ZeroGPU, CUDA is allocated only inside a `@spaces.GPU` worker, so
+  `torch.cuda.is_available()` can be false in normal Gradio app code.
+- The app was therefore selecting the CPU llama.cpp fallback even while the Space hardware was
+  configured as ZeroGPU.
+- The intended runtime behavior is now explicit: `ACCELERATOR=zero-a10g`, `ZERO_GPU=TRUE`, or
+  visible CUDA selects Transformers; CPU-only runtime selects llama.cpp.
+
+Space variables:
+
+```bash
+EXTRACTOR_BACKEND=auto
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+```
+
+CPU fallback after a Transformers failure is now opt-in with:
+
+```bash
+AUTO_FALLBACK_TO_LLAMACPP=1
+```
+
 ## 2026-06-10 — Switch from Docker Space to Gradio ZeroGPU
 
 Decision: use **Gradio ZeroGPU** as the active Hugging Face Space architecture.

README.md

@@ -49,10 +49,11 @@ The knowledge graph is educational context, not diagnosis. The lab-provided refe
 The Hugging Face Space is intentionally deployed as a **Gradio Space** with adaptive extraction.
 This is the active deployment path.
 
-With `EXTRACTOR_BACKEND=auto`, the app checks CUDA availability at runtime. If CUDA is visible, it
-uses the official OpenBMB MiniCPM-V 4.6 Transformers path. If CUDA is not visible, it falls back to
-the CPU `llama.cpp` GGUF path. If the CUDA worker fails, `auto` retries with CPU llama.cpp unless
-`AUTO_FALLBACK_TO_LLAMACPP=0`. The deterministic knowledge-graph enrichment and UI rendering stay
+With `EXTRACTOR_BACKEND=auto`, the app uses the official OpenBMB MiniCPM-V 4.6 Transformers path
+when the runtime is ZeroGPU or CUDA. On CPU-only hardware, it uses the CPU `llama.cpp` GGUF path.
+ZeroGPU is detected from Hugging Face's `ACCELERATOR` runtime variable, for example
+`ACCELERATOR=zero-a10g`, or from explicit runtime flags such as `ZERO_GPU=TRUE`. This matters
+because CUDA is only visible inside the `@spaces.GPU` worker. The deterministic knowledge-graph enrichment and UI rendering stay
 in normal Gradio/Python code.
 
 This workflow should not be further changed back to Docker unless the project intentionally gives up

RUNBOOK.md

@@ -10,7 +10,7 @@ This replaced the Docker + `llama-server` path because ZeroGPU is only available
 |---|---|
 | Space SDK | `gradio` |
 | Hardware | Adaptive: CUDA when available, CPU otherwise |
-| Auto backend | Transformers on CUDA, llama.cpp on CPU |
+| Auto backend | Transformers on ZeroGPU/CUDA, llama.cpp on CPU |
 | Force llama.cpp | `EXTRACTOR_BACKEND=llamacpp-gpu` |
 | Force Transformers | `EXTRACTOR_BACKEND=zerogpu` or `EXTRACTOR_BACKEND=transformers` |
 | llama.cpp variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE` |
@@ -24,7 +24,7 @@ Do not switch the Space back to Docker unless the project intentionally gives up
 
 `EXTRACTOR_BACKEND`:
 
-- `auto`: uses the Transformers backend when `torch.cuda.is_available()` is true; otherwise uses the CPU llama.cpp backend. If the Transformers worker fails, it retries with CPU llama.cpp unless `AUTO_FALLBACK_TO_LLAMACPP=0`.
+- `auto`: uses the Transformers backend when Hugging Face reports a ZeroGPU accelerator such as `ACCELERATOR=zero-a10g`, when `ZERO_GPU=TRUE` is set, or when CUDA is visible; otherwise uses the CPU llama.cpp backend. This runtime signal matters because CUDA is only visible inside a `@spaces.GPU` worker. CPU fallback after a Transformers failure is opt-in with `AUTO_FALLBACK_TO_LLAMACPP=1`.
 - `llamacpp-gpu`: force the GGUF llama.cpp backend.
 - `zerogpu` / `transformers`: force the OpenBMB Transformers backend.
 - `api`: hosted OpenBMB endpoint for development fallback only.
@@ -57,8 +57,8 @@ transformers
 llama-cpp-python
 ```
 
-The Space installs both runtime lanes so `EXTRACTOR_BACKEND=auto` can choose at runtime. CUDA
-hardware uses the official OpenBMB Transformers path. CPU hardware uses the prebuilt
+The Space installs both runtime lanes so `EXTRACTOR_BACKEND=auto` can choose at runtime. ZeroGPU or
+CUDA hardware uses the official OpenBMB Transformers path. CPU hardware uses the prebuilt
 `llama-cpp-python` wheel and avoids a source build.
 
 The active llama.cpp path now uses the official prebuilt CPU manylinux wheel for `llama-cpp-python`:
@@ -131,4 +131,4 @@ python3 -m py_compile app.py src/*.py src/extraction/*.py
 .venv/bin/python -m pytest tests/test_report_pipeline.py
 ```
 
-Then verify the Space build uses Gradio, not Docker, and that CUDA hardware selects Transformers while CPU hardware selects llama.cpp.
+Then verify the Space build uses Gradio, not Docker, and that ZeroGPU/CUDA hardware selects Transformers while CPU hardware selects llama.cpp.

requirements.txt

@@ -4,15 +4,14 @@ requests==2.32.5
 pillow==12.0.0
 pymupdf==1.26.6
 json-repair==0.60.1
-# Optional CUDA path for EXTRACTOR_BACKEND=auto: if torch sees CUDA, the app uses the official
-# OpenBMB Transformers pipeline; otherwise it stays on the CPU llama.cpp path below.
+# ZeroGPU/CUDA path for EXTRACTOR_BACKEND=auto: the app uses the official OpenBMB
+# Transformers pipeline when ACCELERATOR is ZeroGPU, ZERO_GPU=TRUE, or CUDA is visible.
 torch==2.9.1 ; sys_platform == "linux" and platform_machine == "x86_64"
 transformers==4.57.3
 accelerate==1.12.0
 bitsandbytes==0.48.2 ; sys_platform == "linux" and platform_machine == "x86_64"
 torchvision==0.24.1 ; sys_platform == "linux" and platform_machine == "x86_64"
 av==16.0.1 ; sys_platform == "linux" and platform_machine == "x86_64"
-# Active HF Space path: run the official OpenBMB GGUF through llama.cpp on ZeroGPU.
-# Install the prebuilt CPU manylinux wheel directly to avoid a source build on Spaces.
+# CPU fallback path: install the prebuilt manylinux wheel directly to avoid a source build on Spaces.
 llama-cpp-python @ https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.28/llama_cpp_python-0.3.28-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl ; sys_platform == "linux" and platform_machine == "x86_64"
 llama-cpp-python==0.3.28 ; sys_platform != "linux"

src/extraction/__init__.py

@@ -1,7 +1,7 @@
 """Extraction backends behind one interface.
 
 `build_extractor()` returns the right backend for the environment:
-  - **auto**: Transformers when CUDA is visible; CPU llama.cpp otherwise.
+  - **auto**: Transformers on ZeroGPU/CUDA; CPU llama.cpp otherwise.
   - **zerogpu** / **transformers**: force official OpenBMB MiniCPM-V through Transformers.
   - **llamacpp-gpu** / **llama-champion**: force GGUF through llama.cpp.
   - **local**: local llama-server / llama.cpp backends for local experimentation.

src/extraction/auto.py

@@ -10,7 +10,7 @@ from src.extraction.zerogpu_transformers import ZeroGPUTransformersExtractor
 
 
 class AutoExtractor:
-    """Use Transformers when CUDA is visible, otherwise fall back to CPU llama.cpp."""
+    """Use Transformers on ZeroGPU/CUDA, otherwise use CPU llama.cpp."""
 
     def __init__(self, model_id: str | None = None) -> None:
         self.model_id = model_id
@@ -34,24 +34,56 @@ class AutoExtractor:
 
     def _backend(self) -> Extractor:
         if self._selected is None:
-            if cuda_available():
+            target = runtime_target()
+            print(f"[Blood Test Explainer] auto extractor selected {target}", flush=True)
+            if target == "transformers":
                 self._selected = ZeroGPUTransformersExtractor(model_id=self.model_id)
             else:
                 self._selected = LlamaCppGPUExtractor()
         return self._selected
 
 
+def runtime_target() -> str:
+    """Return `transformers` for ZeroGPU/CUDA and `llamacpp` for CPU-only runtime."""
+    if zerogpu_runtime_requested() or cuda_available():
+        return "transformers"
+    return "llamacpp"
+
+
+def zerogpu_runtime_requested() -> bool:
+    """Detect HF ZeroGPU from explicit Space/runtime environment flags.
+
+    ZeroGPU exposes CUDA only inside a `@spaces.GPU` worker, so checking
+    `torch.cuda.is_available()` in normal Gradio app code is not enough.
+    """
+    boolean_flags = ("ZERO_GPU", "SPACES_ZERO_GPU", "HF_ZERO_GPU", "BTE_ZERO_GPU")
+    for name in boolean_flags:
+        value = os.getenv(name, "").strip().lower()
+        if value in {"1", "true", "yes", "on", "zerogpu", "zero-gpu"}:
+            return True
+
+    hardware_flags = ("ACCELERATOR", "BTE_RUNTIME", "BTE_HARDWARE", "SPACE_HARDWARE", "HF_SPACE_HARDWARE")
+    for name in hardware_flags:
+        value = os.getenv(name, "").strip().lower()
+        if "zero" in value and "gpu" in value:
+            return True
+        if value.startswith("zero-"):
+            return True
+
+    return False
+
+
 def cuda_available() -> bool:
     try:
         import torch
     except Exception:
         return False
 
-
-def _fallback_enabled() -> bool:
-    return os.getenv("AUTO_FALLBACK_TO_LLAMACPP", "1").strip().lower() not in {"0", "false", "no"}
-
     try:
         return bool(torch.cuda.is_available())
     except Exception:
         return False
+
+
+def _fallback_enabled() -> bool:
+    return os.getenv("AUTO_FALLBACK_TO_LLAMACPP", "0").strip().lower() in {"1", "true", "yes", "on"}

src/extraction/factory.py

@@ -1,7 +1,7 @@
 """Backend selection.
 
 `EXTRACTOR_BACKEND` env:
-  - `auto`: Transformers when CUDA is visible, CPU llama.cpp otherwise.
+  - `auto`: Transformers on ZeroGPU/CUDA, CPU llama.cpp otherwise.
   - `llamacpp-gpu` / `llama-champion`: llama.cpp GGUF badge path.
   - `zerogpu` / `transformers`: official OpenBMB Transformers backend.
   - `api`: hosted OpenBMB endpoint (dev fallback only).