Select extraction backend by CUDA availability

README.md

@@ -46,28 +46,27 @@ The knowledge graph is educational context, not diagnosis. The lab-provided refe
 
 ## Hugging Face Space Deployment
 
-The Hugging Face Space is intentionally deployed as a **Gradio ZeroGPU Space**. This is the active
-deployment path.
+The Hugging Face Space is intentionally deployed as a **Gradio Space** with adaptive extraction.
+This is the active deployment path.
 
-The badge-target backend runs the official OpenBMB MiniCPM-V 4.6 GGUF through `llama.cpp`
-inside a ZeroGPU-managed function, using a `llama-cpp-python` build that includes MiniCPM-V 4.6
-mtmd support. The deterministic knowledge-graph
-enrichment and UI rendering stay in normal Gradio/Python code.
+With `EXTRACTOR_BACKEND=auto`, the app checks CUDA availability at runtime. If CUDA is visible, it
+uses the official OpenBMB MiniCPM-V 4.6 Transformers path. If CUDA is not visible, it falls back to
+the CPU `llama.cpp` GGUF path. The deterministic knowledge-graph enrichment and UI rendering stay
+in normal Gradio/Python code.
 
 This workflow should not be further changed back to Docker unless the project intentionally gives up
-ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
+ZeroGPU. When the fine-tuned models are ready, only replace the model variables:
 
 ```bash
 EXTRACTOR_BACKEND=auto
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
 LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
 LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
-LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 ```
 
-The future fine-tuned deployment should keep the same Gradio + ZeroGPU + `llama.cpp` architecture
-and only insert the fine-tuned GGUF repository/path into the `LLAMACPP_*` variables. If
-`llama-cpp-python` proves incompatible with MiniCPM-V 4.6 on ZeroGPU, the fallback backend is
-`EXTRACTOR_BACKEND=zerogpu` with `ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6`.
+The future fine-tuned deployment should keep the same Gradio adaptive architecture and only insert
+the fine-tuned Transformers repo into `ZEROGPU_MODEL_ID` and the fine-tuned GGUF repo/path into the
+`LLAMACPP_*` variables.
 
 ## Local Setup
 
@@ -76,15 +75,13 @@ pip install -r requirements.txt
 EXTRACTOR_BACKEND=auto python app.py
 ```
 
-The Space runtime is intentionally slim. The Transformers fallback stack is kept out of the
-default Space dependency set so the build stays faster and more deterministic. The active
-llama.cpp path uses the official prebuilt CPU manylinux wheel for `llama-cpp-python`:
+The Space runtime installs both extraction lanes so `auto` can choose at runtime. The CPU llama.cpp
+path uses the official prebuilt CPU manylinux wheel for `llama-cpp-python`:
 
 ```text
 https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.28/llama_cpp_python-0.3.28-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
 ```
 
 That avoids both the CUDA runtime mismatch that was causing the Space to abort on
-`libcudart.so.12` and the slow source build that was timing out on Hugging Face. The current
-pipeline is PDF/text-only, so the extractor does not load an image encoder or mmproj. Install any
-extra backend-specific packages only when you actually need to run that backend locally.
+`libcudart.so.12` and the slow source build that was timing out on Hugging Face. The CPU fallback is
+PDF/text-only, so the extractor does not load an image encoder or mmproj on CPU.

RUNBOOK.md

@@ -1,6 +1,6 @@
-# Runbook — ZeroGPU Extraction + Fine-Tuned Model Swap
+# Runbook — Adaptive Extraction + Fine-Tuned Model Swap
 
-The active deployment path is now **Gradio ZeroGPU**.
+The active deployment path is now **Gradio with adaptive extraction**.
 
 This replaced the Docker + `llama-server` path because ZeroGPU is only available for Gradio SDK Spaces. The Docker build was also failing on free CPU hardware with `OOMKilled`.
 
@@ -9,12 +9,13 @@ This replaced the Docker + `llama-server` path because ZeroGPU is only available
 | Area | Current choice |
 |---|---|
 | Space SDK | `gradio` |
-| Hardware | ZeroGPU |
-| Badge-target runtime | `llama.cpp` through a `llama-cpp-python` build with MiniCPM-V 4.6 support |
-| Badge-target backend | `EXTRACTOR_BACKEND=auto` or `EXTRACTOR_BACKEND=llamacpp-gpu` |
-| Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
-| Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
-| Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
+| Hardware | Adaptive: CUDA when available, CPU otherwise |
+| Auto backend | Transformers on CUDA, llama.cpp on CPU |
+| Force llama.cpp | `EXTRACTOR_BACKEND=llamacpp-gpu` |
+| Force Transformers | `EXTRACTOR_BACKEND=zerogpu` or `EXTRACTOR_BACKEND=transformers` |
+| llama.cpp variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE` |
+| Transformers variables | `ZEROGPU_MODEL_ID`, `ZEROGPU_MAX_NEW_TOKENS`, `ZEROGPU_QUANTIZE` |
+| Extraction backends | `src/extraction/auto.py`, `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
 | Report enrichment | `src/report_pipeline.py` + `kb/cbc_knowledge_graph.json` |
 
 Do not switch the Space back to Docker unless the project intentionally gives up ZeroGPU.
@@ -23,9 +24,9 @@ Do not switch the Space back to Docker unless the project intentionally gives up
 
 `EXTRACTOR_BACKEND`:
 
-- `auto`: default badge-target path, runs GGUF through `llama.cpp` inside `@spaces.GPU`.
-- `llamacpp-gpu`: explicit alias for the same badge-target path.
-- `zerogpu`: force the ZeroGPU Transformers fallback backend.
+- `auto`: uses the Transformers backend when `torch.cuda.is_available()` is true; otherwise uses the CPU llama.cpp backend.
+- `llamacpp-gpu`: force the GGUF llama.cpp backend.
+- `zerogpu` / `transformers`: force the OpenBMB Transformers backend.
 - `api`: hosted OpenBMB endpoint for development fallback only.
 - `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
 
@@ -51,12 +52,14 @@ Install dependencies from `requirements.txt`, including:
 
 ```text
 spaces
+torch
+transformers
 llama-cpp-python
 ```
 
-The Space build intentionally excludes the heavier Transformers fallback stack from the default
-runtime requirements so the Hugging Face build stays fast. Add backend-specific extras only when
-you are explicitly working on that backend locally.
+The Space installs both runtime lanes so `EXTRACTOR_BACKEND=auto` can choose at runtime. CUDA
+hardware uses the official OpenBMB Transformers path. CPU hardware uses the prebuilt
+`llama-cpp-python` wheel and avoids a source build.
 
 The active llama.cpp path now uses the official prebuilt CPU manylinux wheel for `llama-cpp-python`:
 
@@ -65,11 +68,12 @@ https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.28/llama_cpp_
 ```
 
 This avoids both the CUDA runtime mismatch that was causing the Space to abort on
-`libcudart.so.12` and the slow source build that was timing out on Hugging Face. The current
-pipeline is PDF/text-only, so it keeps the Space on a simpler llama.cpp route without mmproj or
-an image encoder.
+`libcudart.so.12` and the slow source build that was timing out on Hugging Face. The CPU fallback
+is PDF/text-only, so it keeps the Space on a simpler llama.cpp route without mmproj or an image
+encoder.
 
-Both ZeroGPU backends use `@spaces.GPU(duration=120)` for the model generation call.
+The Transformers backend uses `@spaces.GPU(duration=120)` for GPU generation. The llama.cpp CPU
+fallback keeps a longer duration budget because CPU inference is slower.
 
 ## Current Model
 
@@ -79,34 +83,27 @@ Badge-target defaults:
 EXTRACTOR_BACKEND=auto
 LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
 LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
-LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
-```
-
-This is the official OpenBMB GGUF model path. No model files are committed to the Space repo.
-
-Fallback variables if llama.cpp is incompatible on ZeroGPU:
-
-```bash
-EXTRACTOR_BACKEND=zerogpu
 ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
 ```
 
+No model files are committed to the Space repo.
+
 ## Fine-Tuned Model Swap
 
 When the fine-tuned model is ready:
 
-1. Convert/quantize it to GGUF.
-2. Upload it and the compatible mmproj to a Hugging Face model repo.
-3. Keep the same Gradio + ZeroGPU + llama.cpp architecture.
+1. Upload the fine-tuned Transformers checkpoint to a Hugging Face model repo for the CUDA lane.
+2. Convert/quantize it to GGUF for the CPU llama.cpp lane.
+3. Keep the same Gradio adaptive architecture.
 4. Change only:
 
 ```bash
+ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-transformers-repo>
 LLAMACPP_GGUF_REPO=<owner>/<fine-tuned-minicpm-v-gguf-repo>
 LLAMACPP_MODEL_FILE=<fine-tuned-model>.gguf
-LLAMACPP_MMPROJ_FILE=<compatible-mmproj>.gguf
 ```
 
-Do not add model files to the Space git repo. Do not reintroduce Docker or `llama-server` for the ZeroGPU deployment.
+Do not add model files to the Space git repo. Do not reintroduce Docker or `llama-server` for the Space deployment.
 
 ## Local Development
 
@@ -123,7 +120,7 @@ pip install -r requirements.txt
 EXTRACTOR_BACKEND=auto python app.py
 ```
 
-Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF ZeroGPU Space.
+Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF Space.
 
 ## Verification
 
@@ -134,4 +131,4 @@ python3 -m py_compile app.py src/*.py src/extraction/*.py
 .venv/bin/python -m pytest tests/test_report_pipeline.py
 ```
 
-Then verify the Space build uses Gradio, not Docker, and that ZeroGPU can be selected in the Space hardware panel.
+Then verify the Space build uses Gradio, not Docker, and that CUDA hardware selects Transformers while CPU hardware selects llama.cpp.

requirements.txt

@@ -4,6 +4,14 @@ requests==2.32.5
 pillow==12.0.0
 pymupdf==1.26.6
 json-repair==0.60.1
+# Optional CUDA path for EXTRACTOR_BACKEND=auto: if torch sees CUDA, the app uses the official
+# OpenBMB Transformers pipeline; otherwise it stays on the CPU llama.cpp path below.
+torch==2.9.1 ; sys_platform == "linux" and platform_machine == "x86_64"
+transformers==4.57.3
+accelerate==1.12.0
+bitsandbytes==0.48.2 ; sys_platform == "linux" and platform_machine == "x86_64"
+torchvision==0.24.1 ; sys_platform == "linux" and platform_machine == "x86_64"
+av==16.0.1 ; sys_platform == "linux" and platform_machine == "x86_64"
 # Active HF Space path: run the official OpenBMB GGUF through llama.cpp on ZeroGPU.
 # Install the prebuilt CPU manylinux wheel directly to avoid a source build on Spaces.
 llama-cpp-python @ https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.28/llama_cpp_python-0.3.28-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl ; sys_platform == "linux" and platform_machine == "x86_64"

src/extraction/__init__.py

@@ -1,11 +1,13 @@
 """Extraction backends behind one interface.
 
 `build_extractor()` returns the right backend for the environment:
-  - **zerogpu** / **auto**: official OpenBMB MiniCPM-V through Transformers on HF ZeroGPU.
+  - **auto**: Transformers when CUDA is visible; CPU llama.cpp otherwise.
+  - **zerogpu** / **transformers**: force official OpenBMB MiniCPM-V through Transformers.
+  - **llamacpp-gpu** / **llama-champion**: force GGUF through llama.cpp.
   - **local**: local llama-server / llama.cpp backends for local experimentation.
   - **api**: the original OpenBMB hosted endpoint, kept as a dev fallback only.
 
-Default is `auto`, which resolves to the ZeroGPU Transformers backend for the Space.
+Default is `auto`.
 """
 
 from src.extraction.base import Extractor, ExtractionResult

src/extraction/auto.py

@@ -0,0 +1,38 @@
+"""Adaptive extraction backend selection."""
+
+from __future__ import annotations
+
+from src.extraction.base import Extractor
+from src.extraction.llamacpp_gpu import LlamaCppGPUExtractor
+from src.extraction.zerogpu_transformers import ZeroGPUTransformersExtractor
+
+
+class AutoExtractor:
+    """Use Transformers when CUDA is visible, otherwise fall back to CPU llama.cpp."""
+
+    def __init__(self, model_id: str | None = None) -> None:
+        self.model_id = model_id
+        self._selected: Extractor | None = None
+
+    def extract(self, file_path: str, max_pages: int = 3):
+        return self._backend().extract(file_path, max_pages=max_pages)
+
+    def _backend(self) -> Extractor:
+        if self._selected is None:
+            if cuda_available():
+                self._selected = ZeroGPUTransformersExtractor(model_id=self.model_id)
+            else:
+                self._selected = LlamaCppGPUExtractor()
+        return self._selected
+
+
+def cuda_available() -> bool:
+    try:
+        import torch
+    except Exception:
+        return False
+
+    try:
+        return bool(torch.cuda.is_available())
+    except Exception:
+        return False

src/extraction/factory.py

@@ -1,8 +1,9 @@
 """Backend selection.
 
 `EXTRACTOR_BACKEND` env:
-  - `auto` / `llamacpp-gpu` / `llama-champion`: HF ZeroGPU + llama.cpp GGUF badge path.
-  - `zerogpu` / `transformers`: HF ZeroGPU + official OpenBMB Transformers fallback.
+  - `auto`: Transformers when CUDA is visible, CPU llama.cpp otherwise.
+  - `llamacpp-gpu` / `llama-champion`: llama.cpp GGUF badge path.
+  - `zerogpu` / `transformers`: official OpenBMB Transformers backend.
   - `api`: hosted OpenBMB endpoint (dev fallback only).
   - `local` / `server`: local llama-server backend for local development.
   - `llamacpp`: in-process llama-cpp-python backend for local development.
@@ -13,6 +14,7 @@ from __future__ import annotations
 import os
 
 from src.extraction.base import Extractor
+from src.extraction.auto import AutoExtractor
 from src.extraction.llamacpp_gpu import LlamaCppGPUExtractor
 from src.extraction.local_minicpmv import LocalMiniCPMVExtractor
 from src.extraction.local_server import LocalServerExtractor
@@ -27,8 +29,9 @@ def build_extractor(
 ) -> Extractor:
     backend = os.getenv("EXTRACTOR_BACKEND", "auto").strip().lower()
 
-    if backend in ("auto", "llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
-        # llama.cpp on the ZeroGPU GPU -> earns the Llama Champion badge while staying off-grid.
+    if backend == "auto":
+        return AutoExtractor(model_id=model)
+    if backend in ("llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
         return LlamaCppGPUExtractor()
     if backend in ("zerogpu", "zero-gpu", "transformers"):
         return ZeroGPUTransformersExtractor(model_id=model)