Wire llama.cpp ZeroGPU badge path

Co-authored-by: Codex <chatgpt-codex-connector[bot]@users.noreply.github.com>

DEPLOY.md

@@ -5,12 +5,12 @@ The active Hugging Face deployment is a **Gradio ZeroGPU Space**.
 This workflow is intentionally fixed:
 
 1. The Space must stay a Gradio Space, not a Docker Space.
-2. Runtime extraction must use the official OpenBMB MiniCPM-V Transformers path.
+2. Runtime extraction should use the `llamacpp-gpu` backend when we are targeting the Llama Champion badge.
 3. The extraction call must run behind `@spaces.GPU` so Hugging Face allocates ZeroGPU only while the model is needed.
 4. Model files must not be committed to the Space git repo.
-5. When the fine-tuned model is ready, only replace the model repo path in `ZEROGPU_MODEL_ID`.
+5. When the fine-tuned GGUF model is ready, only replace the `LLAMACPP_*` model variables.
 
-Do not change this architecture unless the project intentionally gives up ZeroGPU. The only intended future model-serving change is inserting the fine-tuned model repository path into `ZEROGPU_MODEL_ID`.
+Do not change this architecture unless the project intentionally gives up ZeroGPU or the llama.cpp backend proves incompatible with ZeroGPU. The intended future model-serving change is inserting the fine-tuned GGUF repository path into the existing `LLAMACPP_*` variables.
 
 ## 1. Space Metadata
 
@@ -34,44 +34,53 @@ ZeroGPU is Gradio-only on Hugging Face. It is not available for Docker Spaces, w
 
 ## 2. Model Serving
 
-The current model is the official OpenBMB Transformers model:
+The badge-target model path is the official OpenBMB GGUF repo running through llama.cpp:
 
 ```text
-openbmb/MiniCPM-V-4.6
+LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
+LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
+LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
+EXTRACTOR_BACKEND=llamacpp-gpu
 ```
 
 The backend lives in:
 
 ```text
-src/extraction/zerogpu_transformers.py
+src/extraction/llamacpp_gpu.py
 ```
 
 It uses:
 
 ```python
-AutoProcessor.from_pretrained(model_id)
-AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
+llama_cpp.Llama(...)
 @spaces.GPU(duration=120)
 ```
 
-The app selects this backend by default through:
+This is a valid hackathon badge option because the submitted app remains a Gradio ZeroGPU Space,
+but the model runtime is `llama.cpp` over GGUF rather than a hosted inference API.
+
+The safe fallback backend is the official OpenBMB Transformers model:
 
 ```text
 EXTRACTOR_BACKEND=zerogpu
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
 ```
 
-or through `auto`, which resolves to the same ZeroGPU backend.
+Use the fallback only if `llama-cpp-python` cannot load MiniCPM-V 4.6 on ZeroGPU.
 
 ## 3. Future Fine-Tuned Model
 
 When the fine-tuned model is ready:
 
-1. Upload the fine-tuned model to a Hugging Face model repo.
-2. Keep the same Gradio + ZeroGPU + Transformers architecture.
-3. Change only this variable:
+1. Convert/quantize the fine-tuned model to GGUF.
+2. Upload the fine-tuned GGUF model and compatible mmproj file to a Hugging Face model repo.
+3. Keep the same Gradio + ZeroGPU + llama.cpp architecture.
+4. Change only these variables:
 
 ```bash
-ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-model>
+LLAMACPP_GGUF_REPO=<owner>/<fine-tuned-minicpm-v-gguf-repo>
+LLAMACPP_MODEL_FILE=<fine-tuned-model>.gguf
+LLAMACPP_MMPROJ_FILE=<compatible-mmproj>.gguf
 ```
 
 Do not add model files to the Space git repo. Do not reintroduce Docker or `llama-server` for the ZeroGPU deployment.
@@ -84,8 +93,8 @@ This architecture keeps:
 
 - Free ZeroGPU eligibility.
 - No external hosted inference API calls.
-- Official OpenBMB model loading through Transformers.
-- A clean future swap to a fine-tuned model by changing only `ZEROGPU_MODEL_ID`.
+- A valid `llama.cpp` / GGUF runtime path for the Llama Champion badge, if `llama-cpp-python` is compatible.
+- A clean future swap to a fine-tuned GGUF model by changing only `LLAMACPP_*` variables.
 
 ## 5. Local Development
 
@@ -93,7 +102,7 @@ Local development can run the same backend, although the model may be slow or to
 
 ```bash
 pip install -r requirements.txt
-EXTRACTOR_BACKEND=zerogpu python app.py
+EXTRACTOR_BACKEND=llamacpp-gpu python app.py
 ```
 
 For quick UI-only work, continue using the static reference report without triggering extraction.

DEPLOYMENT_LOG.md

@@ -33,3 +33,36 @@ ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-model>
 ```
 
 Do not reintroduce Docker or `llama-server` while the project is targeting ZeroGPU. Do not commit model files to the Space repository.
+
+## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
+
+Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
+`llamacpp-gpu` backend.
+
+Why:
+
+- ZeroGPU requires Gradio SDK, so Docker is still not the right deployment surface.
+- The llama.cpp badge can still be targeted from a Gradio Space if inference runs through
+  `llama-cpp-python` over GGUF inside `@spaces.GPU`.
+- The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
+  Hugging Face cache at runtime.
+
+Current badge-target variables:
+
+```bash
+EXTRACTOR_BACKEND=llamacpp-gpu
+LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
+LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
+LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
+```
+
+Fallback if `llama-cpp-python` is incompatible with MiniCPM-V 4.6 on ZeroGPU:
+
+```bash
+EXTRACTOR_BACKEND=zerogpu
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+```
+
+Future fine-tuned model:
+
+Only change the `LLAMACPP_*` variables to point at the fine-tuned GGUF repo/files.

README.md

@@ -48,23 +48,28 @@ The knowledge graph is educational context, not diagnosis. The lab-provided refe
 The Hugging Face Space is intentionally deployed as a **Gradio ZeroGPU Space**. This is the active
 deployment path.
 
-The app uses the official OpenBMB Transformers model path for MiniCPM-V 4.6 and allocates a ZeroGPU
-worker only for the extraction call through `@spaces.GPU`. The deterministic knowledge-graph
+The badge-target backend runs the official OpenBMB MiniCPM-V 4.6 GGUF through `llama.cpp`
+(`llama-cpp-python`) inside a ZeroGPU-managed function. The deterministic knowledge-graph
 enrichment and UI rendering stay in normal Gradio/Python code.
 
 This workflow should not be further changed back to Docker unless the project intentionally gives up
-ZeroGPU. When the fine-tuned model is ready, only replace the model variable:
+ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
 
 ```bash
-ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+EXTRACTOR_BACKEND=llamacpp-gpu
+LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
+LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
+LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 ```
 
-The future fine-tuned deployment should keep the same Gradio + ZeroGPU + Transformers architecture
-and only insert the fine-tuned model repository path into `ZEROGPU_MODEL_ID`.
+The future fine-tuned deployment should keep the same Gradio + ZeroGPU + `llama.cpp` architecture
+and only insert the fine-tuned GGUF repository/path into the `LLAMACPP_*` variables. If
+`llama-cpp-python` proves incompatible with MiniCPM-V 4.6 on ZeroGPU, the fallback backend is
+`EXTRACTOR_BACKEND=zerogpu` with `ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6`.
 
 ## Local Setup
 
 ```bash
 pip install -r requirements.txt
-EXTRACTOR_BACKEND=zerogpu python app.py
+EXTRACTOR_BACKEND=llamacpp-gpu python app.py
 ```

RUNBOOK.md

@@ -10,10 +10,11 @@ This replaced the Docker + `llama-server` path because ZeroGPU is only available
 |---|---|
 | Space SDK | `gradio` |
 | Hardware | ZeroGPU |
-| Model runtime | Official OpenBMB Transformers path |
-| Default backend | `EXTRACTOR_BACKEND=auto`, resolving to `zerogpu` |
-| Model variable | `ZEROGPU_MODEL_ID` |
-| Extraction backend | `src/extraction/zerogpu_transformers.py` |
+| Badge-target runtime | `llama.cpp` through `llama-cpp-python` |
+| Badge-target backend | `EXTRACTOR_BACKEND=llamacpp-gpu` |
+| Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
+| Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
+| Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
 | Report enrichment | `src/report_pipeline.py` + `kb/cbc_knowledge_graph.json` |
 
 Do not switch the Space back to Docker unless the project intentionally gives up ZeroGPU.
@@ -22,8 +23,9 @@ Do not switch the Space back to Docker unless the project intentionally gives up
 
 `EXTRACTOR_BACKEND`:
 
-- `auto`: default, uses ZeroGPU Transformers.
-- `zerogpu`: force the ZeroGPU Transformers backend.
+- `auto`: default, uses ZeroGPU Transformers as the safest startup path.
+- `llamacpp-gpu`: badge-target path, runs GGUF through `llama.cpp` inside `@spaces.GPU`.
+- `zerogpu`: force the ZeroGPU Transformers fallback backend.
 - `api`: hosted OpenBMB endpoint for development fallback only.
 - `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
 
@@ -49,6 +51,7 @@ Install dependencies from `requirements.txt`, including:
 
 ```text
 spaces
+llama-cpp-python
 transformers[torch]>=5.7.0
 torch
 torchvision
@@ -56,28 +59,41 @@ av
 accelerate
 ```
 
-The backend uses `@spaces.GPU(duration=120)` for the model generation call.
+Both ZeroGPU backends use `@spaces.GPU(duration=120)` for the model generation call.
 
 ## Current Model
 
-Current default:
+Badge-target Space variables:
 
 ```bash
-ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+EXTRACTOR_BACKEND=llamacpp-gpu
+LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
+LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
+LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 ```
 
-This is the official OpenBMB model. No model files are committed to the Space repo.
+This is the official OpenBMB GGUF model path. No model files are committed to the Space repo.
+
+Fallback variables if llama.cpp is incompatible on ZeroGPU:
+
+```bash
+EXTRACTOR_BACKEND=zerogpu
+ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
+```
 
 ## Fine-Tuned Model Swap
 
 When the fine-tuned model is ready:
 
-1. Upload it to a Hugging Face model repo.
-2. Keep the same Gradio + ZeroGPU + Transformers architecture.
-3. Change only:
+1. Convert/quantize it to GGUF.
+2. Upload it and the compatible mmproj to a Hugging Face model repo.
+3. Keep the same Gradio + ZeroGPU + llama.cpp architecture.
+4. Change only:
 
 ```bash
-ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-model>
+LLAMACPP_GGUF_REPO=<owner>/<fine-tuned-minicpm-v-gguf-repo>
+LLAMACPP_MODEL_FILE=<fine-tuned-model>.gguf
+LLAMACPP_MMPROJ_FILE=<compatible-mmproj>.gguf
 ```
 
 Do not add model files to the Space git repo. Do not reintroduce Docker or `llama-server` for the ZeroGPU deployment.

requirements.txt

@@ -10,3 +10,8 @@ torch
 torchvision
 av
 transformers[torch]>=5.7.0
+
+# Llama Champion path: run the official OpenBMB GGUF through llama.cpp on ZeroGPU.
+# The extra index provides prebuilt CUDA wheels so the Space does not compile llama.cpp.
+--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
+llama-cpp-python==0.3.16

src/extraction/factory.py

@@ -2,6 +2,7 @@
 
 `EXTRACTOR_BACKEND` env:
   - `auto` / `zerogpu` (default): HF ZeroGPU + official OpenBMB Transformers model.
+  - `llamacpp-gpu` / `llama-champion`: HF ZeroGPU + llama.cpp GGUF badge path.
   - `api`: hosted OpenBMB endpoint (dev fallback only).
   - `local` / `server`: local llama-server backend for local development.
   - `llamacpp`: in-process llama-cpp-python backend for local development.

src/extraction/llamacpp_gpu.py

@@ -7,7 +7,7 @@ the model + vision projector are downloaded from the Hub, all layers are offload
   - 🔌 Off the Grid    (model in the Space, no external inference API)
   - quantized GGUF on GPU (fast, low VRAM)
 
-Requires a llama-cpp-python build with MiniCPM-V 4.6 support (>= 0.3.39) and a CUDA wheel.
+Requires a llama-cpp-python CUDA wheel with the MiniCPM-V chat handler available.
 
 Config (env):
   LLAMACPP_GGUF_REPO    HF repo with the GGUF + mmproj   (default: openbmb/MiniCPM-V-4.6-gguf;
@@ -30,6 +30,7 @@ from src.openbmb_client import (
     EXTRACTION_PROMPT,
     ExtractionResult,
     _normalize_notes,
+    _normalize_patient,
     _normalize_tests,
     _parse_json_response,
 )
@@ -74,6 +75,7 @@ class LlamaCppGPUExtractor:
         )
         parsed = _parse_json_response(raw)
         return ExtractionResult(
+            patient=_normalize_patient(parsed.get("patient", {})),
             tests=_normalize_tests(parsed.get("tests", [])),
             notes=_normalize_notes(parsed.get("notes", [])),
             raw_response=raw,