Make llama.cpp ZeroGPU backend default
Browse filesCo-authored-by: Codex <chatgpt-codex-connector[bot]@users.noreply.github.com>
- DEPLOY.md +2 -2
- DEPLOYMENT_LOG.md +4 -4
- README.md +2 -2
- RUNBOOK.md +6 -6
- src/extraction/factory.py +5 -5
DEPLOY.md
CHANGED
|
@@ -40,7 +40,7 @@ The badge-target model path is the official OpenBMB GGUF repo running through ll
|
|
| 40 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 41 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 42 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
| 43 |
-
EXTRACTOR_BACKEND=
|
| 44 |
```
|
| 45 |
|
| 46 |
The backend lives in:
|
|
@@ -102,7 +102,7 @@ Local development can run the same backend, although the model may be slow or to
|
|
| 102 |
|
| 103 |
```bash
|
| 104 |
pip install -r requirements.txt
|
| 105 |
-
EXTRACTOR_BACKEND=
|
| 106 |
```
|
| 107 |
|
| 108 |
For quick UI-only work, continue using the static reference report without triggering extraction.
|
|
|
|
| 40 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 41 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 42 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
| 43 |
+
EXTRACTOR_BACKEND=auto
|
| 44 |
```
|
| 45 |
|
| 46 |
The backend lives in:
|
|
|
|
| 102 |
|
| 103 |
```bash
|
| 104 |
pip install -r requirements.txt
|
| 105 |
+
EXTRACTOR_BACKEND=auto python app.py
|
| 106 |
```
|
| 107 |
|
| 108 |
For quick UI-only work, continue using the static reference report without triggering extraction.
|
DEPLOYMENT_LOG.md
CHANGED
|
@@ -15,7 +15,7 @@ What changed:
|
|
| 15 |
- `README.md` metadata changed from `sdk: docker` to `sdk: gradio`.
|
| 16 |
- `DEPLOY.md` and `RUNBOOK.md` now describe Gradio + ZeroGPU + Transformers as the active path.
|
| 17 |
- `src/extraction/zerogpu_transformers.py` adds the official OpenBMB MiniCPM-V Transformers backend.
|
| 18 |
-
- `src/extraction/factory.py`
|
| 19 |
- Docker-only files (`Dockerfile`, `start.sh`, `.dockerignore`) were removed from the active deployment.
|
| 20 |
|
| 21 |
Current model:
|
|
@@ -37,7 +37,7 @@ Do not reintroduce Docker or `llama-server` while the project is targeting ZeroG
|
|
| 37 |
## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
|
| 38 |
|
| 39 |
Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
|
| 40 |
-
`llamacpp-gpu` backend.
|
| 41 |
|
| 42 |
Why:
|
| 43 |
|
|
@@ -47,10 +47,10 @@ Why:
|
|
| 47 |
- The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
|
| 48 |
Hugging Face cache at runtime.
|
| 49 |
|
| 50 |
-
Current badge-target
|
| 51 |
|
| 52 |
```bash
|
| 53 |
-
EXTRACTOR_BACKEND=
|
| 54 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 55 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 56 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
|
|
|
| 15 |
- `README.md` metadata changed from `sdk: docker` to `sdk: gradio`.
|
| 16 |
- `DEPLOY.md` and `RUNBOOK.md` now describe Gradio + ZeroGPU + Transformers as the active path.
|
| 17 |
- `src/extraction/zerogpu_transformers.py` adds the official OpenBMB MiniCPM-V Transformers backend.
|
| 18 |
+
- `src/extraction/factory.py` initially resolved `auto` to the ZeroGPU Transformers backend.
|
| 19 |
- Docker-only files (`Dockerfile`, `start.sh`, `.dockerignore`) were removed from the active deployment.
|
| 20 |
|
| 21 |
Current model:
|
|
|
|
| 37 |
## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
|
| 38 |
|
| 39 |
Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
|
| 40 |
+
`auto` / `llamacpp-gpu` backend.
|
| 41 |
|
| 42 |
Why:
|
| 43 |
|
|
|
|
| 47 |
- The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
|
| 48 |
Hugging Face cache at runtime.
|
| 49 |
|
| 50 |
+
Current badge-target defaults:
|
| 51 |
|
| 52 |
```bash
|
| 53 |
+
EXTRACTOR_BACKEND=auto
|
| 54 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 55 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 56 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
README.md
CHANGED
|
@@ -56,7 +56,7 @@ This workflow should not be further changed back to Docker unless the project in
|
|
| 56 |
ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
|
| 57 |
|
| 58 |
```bash
|
| 59 |
-
EXTRACTOR_BACKEND=
|
| 60 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 61 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 62 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
|
@@ -71,5 +71,5 @@ and only insert the fine-tuned GGUF repository/path into the `LLAMACPP_*` variab
|
|
| 71 |
|
| 72 |
```bash
|
| 73 |
pip install -r requirements.txt
|
| 74 |
-
EXTRACTOR_BACKEND=
|
| 75 |
```
|
|
|
|
| 56 |
ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
|
| 57 |
|
| 58 |
```bash
|
| 59 |
+
EXTRACTOR_BACKEND=auto
|
| 60 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 61 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 62 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
|
|
|
| 71 |
|
| 72 |
```bash
|
| 73 |
pip install -r requirements.txt
|
| 74 |
+
EXTRACTOR_BACKEND=auto python app.py
|
| 75 |
```
|
RUNBOOK.md
CHANGED
|
@@ -11,7 +11,7 @@ This replaced the Docker + `llama-server` path because ZeroGPU is only available
|
|
| 11 |
| Space SDK | `gradio` |
|
| 12 |
| Hardware | ZeroGPU |
|
| 13 |
| Badge-target runtime | `llama.cpp` through `llama-cpp-python` |
|
| 14 |
-
| Badge-target backend | `EXTRACTOR_BACKEND=llamacpp-gpu` |
|
| 15 |
| Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
|
| 16 |
| Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
|
| 17 |
| Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
|
|
@@ -23,8 +23,8 @@ Do not switch the Space back to Docker unless the project intentionally gives up
|
|
| 23 |
|
| 24 |
`EXTRACTOR_BACKEND`:
|
| 25 |
|
| 26 |
-
- `auto`: default
|
| 27 |
-
- `llamacpp-gpu`:
|
| 28 |
- `zerogpu`: force the ZeroGPU Transformers fallback backend.
|
| 29 |
- `api`: hosted OpenBMB endpoint for development fallback only.
|
| 30 |
- `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
|
|
@@ -63,10 +63,10 @@ Both ZeroGPU backends use `@spaces.GPU(duration=120)` for the model generation c
|
|
| 63 |
|
| 64 |
## Current Model
|
| 65 |
|
| 66 |
-
Badge-target
|
| 67 |
|
| 68 |
```bash
|
| 69 |
-
EXTRACTOR_BACKEND=
|
| 70 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 71 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 72 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
|
@@ -110,7 +110,7 @@ For local extraction testing with the same backend:
|
|
| 110 |
|
| 111 |
```bash
|
| 112 |
pip install -r requirements.txt
|
| 113 |
-
EXTRACTOR_BACKEND=
|
| 114 |
```
|
| 115 |
|
| 116 |
Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF ZeroGPU Space.
|
|
|
|
| 11 |
| Space SDK | `gradio` |
|
| 12 |
| Hardware | ZeroGPU |
|
| 13 |
| Badge-target runtime | `llama.cpp` through `llama-cpp-python` |
|
| 14 |
+
| Badge-target backend | `EXTRACTOR_BACKEND=auto` or `EXTRACTOR_BACKEND=llamacpp-gpu` |
|
| 15 |
| Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
|
| 16 |
| Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
|
| 17 |
| Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
|
|
|
|
| 23 |
|
| 24 |
`EXTRACTOR_BACKEND`:
|
| 25 |
|
| 26 |
+
- `auto`: default badge-target path, runs GGUF through `llama.cpp` inside `@spaces.GPU`.
|
| 27 |
+
- `llamacpp-gpu`: explicit alias for the same badge-target path.
|
| 28 |
- `zerogpu`: force the ZeroGPU Transformers fallback backend.
|
| 29 |
- `api`: hosted OpenBMB endpoint for development fallback only.
|
| 30 |
- `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
|
|
|
|
| 63 |
|
| 64 |
## Current Model
|
| 65 |
|
| 66 |
+
Badge-target defaults:
|
| 67 |
|
| 68 |
```bash
|
| 69 |
+
EXTRACTOR_BACKEND=auto
|
| 70 |
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 71 |
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 72 |
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
|
|
|
| 110 |
|
| 111 |
```bash
|
| 112 |
pip install -r requirements.txt
|
| 113 |
+
EXTRACTOR_BACKEND=auto python app.py
|
| 114 |
```
|
| 115 |
|
| 116 |
Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF ZeroGPU Space.
|
src/extraction/factory.py
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
"""Backend selection.
|
| 2 |
|
| 3 |
`EXTRACTOR_BACKEND` env:
|
| 4 |
-
- `auto` / `
|
| 5 |
-
- `
|
| 6 |
- `api`: hosted OpenBMB endpoint (dev fallback only).
|
| 7 |
- `local` / `server`: local llama-server backend for local development.
|
| 8 |
- `llamacpp`: in-process llama-cpp-python backend for local development.
|
|
@@ -27,11 +27,11 @@ def build_extractor(
|
|
| 27 |
) -> Extractor:
|
| 28 |
backend = os.getenv("EXTRACTOR_BACKEND", "auto").strip().lower()
|
| 29 |
|
| 30 |
-
if backend in ("auto", "
|
| 31 |
-
return ZeroGPUTransformersExtractor(model_id=model)
|
| 32 |
-
if backend in ("llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
|
| 33 |
# llama.cpp on the ZeroGPU GPU -> earns the Llama Champion badge while staying off-grid.
|
| 34 |
return LlamaCppGPUExtractor()
|
|
|
|
|
|
|
| 35 |
if backend == "api":
|
| 36 |
return OpenBMBExtractor(api_url=api_url, model=model, api_key=api_key)
|
| 37 |
if backend in ("local", "server", "local-server"):
|
|
|
|
| 1 |
"""Backend selection.
|
| 2 |
|
| 3 |
`EXTRACTOR_BACKEND` env:
|
| 4 |
+
- `auto` / `llamacpp-gpu` / `llama-champion`: HF ZeroGPU + llama.cpp GGUF badge path.
|
| 5 |
+
- `zerogpu` / `transformers`: HF ZeroGPU + official OpenBMB Transformers fallback.
|
| 6 |
- `api`: hosted OpenBMB endpoint (dev fallback only).
|
| 7 |
- `local` / `server`: local llama-server backend for local development.
|
| 8 |
- `llamacpp`: in-process llama-cpp-python backend for local development.
|
|
|
|
| 27 |
) -> Extractor:
|
| 28 |
backend = os.getenv("EXTRACTOR_BACKEND", "auto").strip().lower()
|
| 29 |
|
| 30 |
+
if backend in ("auto", "llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
|
|
|
|
|
|
|
| 31 |
# llama.cpp on the ZeroGPU GPU -> earns the Llama Champion badge while staying off-grid.
|
| 32 |
return LlamaCppGPUExtractor()
|
| 33 |
+
if backend in ("zerogpu", "zero-gpu", "transformers"):
|
| 34 |
+
return ZeroGPUTransformersExtractor(model_id=model)
|
| 35 |
if backend == "api":
|
| 36 |
return OpenBMBExtractor(api_url=api_url, model=model, api_key=api_key)
|
| 37 |
if backend in ("local", "server", "local-server"):
|