r0mant1c Codex commited on
Commit
7b07a5c
·
1 Parent(s): f9af049

Make llama.cpp ZeroGPU backend default

Browse files

Co-authored-by: Codex <chatgpt-codex-connector[bot]@users.noreply.github.com>

Files changed (5) hide show
  1. DEPLOY.md +2 -2
  2. DEPLOYMENT_LOG.md +4 -4
  3. README.md +2 -2
  4. RUNBOOK.md +6 -6
  5. src/extraction/factory.py +5 -5
DEPLOY.md CHANGED
@@ -40,7 +40,7 @@ The badge-target model path is the official OpenBMB GGUF repo running through ll
40
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
41
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
42
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
43
- EXTRACTOR_BACKEND=llamacpp-gpu
44
  ```
45
 
46
  The backend lives in:
@@ -102,7 +102,7 @@ Local development can run the same backend, although the model may be slow or to
102
 
103
  ```bash
104
  pip install -r requirements.txt
105
- EXTRACTOR_BACKEND=llamacpp-gpu python app.py
106
  ```
107
 
108
  For quick UI-only work, continue using the static reference report without triggering extraction.
 
40
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
41
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
42
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
43
+ EXTRACTOR_BACKEND=auto
44
  ```
45
 
46
  The backend lives in:
 
102
 
103
  ```bash
104
  pip install -r requirements.txt
105
+ EXTRACTOR_BACKEND=auto python app.py
106
  ```
107
 
108
  For quick UI-only work, continue using the static reference report without triggering extraction.
DEPLOYMENT_LOG.md CHANGED
@@ -15,7 +15,7 @@ What changed:
15
  - `README.md` metadata changed from `sdk: docker` to `sdk: gradio`.
16
  - `DEPLOY.md` and `RUNBOOK.md` now describe Gradio + ZeroGPU + Transformers as the active path.
17
  - `src/extraction/zerogpu_transformers.py` adds the official OpenBMB MiniCPM-V Transformers backend.
18
- - `src/extraction/factory.py` now resolves `auto` to the ZeroGPU backend.
19
  - Docker-only files (`Dockerfile`, `start.sh`, `.dockerignore`) were removed from the active deployment.
20
 
21
  Current model:
@@ -37,7 +37,7 @@ Do not reintroduce Docker or `llama-server` while the project is targeting ZeroG
37
  ## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
38
 
39
  Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
40
- `llamacpp-gpu` backend.
41
 
42
  Why:
43
 
@@ -47,10 +47,10 @@ Why:
47
  - The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
48
  Hugging Face cache at runtime.
49
 
50
- Current badge-target variables:
51
 
52
  ```bash
53
- EXTRACTOR_BACKEND=llamacpp-gpu
54
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
55
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
56
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 
15
  - `README.md` metadata changed from `sdk: docker` to `sdk: gradio`.
16
  - `DEPLOY.md` and `RUNBOOK.md` now describe Gradio + ZeroGPU + Transformers as the active path.
17
  - `src/extraction/zerogpu_transformers.py` adds the official OpenBMB MiniCPM-V Transformers backend.
18
+ - `src/extraction/factory.py` initially resolved `auto` to the ZeroGPU Transformers backend.
19
  - Docker-only files (`Dockerfile`, `start.sh`, `.dockerignore`) were removed from the active deployment.
20
 
21
  Current model:
 
37
  ## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
38
 
39
  Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
40
+ `auto` / `llamacpp-gpu` backend.
41
 
42
  Why:
43
 
 
47
  - The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
48
  Hugging Face cache at runtime.
49
 
50
+ Current badge-target defaults:
51
 
52
  ```bash
53
+ EXTRACTOR_BACKEND=auto
54
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
55
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
56
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
README.md CHANGED
@@ -56,7 +56,7 @@ This workflow should not be further changed back to Docker unless the project in
56
  ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
57
 
58
  ```bash
59
- EXTRACTOR_BACKEND=llamacpp-gpu
60
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
61
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
62
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
@@ -71,5 +71,5 @@ and only insert the fine-tuned GGUF repository/path into the `LLAMACPP_*` variab
71
 
72
  ```bash
73
  pip install -r requirements.txt
74
- EXTRACTOR_BACKEND=llamacpp-gpu python app.py
75
  ```
 
56
  ZeroGPU. When the fine-tuned GGUF model is ready, only replace the model variables:
57
 
58
  ```bash
59
+ EXTRACTOR_BACKEND=auto
60
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
61
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
62
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 
71
 
72
  ```bash
73
  pip install -r requirements.txt
74
+ EXTRACTOR_BACKEND=auto python app.py
75
  ```
RUNBOOK.md CHANGED
@@ -11,7 +11,7 @@ This replaced the Docker + `llama-server` path because ZeroGPU is only available
11
  | Space SDK | `gradio` |
12
  | Hardware | ZeroGPU |
13
  | Badge-target runtime | `llama.cpp` through `llama-cpp-python` |
14
- | Badge-target backend | `EXTRACTOR_BACKEND=llamacpp-gpu` |
15
  | Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
16
  | Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
17
  | Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
@@ -23,8 +23,8 @@ Do not switch the Space back to Docker unless the project intentionally gives up
23
 
24
  `EXTRACTOR_BACKEND`:
25
 
26
- - `auto`: default, uses ZeroGPU Transformers as the safest startup path.
27
- - `llamacpp-gpu`: badge-target path, runs GGUF through `llama.cpp` inside `@spaces.GPU`.
28
  - `zerogpu`: force the ZeroGPU Transformers fallback backend.
29
  - `api`: hosted OpenBMB endpoint for development fallback only.
30
  - `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
@@ -63,10 +63,10 @@ Both ZeroGPU backends use `@spaces.GPU(duration=120)` for the model generation c
63
 
64
  ## Current Model
65
 
66
- Badge-target Space variables:
67
 
68
  ```bash
69
- EXTRACTOR_BACKEND=llamacpp-gpu
70
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
71
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
72
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
@@ -110,7 +110,7 @@ For local extraction testing with the same backend:
110
 
111
  ```bash
112
  pip install -r requirements.txt
113
- EXTRACTOR_BACKEND=zerogpu python app.py
114
  ```
115
 
116
  Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF ZeroGPU Space.
 
11
  | Space SDK | `gradio` |
12
  | Hardware | ZeroGPU |
13
  | Badge-target runtime | `llama.cpp` through `llama-cpp-python` |
14
+ | Badge-target backend | `EXTRACTOR_BACKEND=auto` or `EXTRACTOR_BACKEND=llamacpp-gpu` |
15
  | Fallback backend | `EXTRACTOR_BACKEND=zerogpu` with Transformers |
16
  | Model variables | `LLAMACPP_GGUF_REPO`, `LLAMACPP_MODEL_FILE`, `LLAMACPP_MMPROJ_FILE` |
17
  | Extraction backends | `src/extraction/llamacpp_gpu.py`, `src/extraction/zerogpu_transformers.py` |
 
23
 
24
  `EXTRACTOR_BACKEND`:
25
 
26
+ - `auto`: default badge-target path, runs GGUF through `llama.cpp` inside `@spaces.GPU`.
27
+ - `llamacpp-gpu`: explicit alias for the same badge-target path.
28
  - `zerogpu`: force the ZeroGPU Transformers fallback backend.
29
  - `api`: hosted OpenBMB endpoint for development fallback only.
30
  - `local` / `server` / `llamacpp`: local experimental backends, not the active HF Space path.
 
63
 
64
  ## Current Model
65
 
66
+ Badge-target defaults:
67
 
68
  ```bash
69
+ EXTRACTOR_BACKEND=auto
70
  LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
71
  LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
72
  LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
 
110
 
111
  ```bash
112
  pip install -r requirements.txt
113
+ EXTRACTOR_BACKEND=auto python app.py
114
  ```
115
 
116
  Local machines without a suitable GPU may be slow or may not have enough memory for full model inference. In that case, test UI/report rendering locally and test extraction on the HF ZeroGPU Space.
src/extraction/factory.py CHANGED
@@ -1,8 +1,8 @@
1
  """Backend selection.
2
 
3
  `EXTRACTOR_BACKEND` env:
4
- - `auto` / `zerogpu` (default): HF ZeroGPU + official OpenBMB Transformers model.
5
- - `llamacpp-gpu` / `llama-champion`: HF ZeroGPU + llama.cpp GGUF badge path.
6
  - `api`: hosted OpenBMB endpoint (dev fallback only).
7
  - `local` / `server`: local llama-server backend for local development.
8
  - `llamacpp`: in-process llama-cpp-python backend for local development.
@@ -27,11 +27,11 @@ def build_extractor(
27
  ) -> Extractor:
28
  backend = os.getenv("EXTRACTOR_BACKEND", "auto").strip().lower()
29
 
30
- if backend in ("auto", "zerogpu", "zero-gpu", "transformers"):
31
- return ZeroGPUTransformersExtractor(model_id=model)
32
- if backend in ("llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
33
  # llama.cpp on the ZeroGPU GPU -> earns the Llama Champion badge while staying off-grid.
34
  return LlamaCppGPUExtractor()
 
 
35
  if backend == "api":
36
  return OpenBMBExtractor(api_url=api_url, model=model, api_key=api_key)
37
  if backend in ("local", "server", "local-server"):
 
1
  """Backend selection.
2
 
3
  `EXTRACTOR_BACKEND` env:
4
+ - `auto` / `llamacpp-gpu` / `llama-champion`: HF ZeroGPU + llama.cpp GGUF badge path.
5
+ - `zerogpu` / `transformers`: HF ZeroGPU + official OpenBMB Transformers fallback.
6
  - `api`: hosted OpenBMB endpoint (dev fallback only).
7
  - `local` / `server`: local llama-server backend for local development.
8
  - `llamacpp`: in-process llama-cpp-python backend for local development.
 
27
  ) -> Extractor:
28
  backend = os.getenv("EXTRACTOR_BACKEND", "auto").strip().lower()
29
 
30
+ if backend in ("auto", "llamacpp-gpu", "gpu-llamacpp", "llama-champion"):
 
 
31
  # llama.cpp on the ZeroGPU GPU -> earns the Llama Champion badge while staying off-grid.
32
  return LlamaCppGPUExtractor()
33
+ if backend in ("zerogpu", "zero-gpu", "transformers"):
34
+ return ZeroGPUTransformersExtractor(model_id=model)
35
  if backend == "api":
36
  return OpenBMBExtractor(api_url=api_url, model=model, api_key=api_key)
37
  if backend in ("local", "server", "local-server"):