DEPLOY.md

Deploying the Space

The active Hugging Face deployment is a Gradio Space with hardware-aware extraction:

• CPU Basic: llama.cpp with the base MiniCPM-V GGUF model.
• ZeroGPU / GPU: Transformers vision with the fine-tuned MiniCPM-V checkpoint.

This workflow is intentionally fixed:

1. The Space must stay a Gradio Space, not a Docker Space.
2. Runtime extraction should use EXTRACTOR_BACKEND=auto unless a lane is being forced for testing.
3. ZeroGPU extraction calls must run behind @spaces.GPU; CPU Basic llama.cpp calls must not require ZeroGPU.
4. Model files must not be committed to the Space git repo.
5. When the fine-tuned model is ready, replace only the model variables for the active lanes.
6. The llama.cpp lane is automatic on CPU Basic and can still be enabled explicitly with environment variables.

Do not change this architecture unless the project intentionally gives up hardware-aware deployment. To swap models, change ZEROGPU_MODEL_ID (or DEFAULT_HF_REPO in src/model_paths.py) and optional LLAMACPP_* for the llama.cpp lane.

1. Space Metadata

The Space build-small-hackathon/blood-test-explainer must use sdk: gradio so ZeroGPU is available. The top of README.md must stay:

---
title: Blood Test Explainer
emoji: 📊
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.17.3
python_version: "3.10.13"
app_file: app.py
pinned: false
---

ZeroGPU is Gradio-only on Hugging Face. It is not available for Docker Spaces, which is why the previous Docker + llama-server deployment was replaced.

2. Default Model Serving

Leave EXTRACTOR_BACKEND unset or set it to:

EXTRACTOR_BACKEND=auto

On CPU Basic, auto detects cpu-basic and selects llama.cpp vision with the base GGUF model:

LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf

On ZeroGPU/GPU, auto selects the fine-tuned Transformers repo:

ZEROGPU_MODEL_ID=build-small-hackathon/blood-test-minicpmv-4_6-medreason

Hub: build-small-hackathon/blood-test-minicpmv-4_6-medreason

ZEROGPU_MODEL_ID is optional when it matches the code default in src/model_paths.py. Use openbmb/MiniCPM-V-4.6 only for base-model baselines.

The backend lives in:

src/extraction/zerogpu_transformers.py

It uses:

@spaces.GPU(duration=120)
transformers.AutoModelForImageTextToText

This is the correct runtime for PDF/image blood-test uploads on ZeroGPU because the GPU is allocated only inside the decorated worker.

Aliases zerogpu and zero-gpu force the Transformers path in src/extraction/factory.py; auto is hardware-aware.

3. llama.cpp Lane

The app ships a second extraction lane for CPU Basic, hackathon badges, and GGUF deployment experiments. It is automatic on CPU Basic.

Why it exists

• Llama Champion badge — inference through llama-cpp-python over GGUF inside @spaces.GPU.
• Fine-tuned GGUF swap — deploy a quantized model without changing the Gradio app structure.
• Text-only fallback — lighter lane for plain-text lab exports when vision is not needed.

For normal CPU Basic PDF/image uploads, keep EXTRACTOR_BACKEND=auto and let the code select llama.cpp vision.

Force vision llama.cpp on the Space

Set these variables in the Space settings:

EXTRACTOR_BACKEND=llamacpp-gpu
LLAMACPP_VISION=1
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
LLAMACPP_CHAT_HANDLER=MiniCPMv26ChatHandler

Implementation: src/extraction/llamacpp_gpu.py with shared vision loading in src/extraction/llamacpp_vision.py.

Without LLAMACPP_VISION=1, the llama.cpp lane accepts .txt / .csv only and rejects PDF/image uploads.

Enable text-only llama.cpp

EXTRACTOR_BACKEND=llamacpp-gpu
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf

4. Swapping or Retraining the Model

To publish a newer fine-tune:

1. Upload the Transformers checkpoint to a Hugging Face model repo.
2. Optionally convert/quantize to GGUF (+ mmproj) for the llama.cpp lane.
3. Keep the same Gradio + ZeroGPU architecture.
4. Point extraction at the new repo:

ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-transformers-repo>
LLAMACPP_GGUF_REPO=<owner>/<fine-tuned-minicpm-v-gguf-repo>
LLAMACPP_MODEL_FILE=<fine-tuned-model>.gguf
LLAMACPP_MMPROJ_FILE=<mmproj-file>.gguf

Or update DEFAULT_HF_REPO in src/model_paths.py so local runs and the Space pick it up without an env override.

Do not add model files to the Space git repo. Do not reintroduce Docker or llama-server for the ZeroGPU deployment.

5. Why This Architecture

The Docker path failed on free CPU hardware with OOMKilled during build. ZeroGPU is only available for Gradio SDK Spaces, so the deployment must be a Gradio Space to use the free dynamic GPU resource.

This architecture keeps:

• Free ZeroGPU eligibility.
• A CPU Basic fallback that uses llama.cpp + base GGUF instead of Transformers.
• No external hosted inference API calls.
• The fine-tuned Transformers runtime on ZeroGPU for PDF/image lab reports.
• A llama.cpp / GGUF lane for CPU Basic, badges, and fine-tuned GGUF deployment.
• A clean model swap by changing ZEROGPU_MODEL_ID / DEFAULT_HF_REPO and optional LLAMACPP_* variables.

6. Local Development

Default (Transformers vision):

pip install -r requirements.txt
python app.py

Optional llama.cpp vision:

pip install -r requirements.txt
EXTRACTOR_BACKEND=llamacpp-gpu LLAMACPP_VISION=1 python app.py

For quick UI-only work, continue using the static reference report without triggering extraction.