docs/HOSTING.md

Hosted evidence workbench

The hosted workbench opens the prebuilt evidence bank in demos/. Visitors can browse the slide, regions, anchors, and patch captions without model inference. On ZeroGPU, questions run through the shared question policy, agent router with rule fallback, compact evidence retrieval, and final answer generation.

The hosted model is Qwen2.5-VL-3B-Instruct. It is a demonstration backbone, separate from the paper's pathology models and from the models used to construct the stored observations. Hosted answers are generated when a visitor submits a question; the viewer does not replay saved answers.

CPU preview

python -m pip install -e '.[ui]'
python space_app.py

Open http://127.0.0.1:7860. The launcher binds to 0.0.0.0 for hosted use. On ordinary CPU hardware it starts the evidence browser and disables live QA. It does not load a language model or send a request to an inference provider.

The hosted interface uses the bundled slides. Upload, model/API configuration, and rebuilding controls remain available in the local python app.py workbench.

Hugging Face Space

Prepare the deployment directory:

python scripts/stage_space.py --out-dir build/hf-space

The staging command packages the code and demo assets, using hf_space/README.md as the Space card and hf_space/requirements.txt as its root requirements file. The repository's main README and requirements stay unchanged. The Space requirements list dependencies directly because the Gradio builder installs them before copying application source; the app imports the bundled Python package from its working directory. Upload the staged directory to the intended Space:

hf upload gexinh/SlideBank build/hf-space --repo-type space

For a fork, replace gexinh/SlideBank with its Space repository ID. Select ZeroGPU in the Space hardware settings for live QA. The launcher checks the platform's SPACES_ZERO_GPU flag and the Spaces SDK; it keeps QA disabled on cpu-basic even though the GPU decorator alone would be a no-op there.

The Space uses Python 3.10, Gradio 5.34.2, PyTorch 2.8.0, and Transformers 4.51.3. The standalone Space requirements and the hosted package extra declare the same model/runtime dependencies. Quilt-LLaVA's external legacy runtime is not needed by this hosted model.

Model preparation and GPU execution

The Space card's preload_from_hub entry caches the Qwen checkpoint during the Space build. Startup then explicitly prepares the model using the Spaces SDK's CUDA emulation, before starting the web interface. Importing the launcher does not prepare a model; CPU-only startup skips this step entirely.

The complete question-answering call runs inside spaces.GPU(duration=60). This includes routing and final answer generation, so both calls use the same allocated GPU and cached client. The workbench queues one request at a time, with up to eight queued requests. GPU availability and daily quota are managed by Hugging Face. An allocation, quota, or inference error leaves evidence browsing available and asks the visitor to retry later.

The SDK can scale this reservation for its GPU profile; a 60-second duration currently reserves 90 seconds on the Blackwell large profile. A request can be declined while some allowance remains if that allowance is below the reserved duration. The workbench explains this distinction and suggests signing in to Hugging Face or returning after the allowance resets.

No visitor API key or paid inference endpoint is used. There is no automatic fallback to an API or to multi-billion-parameter CPU inference.

The runtime follows the ZeroGPU SDK documentation. The Space's hardware selection and build status determine whether live QA is available; a successful CPU preview checks browsing, not GPU inference.

Configuration

Setting	Hosted default
Entry point	space_app.py
Inference configuration	configs/hosted.yaml
Evidence trees	demos/ui_tree
Retrieval bank	demos/bank
Image artifacts	demos/artifacts
Model	Qwen/Qwen2.5-VL-3B-Instruct
Device and precision	cuda:0, bf16
Output limit	256 generated tokens per model call
GPU duration	60 seconds for the complete QA call
Image budget	401,408 pixels, about 512 visual tokens for Qwen
Question limit	2,000 characters
Host and port	0.0.0.0:7860

The existing SLIDEBANK_UI_TREE_DIR, SLIDEBANK_BANK_DIR, SLIDEBANK_ARTIFACT_DIR, SLIDEBANK_INFER_CONFIG, SLIDEBANK_HOST, and SLIDEBANK_PORT variables can override the corresponding launcher paths or network settings. SLIDEBANK_GPU_DURATION sets a GPU duration between 15 and 120 seconds, defaulting to 60. Changing model, path, or hardware settings requires a Space restart.

allow_remote_download: true in configs/hosted.yaml explicitly permits model downloads on the hosted path. Local Hugging Face inference remains cache-only by default; enabling downloads for the Space does not alter that default.

Checks without model downloads

python -m unittest discover -s tests -p 'test_hosted_runtime.py' -v

These checks exercise CPU-only startup, hosted control restrictions, model configuration, the GPU boundary, and decoder output parsing with temporary fixtures. Actual ZeroGPU generation is checked on the deployed Space.