runtime/README.md

glance-runtime

The runtime for Glance decision models: the prompt layout, two-way attention, answer-code readout and calibration that turn the weights in this repository into a decision model.

pip install -U huggingface_hub
hf download bpHigh/glance-0.1 --include "runtime/*" --local-dir glance-0.1
pip install "./glance-0.1/runtime[gpu]"        # on a GPU

On a CPU-only machine, install the CPU build of PyTorch first (the default Linux build pulls several GB of CUDA libraries):

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install ./glance-0.1/runtime

from glance_runtime import Glance

glance = Glance.load("bpHigh/glance-0.1", device="cuda")   # or "cpu"
glance(state="...", questions={...})                       # -> answers
glance.decide({"state": ..., "questions": {...}})          # the full /v1/systemone response

python -m glance_runtime serve --model bpHigh/glance-0.1 --device cuda --port 8741
curl -s localhost:8741/v1/systemone -d @request.json

See the model card for the request format.