| # glance-runtime | |
| The runtime for Glance decision models: the prompt layout, two-way attention, answer-code readout | |
| and calibration that turn the weights in this repository into a decision model. | |
| ```bash | |
| pip install -U huggingface_hub | |
| hf download bpHigh/glance-0.1 --include "runtime/*" --local-dir glance-0.1 | |
| pip install "./glance-0.1/runtime[gpu]" # on a GPU | |
| ``` | |
| On a CPU-only machine, install the CPU build of PyTorch first (the default Linux build pulls | |
| several GB of CUDA libraries): | |
| ```bash | |
| pip install torch --index-url https://download.pytorch.org/whl/cpu | |
| pip install ./glance-0.1/runtime | |
| ``` | |
| ```python | |
| from glance_runtime import Glance | |
| glance = Glance.load("bpHigh/glance-0.1", device="cuda") # or "cpu" | |
| glance(state="...", questions={...}) # -> answers | |
| glance.decide({"state": ..., "questions": {...}}) # the full /v1/systemone response | |
| ``` | |
| ```bash | |
| python -m glance_runtime serve --model bpHigh/glance-0.1 --device cuda --port 8741 | |
| curl -s localhost:8741/v1/systemone -d @request.json | |
| ``` | |
| See the model card for the request format. | |