| # Run Banyan Sapling |
|
|
| The Hugging Face release is published as `MubitLabs/banyan-sapling`. The package |
| includes complete trained weights, tokenizer, calibration, inference code, and |
| a distribution manifest. A separate base-model download is not required. |
|
|
| ## Reference environment |
|
|
| The model was evaluated with Python 3.12, CUDA 13.0, and an NVIDIA H200. Use a |
| CUDA environment compatible with the package's pinned PyTorch build and install |
| `requirements.txt`. The checkpoint is approximately 46 GB (43 GiB); allow |
| additional GPU memory for the CUDA runtime and activations. The verified |
| hardware is an H200. Memory use increases with input length. |
|
|
| The loader checks exact package versions against `runtime_versions.json`. |
| Different library versions, precision, kernels, or hardware require separate |
| validation. CPU and quantized inference are outside this reference runtime. |
|
|
| ## Download |
|
|
| Install the Hugging Face CLI using the |
| [official CLI instructions](https://huggingface.co/docs/huggingface_hub/guides/cli). |
| Download to a local directory: |
|
|
| ```sh |
| hf download MubitLabs/banyan-sapling --local-dir ./banyan-sapling-model |
| cd banyan-sapling-model |
| python3.12 -m venv .venv |
| .venv/bin/python -m pip install -r requirements.txt |
| ``` |
|
|
| For repeatable deployments, pass `--revision` with the Hugging Face commit SHA. |
| Use a fresh environment for the pinned runtime. |
|
|
| ## Verify and run |
|
|
| ```sh |
| .venv/bin/python inference.py --check-files |
| .venv/bin/python inference.py --input examples/policy_claim.json |
| ``` |
|
|
| The input is a JSON object containing `policy`, `evidence`, and `claim`. Each |
| must be a nonempty string. The combined encoded input, including formatting, |
| is limited to 16,384 tokens. The output contains `raw_label`, `status`, both |
| prediction sets, calibrated probabilities, and the source weights identity. |
| Checksum verification reads the complete checkpoint before loading. |
|
|
| The supplied example asks whether a deployment is permitted after a rejected |
| security review. Its expected semantic label is `violation`. In the reference run, |
| the preferred classification was `violation` and the calibrated status was |
| `unsure`, with `violation` and `insufficient_information` in the prediction set. |
| This example illustrates why integrations should retain both output fields. |
| The recorded response is in `evidence/package_validation.json`. |
|
|
| ## Repeated requests in Python |
|
|
| Run from the downloaded model directory: |
|
|
| ```python |
| from pathlib import Path |
| from inference import load_model, predict |
| |
| model, tokenizer, calibration, manifest = load_model(Path('.')) |
| result = predict(model, tokenizer, calibration, manifest, { |
| 'policy': 'Deployment requires an approved security review. A rejected review prohibits deployment.', |
| 'evidence': 'The security review for deployment D-204 was rejected.', |
| 'claim': 'Deployment D-204 is permitted.', |
| }) |
| print(result) |
| ``` |
|
|
| Load once and issue requests sequentially. The reference path uses one example |
| per model invocation. The weights contain a custom verification head and |
| unmerged attention adapters, so use this loader rather than a generic |
| text-generation pipeline. |
|
|
| ## Package identity |
|
|
| `distribution_manifest.json` records SHA-256 hashes for distributed files. |
| The original trained weights hash is |
| `103a67f7a112589346c3e1090f5bca182d7fdafee85dfd43e67ac5d75eb04dd4`. |
| Original evaluation and runtime summaries are included under `evidence/`. |
| See [provenance](provenance.md) for the relationship to the archived release. |
|
|
