File size: 3,460 Bytes
a82e638 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | # Run Banyan Sapling
The Hugging Face release is published as `MubitLabs/banyan-sapling`. The package
includes complete trained weights, tokenizer, calibration, inference code, and
a distribution manifest. A separate base-model download is not required.
## Reference environment
The model was evaluated with Python 3.12, CUDA 13.0, and an NVIDIA H200. Use a
CUDA environment compatible with the package's pinned PyTorch build and install
`requirements.txt`. The checkpoint is approximately 46 GB (43 GiB); allow
additional GPU memory for the CUDA runtime and activations. The verified
hardware is an H200. Memory use increases with input length.
The loader checks exact package versions against `runtime_versions.json`.
Different library versions, precision, kernels, or hardware require separate
validation. CPU and quantized inference are outside this reference runtime.
## Download
Install the Hugging Face CLI using the
[official CLI instructions](https://huggingface.co/docs/huggingface_hub/guides/cli).
Download to a local directory:
```sh
hf download MubitLabs/banyan-sapling --local-dir ./banyan-sapling-model
cd banyan-sapling-model
python3.12 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
```
For repeatable deployments, pass `--revision` with the Hugging Face commit SHA.
Use a fresh environment for the pinned runtime.
## Verify and run
```sh
.venv/bin/python inference.py --check-files
.venv/bin/python inference.py --input examples/policy_claim.json
```
The input is a JSON object containing `policy`, `evidence`, and `claim`. Each
must be a nonempty string. The combined encoded input, including formatting,
is limited to 16,384 tokens. The output contains `raw_label`, `status`, both
prediction sets, calibrated probabilities, and the source weights identity.
Checksum verification reads the complete checkpoint before loading.
The supplied example asks whether a deployment is permitted after a rejected
security review. Its expected semantic label is `violation`. In the reference run,
the preferred classification was `violation` and the calibrated status was
`unsure`, with `violation` and `insufficient_information` in the prediction set.
This example illustrates why integrations should retain both output fields.
The recorded response is in `evidence/package_validation.json`.
## Repeated requests in Python
Run from the downloaded model directory:
```python
from pathlib import Path
from inference import load_model, predict
model, tokenizer, calibration, manifest = load_model(Path('.'))
result = predict(model, tokenizer, calibration, manifest, {
'policy': 'Deployment requires an approved security review. A rejected review prohibits deployment.',
'evidence': 'The security review for deployment D-204 was rejected.',
'claim': 'Deployment D-204 is permitted.',
})
print(result)
```
Load once and issue requests sequentially. The reference path uses one example
per model invocation. The weights contain a custom verification head and
unmerged attention adapters, so use this loader rather than a generic
text-generation pipeline.
## Package identity
`distribution_manifest.json` records SHA-256 hashes for distributed files.
The original trained weights hash is
`103a67f7a112589346c3e1090f5bca182d7fdafee85dfd43e67ac5d75eb04dd4`.
Original evaluation and runtime summaries are included under `evidence/`.
See [provenance](provenance.md) for the relationship to the archived release.
|