BranchOne

BranchOne chooses among candidate descriptions supplied with each request. Give it a situation, a question, and two or more choices, optionally with one image. It returns one probability per choice and the selected candidate's ID.

This release contains trained LoRA adapters, a shared decision head, a processor, calibration, and the inference code. The loader downloads the pinned public Qwen3.5-0.8B-Base backbone separately on first use. No reproduction repository, training scripts, or GitHub access is required.

Install

The supported setup is Linux x86-64, Python 3.13, and an NVIDIA CUDA GPU. The reference environment uses an A100, PyTorch 2.11.0 with CUDA 12.8, Transformers 5.17.0, and FP16 backbone weights. The head computes in FP32. CPU inference, other precisions, and quantization are not qualified for this release. Leave several GB of GPU memory available; requirements increase with input length. A maximum-memory benchmark is not provided.

On a fresh Colab GPU runtime or a Linux CUDA machine:

python3 -m pip install uv==0.12.17
python3 -m uv tool run --from huggingface-hub==1.33.0 hf download ndemirdemir/BranchOne --local-dir BranchOne
python3 BranchOne/install.py --venv .venv

The installer creates an isolated environment, installs the locked dependencies, and compiles causal-conv1d==1.7.0. It needs CUDA 12.8 development tools. On Colab it can install those tools through the configured NVIDIA apt repository; on a machine without root access, install the toolkit first and set CUDA_HOME. The initial dependency installation and kernel build can take several minutes. Run the examples below with .venv/bin/python, or select that environment in your notebook. These setup scripts install inference dependencies only.

For a complete Colab cell using this environment, see Ten questions with timing in Colab. Creating .venv does not switch the notebook's own Python environment.

Text example

import os
from transformers import AutoModel

# Defaults to the Hub. A downloaded directory works with the same interface.
model = AutoModel.from_pretrained(
    os.environ.get("BRANCHONE_MODEL", "ndemirdemir/BranchOne"),
    trust_remote_code=True,
    device="cuda",
)

result = model.predict({
    "state": "The parcel arrived, but the item inside is broken.",
    "question": "What should happen next?",
    "candidates": [
        {"id": "replace", "description": "Arrange a replacement for the damaged item."},
        {"id": "track", "description": "Track the parcel because it has not arrived."},
    ],
})
print(result["selected_id"])
print(dict(zip(result["candidate_ids"], result["probabilities"])))

trust_remote_code=True loads the BranchOne inference implementation included in this model repository. For reproducible deployments, add revision="<release commit SHA>" once a release is published. To use the directory downloaded above, set BRANCHONE_MODEL=./BranchOne. Both the model package and the base model must already be cached when using local_files_only=True.

Image example

Use the same loaded model. This complete example creates a sample picture; replace picture with a PIL image or a local image path for your own input.

from PIL import Image, ImageDraw

picture = Image.new("RGB", (512, 512), "white")
ImageDraw.Draw(picture).ellipse((128, 128, 384, 384), fill="blue")
image_result = model.predict({
    "state": "Look at the supplied image.",
    "question": "Which shape is shown?",
    "image": picture,
    "candidates": [
        {"id": "circle", "description": "A circle."},
        {"id": "square", "description": "A square."},
        {"id": "triangle", "description": "A triangle."},
    ],
})
print(image_result["selected_id"])
print(dict(zip(image_result["candidate_ids"], image_result["probabilities"])))

Ten questions with timing in Colab

After installation finishes, paste this entire block into one Colab code cell. It assumes the environment is at /content/.venv, as with the setup above run from Colab's default /content directory. The first line must be %%bash.

The model loads once, runs a separate warm-up example, then answers ten different text questions with three choices each. Each prediction's timer includes input preparation and inference, with CUDA synchronization before and after the call. Model loading, warm-up, and printing are excluded from those prediction times. The summary reports loading and warm-up separately, plus the total, mean, median, fastest, and slowest prediction times.

%%bash
set -e
/content/.venv/bin/python -u - <<'PY'
import os
import time
import statistics
import torch
from transformers import AutoModel


def make_request(state, question, choices):
    return {
        "state": state,
        "question": question,
        "candidates": [
            {"id": str(i), "description": choice}
            for i, choice in enumerate(choices)
        ],
    }


# Load once.
started = time.perf_counter()
model = AutoModel.from_pretrained(
    os.environ.get("BRANCHONE_MODEL", "ndemirdemir/BranchOne"),
    trust_remote_code=True,
    device="cuda",
)
torch.cuda.synchronize()
loading_seconds = time.perf_counter() - started

print("GPU:", torch.cuda.get_device_name(0))
print(f"Model loading: {loading_seconds:.3f} seconds")


questions = [
    (
        "The parcel arrived, but the item inside is broken.",
        "What should happen next?",
        [
            "Arrange a replacement for the damaged item.",
            "Track the parcel because it has not arrived.",
            "Send a receipt for a new purchase.",
        ],
    ),
    (
        "Buenos días. ¿A qué hora abre la tienda?",
        "Which language is this text written in?",
        ["Spanish.", "French.", "English."],
    ),
    (
        "Alex cannot attend on Tuesday. Alex is available Thursday after 2 PM.",
        "Which appointment fits Alex's availability?",
        ["Tuesday at 10 AM.", "Thursday at 3 PM.", "Thursday at 9 AM."],
    ),
    (
        "Refunds are allowed within 30 days with a receipt. "
        "The customer purchased the item 10 days ago and has the receipt.",
        "Does this request meet the stated refund conditions?",
        [
            "Yes, both conditions are satisfied.",
            "No, the purchase was too long ago.",
            "No, the receipt is missing.",
        ],
    ),
    (
        "Ingredients: flour, eggs, and milk. Mix together, "
        "pour into a pan, and bake for 25 minutes.",
        "What kind of document is this?",
        ["A recipe.", "An invoice.", "Meeting minutes."],
    ),
    (
        "A Python program reports: FileNotFoundError: "
        "No such file or directory: '/data/input.csv'.",
        "What problem does this error describe?",
        [
            "The requested file cannot be found at that path.",
            "The network connection timed out.",
            "The password was rejected.",
        ],
    ),
    (
        "Maya will send the draft on Friday. Leon will review it on Monday.",
        "Who is responsible for sending the draft?",
        ["Maya.", "Leon.", "Both Maya and Leon."],
    ),
    (
        "The rule says: reorder when stock falls below five items. "
        "There are currently two items in stock.",
        "What action follows the rule?",
        [
            "Place a replenishment order.",
            "Wait because stock is above the threshold.",
            "Permanently stop selling the item.",
        ],
    ),
    (
        "The picnic will be indoors if it rains and outdoors otherwise. "
        "It is raining.",
        "Where should the picnic take place?",
        ["Indoors.", "Outdoors.", "The location cannot be determined."],
    ),
    (
        "This product broke after one day. I regret buying it "
        "and would not recommend it.",
        "What is the overall sentiment?",
        ["Positive.", "Negative.", "Neutral."],
    ),
]


# Separate warm-up: excluded from the ten measured questions.
warmup = make_request(
    "The delivery arrived on time and everything is intact.",
    "Which description matches the delivery?",
    ["On time and undamaged.", "Missing.", "Damaged."],
)

torch.cuda.synchronize()
started = time.perf_counter()
model.predict(warmup)
torch.cuda.synchronize()
warmup_seconds = time.perf_counter() - started

print(f"Warm-up: {warmup_seconds:.3f} seconds")
print("\nRunning 10 timed predictions...")


# Reuse the same loaded model for every question.
timings = []

for number, (state, question, choices) in enumerate(questions, start=1):
    request = make_request(state, question, choices)

    torch.cuda.synchronize()
    started = time.perf_counter()

    result = model.predict(request)

    torch.cuda.synchronize()
    elapsed = time.perf_counter() - started
    timings.append(elapsed)

    descriptions = {
        candidate["id"]: candidate["description"]
        for candidate in request["candidates"]
    }

    print(f"\n{number}. {question}")
    print("Selected:", descriptions[result["selected_id"]])
    print(f"Prediction time: {elapsed:.3f} seconds")
    for candidate_id, probability in zip(
        result["candidate_ids"], result["probabilities"]
    ):
        print(f"  {probability:.1%} — {descriptions[candidate_id]}")


print("\n--- Timing summary ---")
print(f"Model loading:       {loading_seconds:.3f} seconds")
print(f"Warm-up:             {warmup_seconds:.3f} seconds")
print(f"10 predictions:      {sum(timings):.3f} seconds")
print(f"Mean per prediction: {statistics.mean(timings):.3f} seconds")
print(f"Median:              {statistics.median(timings):.3f} seconds")
print(f"Fastest:             {min(timings):.3f} seconds")
print(f"Slowest:             {max(timings):.3f} seconds")
PY

All ten predictions share the loaded model within this process. Re-running the whole cell starts a new Python process and reloads the model; separate %%bash cells cannot share the model variable. On first use, loading can also include downloading the Qwen backbone. New input shapes may still trigger additional kernel initialization after the warm-up.

This is a short-text timing example for your runtime, not a representative accuracy benchmark or a measurement of long-context or image performance. It does not establish sub-second latency. Save any results you need and stop your Colab runtime when finished.

Inputs and outputs

• state: situation or evidence as text.
• question: the decision to make.
• candidates: 2–150 unique string IDs with nonempty text descriptions. IDs are external labels; the model reads the descriptions. This is an input guard, not a claim of equal quality or speed at all candidate counts.
• image: optional single PIL image or local path. Image URLs, multiple images, video, and audio are unsupported.

The limit is 8,192 expanded tokens per candidate branch, including the situation, question, prompt formatting, that candidate's description, and image tokens. Images are resized to a budget of at most 256 visual tokens. Oversized inputs raise an error; they are not silently truncated. Increasing the context setting is not a supported extension of this trained release.

predict() returns selected_id, input-aligned candidate_ids, probabilities, raw logits, temperature, token counts, and calibration/policy metadata. The probability distribution is conditional on the supplied choices. It does not establish that a correct choice is present.

By default, action is "defer" and deferral_status is "no validated policy": the release has no validated automatic-acceptance threshold. The predicted candidate and probabilities are still returned. An optional caller-supplied threshold changes the action rule; callers must validate their own threshold.

Architecture and training

Each candidate is encoded against the same evidence on an independent Qwen branch. A shared head maps each 1,024-wide candidate representation through a 256-wide projection and residual MLP to one scalar. Softmax over the scores, with the supplied temperature, gives the candidate probabilities. This checkpoint uses the independent scorer without a candidate-comparison layer.

The vision backbone is frozen. The text backbone has trained rank-8 LoRA adapters; the decision head is trained. The vocabulary generation head is not used. Prefix-cache reuse is disabled for this release. Additional candidates and longer inputs still require additional computation.

Training used supervised single-label choice examples from procedural synthetic text and rendered images. The checkpoint was selected using development loss; temperature scaling was fitted on a separate calibration split. This release does not establish general-purpose decision accuracy, natural-image competence, or calibrated probabilities on a new domain. Boolean and ordered-score quality have not been separately validated.

Limitations

Evaluate on representative data before deploying. Missing evidence, ambiguous choices, unsupported image content, and distribution shifts can produce wrong answers with high confidence. Temperature scaling does not guarantee that a particular prediction is correct. Duplicate descriptions share evidence and can change the probability distribution.

This is an experimental inference release. It does not claim sub-second latency for the trained checkpoint, fast long-context CPU inference, or superiority to other decision models. The package is intended for AutoModel.from_pretrained(...).predict(...); standard generation pipelines, hosted inference-provider support, and save_pretrained() are not provided.

License

Apache-2.0. See LICENSE and NOTICE. The pinned Qwen backbone is Qwen/Qwen3.5-0.8B-Base at revision dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68 and retains its upstream attribution.