TernaryDecider v1 · one-token decisions for Ternary-Bonsai-2-9B

A LoRA that turns the 9B ternary Bonsai into a fast, calibrated decision model you switch on per request. State + question + options in, one probability per option out, from a single forward pass.

llama.cpp rank System One EN held out EN public sets status GitHub

By Adrian Ciprian Iancu · GitHub · Hugging Face · LinkedIn

Not a chat model. This adapter answers multiple-choice and yes/no decisions in the format below, read from the probabilities of the first answer token. Load it at scale 0 next to the base model and turn it on only for decision requests: chat stays exactly the base model.

Results at a glance

Accuracy of the base model and TernaryDecider v1 on four agent decision sets

Accuracy of the base model and TernaryDecider v1 on 13 public decision sets

Same base model, same prompt, same readout. "Base" is the adapter loaded at scale 0, "LoRA" at scale 1.

benchmark questions base TernaryDecider v1 gain
System One decisions, English 100 83.0% 96.0% +13.0
System One decisions, Italian 100 72.0% 90.0% +18.0
Held-out agent decisions, English (never used for training or tuning) 62 75.8% 93.5% +17.7
Held-out agent decisions, Italian 62 72.6% 88.7% +16.1
Public decision sets, official validation / test splits (13 sets, 5 languages) 13,000 78.8% 89.6% +10.8

The tool-choice questions are where the base model struggles and the adapter helps most: 17/40 → 34/40 in Italian and 25/40 → 36/40 in English on the System One set.

llama.cpp gives the same answers as my PyTorch evaluation. On all 324 agent questions above, base and LoRA, the count of right answers from the GGUF in llama.cpp is identical to the PyTorch rebuild (for example held-out EN 47 → 58, System One IT 72 → 90).

Pick a file

file for size what it is
TernaryDecider-v1-Ternary-Bonsai-2-9B-LoRA-F16.gguf llama.cpp (PrismML fork) --lora 241 MB rank 48, alpha 96, F16, 200 tensor pairs
calibration.json your readout 1 KB softmax temperature fitted on the dev set (T = 1.25)
eval/ checking my numbers small every benchmark summary, base and LoRA
training/ rebuilding it small data builder, trainer, merge, evaluation, llama.cpp check

Results

Agent decisions (what I built it for)

Questions taken from the local assistant I am building (Vergilius): which read-only tool to call now, which UI element to act on, what state a web page is in. The 100-question System One set and the 62 held-out questions were excluded from the training data by id and by wording.

set category base LoRA
System One EN (100) tool to call (40) 25 36
UI element (35) 34 35
page state (25) 24 25
System One IT (100) tool to call (40) 17 34
UI element (35) 32 31
page state (25) 23 25
held-out EN (62) tool (24) / UI (22) / page (16) 14 / 18 / 15 22 / 20 / 16
held-out IT (62) tool (24) / UI (22) / page (16) 9 / 20 / 16 20 / 19 / 16

Public decision sets

The official validation or test split of each public set (training only read the train splits), 1,000 questions each, sampled with a fixed seed, natural label distribution (Civil Comments balanced toxic / non-toxic). Read through llama.cpp with the GGUF.

set language questions base TernaryDecider v1 gain
BoolQ (validation) en 1,000 83.2% 89.8% +6.6
Civil Comments, toxic? (test) en 1,000 80.6% 91.3% +10.7
MASSIVE intent, 8 options (test) en 1,000 91.2% 97.5% +6.3
MASSIVE intent, 8 options (test) it 1,000 88.5% 96.7% +8.2
PAWS-X (test) de 1,000 73.1% 89.8% +16.7
PAWS-X (test) en 1,000 78.9% 93.5% +14.6
PAWS-X (test) es 1,000 74.8% 89.6% +14.8
PAWS-X (test) fr 1,000 73.4% 89.8% +16.4
SQuAD 2.0, answerable? (validation) en 1,000 83.1% 90.2% +7.1
XNLI (test) de 1,000 72.2% 82.3% +10.1
XNLI (test) en 1,000 79.5% 88.3% +8.8
XNLI (test) es 1,000 74.8% 83.6% +8.8
XNLI (test) fr 1,000 71.3% 82.5% +11.2
all 13 sets 5 languages 13,000 78.8% 89.6% +10.8

The adapter is better on every one of the 13 sets, in all five languages. It never saw these splits, and the agent questions it was built for are a different task from all of them.

PubMedQA was used in training but has no held-out labeled split, so it is not in this table.

Calibration

The option probabilities are the softmax over the option tokens. On the dev set (1,620 held-out rows) the negative log-likelihood of the right answer goes from 0.615 (base) to 0.184 (LoRA), and to 0.178 with temperature 1.25 from calibration.json. Use the temperature if you threshold on confidence.

Use it with llama.cpp

You need the PrismML fork of llama.cpp (ternary kernels) and the base model from prism-ml/Ternary-Bonsai-2-9B-gguf. I tested with Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf.

llama-server -m Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf \
  --lora-scaled TernaryDecider-v1-Ternary-Bonsai-2-9B-LoRA-F16.gguf:0 \
  -ngl 999 -fa on -c 16384

Decision request (adapter on for this request only):

{"prompt": "<rendered prompt, see below>", "n_predict": 1, "n_probs": 400, "temperature": 0,
 "lora": [{"id": 0, "scale": 1.0}]}

Read completion_probabilities[0].top_logprobs. Each option is the sum of the probabilities of its token forms (A and A; yes, Yes, YES against no, No, NO), then normalize over the options and, if you want calibrated confidence, divide the log-probabilities by 1.25 before the softmax. Send every other request with "lora": [{"id": 0, "scale": 0}], or give decisions their own slot (id_slot): the scale set by a request stays on its slot.

On Windows --lora-scaled splits on :, so D:\models\x.gguf:0 breaks: start the server from the adapter's folder and pass the file name only.

Prompt format

The d1 layout of LiquidAI/d1-2B, rendered with the Qwen ChatML template and the non-thinking lead. Two question types:

<|im_start|>user
{
  "user_message": "will it rain in Lisbon tomorrow morning?",
  "tool_results": "none"
}


QUESTION:
Which read-only tool must be called NOW, before answering the user?

Options:
A weather: forecast for a named place, next 7 days
B news_search: news about a subject within a window of days
C none: no tool is needed now, answer directly

Reply with the option code only.<|im_end|>
<|im_start|>assistant
<think>

</think>

The state is any JSON object (or plain text), indented with 2 spaces. A yes/no question ends with Reply with yes or no only. instead of the options. Put a "none" option last when "none" is a valid answer: the model always picks one of the listed options. training/jev_prompt.py renders it for you.

How I trained it

  • No distillation. Gold labels only, cross-entropy computed over the option tokens only (each option's token forms pooled with logsumexp), so the training loss is exactly the readout used at inference.
  • Data, 30,916 training rows:
part rows source license
agent decisions: tool, skill, UI element, page state (EN + IT) 8,562 generated from my assistant's real tool, skill and UI catalogs, with the label the generator intended; each question twice with the options shuffled ("none" stays last) mine, Apache-2.0
BoolQ 2,394 train split CC BY-SA 3.0
SQuAD 2.0, "does the passage answer it?" 2,385 train split CC BY-SA 4.0
Civil Comments, "is it toxic?" 2,373 train split, balanced CC0 1.0
MASSIVE intents, EN + IT, 8 options 4,757 train split CC BY 4.0
PubMedQA 955 labeled split MIT
XNLI, EN FR ES DE 4,754 train split CC BY-NC 4.0
PAWS-X, EN FR ES DE 4,736 train split see dataset

5% of the questions (by question, so the two shuffled copies stay together) form the dev set (1,620 rows); the counts above are the training rows left after that split.

  • Recipe: three runs with seeds 0, 1, 2, each rank 16, alpha 32, half an epoch (966 steps of 16), AdamW lr 1e-4, warmup, constant, then a final decay. The three adapters are averaged exactly by concatenation (A stacked, B divided by 3): one rank-48 adapter whose output equals the mean of the three. Then one softmax temperature is fitted on the dev set.
  • Model: Ternary-Bonsai-2-9B rebuilt exactly in PyTorch from the PQ2_0 GGUF (2-bit codes plus the Hadamard rotation of the PrismML format), weights frozen; LoRA on the attention and GatedDeltaNet projections and on the MLP of all 32 layers. Exported in the GGUF LoRA format the fork reads with --lora.
  • Hardware and time: one NVIDIA RTX PRO 6000 Blackwell (96 GB). The frozen ternary weights are unpacked once to BF16; batches are bucketed by length (padding from 2x to 1.03x) and gradient checkpointing is used only for long batches. That took a run from 43 to 16 minutes; the three seeds, merge, calibration and evaluation took about 1.5 hours.

How much to trust this

  • The agent sets are small (100 + 62 questions per language): one question is 1 or 1.6 points. The gains on tool choice (+11 to +17 questions) are far outside that noise; the small differences on UI elements are not.
  • The agent questions come from the same harness as the training questions, written by the same generator, but the test questions were excluded by id and wording.
  • The public numbers are on official held-out splits, 1,000 questions per set, one seed for the sampling.

Limitations

  • Format bound. It is trained for this prompt layout and the option-token readout. Free-form answers are not its job.
  • Always picks an option. Add a "none" option when that is valid, and use the calibrated confidence to hand hard cases to a slower step.
  • Languages. Agent questions in English and Italian, public sets in English, Italian, French, Spanish and German.
  • Non-commercial data. XNLI (CC BY-NC 4.0) is part of the training data. Check whether that matters for your use.

License and credits

Adapter weights: Apache-2.0. Base model: prism-ml/Ternary-Bonsai-2-9B-gguf (Apache-2.0), from Qwen3.5-9B. The decision layout and the option-token readout follow LiquidAI/d1-2B; no LiquidAI weights or code are included. Dataset licenses are listed above. Not affiliated with PrismML, Qwen or LiquidAI.

Citation

@misc{iancu2026ternarydecider,
  author       = {Iancu, Adrian Ciprian},
  title        = {TernaryDecider v1: a one-token decision LoRA for Ternary-Bonsai-2-9B},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/adriandj3/TernaryDecider-v1-Decision-LoRA}}
}

Its sibling for screens: TernaryPilot v2, computer and browser use for the same base model. If you try it, open a discussion here or find me on GitHub.

Keywords: decision model, System One, multiple choice, classification, tool routing, agent, calibrated probabilities, LoRA, GGUF, llama.cpp, ternary, 1.58-bit, Bonsai, Qwen3.5, BoolQ, XNLI, PAWS-X, MASSIVE, local AI

Downloads last month
122
GGUF
Model size
0.1B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for KeytechSRL/TernaryDecider-v1-Decision-LoRA

Finetuned
Qwen/Qwen3.5-9B
Adapter
(6)
this model

Datasets used to train KeytechSRL/TernaryDecider-v1-Decision-LoRA