TernaryDecider v1 · one-token decisions for Ternary-Bonsai-2-9B
A LoRA that turns the 9B ternary Bonsai into a fast, calibrated decision model you switch on per request. State + question + options in, one probability per option out, from a single forward pass.
By Adrian Ciprian Iancu · GitHub · Hugging Face · LinkedIn
Not a chat model. This adapter answers multiple-choice and yes/no decisions in the format below, read from the probabilities of the first answer token. Load it at scale 0 next to the base model and turn it on only for decision requests: chat stays exactly the base model.
Results at a glance


Same base model, same prompt, same readout. "Base" is the adapter loaded at scale 0, "LoRA" at scale 1.
| benchmark | questions | base | TernaryDecider v1 | gain |
|---|---|---|---|---|
| System One decisions, English | 100 | 83.0% | 96.0% | +13.0 |
| System One decisions, Italian | 100 | 72.0% | 90.0% | +18.0 |
| Held-out agent decisions, English (never used for training or tuning) | 62 | 75.8% | 93.5% | +17.7 |
| Held-out agent decisions, Italian | 62 | 72.6% | 88.7% | +16.1 |
| Public decision sets, official validation / test splits (13 sets, 5 languages) | 13,000 | 78.8% | 89.6% | +10.8 |
The tool-choice questions are where the base model struggles and the adapter helps most: 17/40 → 34/40 in Italian and 25/40 → 36/40 in English on the System One set.
llama.cpp gives the same answers as my PyTorch evaluation. On all 324 agent questions above, base and LoRA, the count of right answers from the GGUF in llama.cpp is identical to the PyTorch rebuild (for example held-out EN 47 → 58, System One IT 72 → 90).
Pick a file
| file | for | size | what it is |
|---|---|---|---|
TernaryDecider-v1-Ternary-Bonsai-2-9B-LoRA-F16.gguf |
llama.cpp (PrismML fork) --lora |
241 MB | rank 48, alpha 96, F16, 200 tensor pairs |
calibration.json |
your readout | 1 KB | softmax temperature fitted on the dev set (T = 1.25) |
eval/ |
checking my numbers | small | every benchmark summary, base and LoRA |
training/ |
rebuilding it | small | data builder, trainer, merge, evaluation, llama.cpp check |
Results
Agent decisions (what I built it for)
Questions taken from the local assistant I am building (Vergilius): which read-only tool to call now, which UI element to act on, what state a web page is in. The 100-question System One set and the 62 held-out questions were excluded from the training data by id and by wording.
| set | category | base | LoRA |
|---|---|---|---|
| System One EN (100) | tool to call (40) | 25 | 36 |
| UI element (35) | 34 | 35 | |
| page state (25) | 24 | 25 | |
| System One IT (100) | tool to call (40) | 17 | 34 |
| UI element (35) | 32 | 31 | |
| page state (25) | 23 | 25 | |
| held-out EN (62) | tool (24) / UI (22) / page (16) | 14 / 18 / 15 | 22 / 20 / 16 |
| held-out IT (62) | tool (24) / UI (22) / page (16) | 9 / 20 / 16 | 20 / 19 / 16 |
Public decision sets
The official validation or test split of each public set (training only read the train splits), 1,000 questions each, sampled with a fixed seed, natural label distribution (Civil Comments balanced toxic / non-toxic). Read through llama.cpp with the GGUF.
| set | language | questions | base | TernaryDecider v1 | gain |
|---|---|---|---|---|---|
| BoolQ (validation) | en | 1,000 | 83.2% | 89.8% | +6.6 |
| Civil Comments, toxic? (test) | en | 1,000 | 80.6% | 91.3% | +10.7 |
| MASSIVE intent, 8 options (test) | en | 1,000 | 91.2% | 97.5% | +6.3 |
| MASSIVE intent, 8 options (test) | it | 1,000 | 88.5% | 96.7% | +8.2 |
| PAWS-X (test) | de | 1,000 | 73.1% | 89.8% | +16.7 |
| PAWS-X (test) | en | 1,000 | 78.9% | 93.5% | +14.6 |
| PAWS-X (test) | es | 1,000 | 74.8% | 89.6% | +14.8 |
| PAWS-X (test) | fr | 1,000 | 73.4% | 89.8% | +16.4 |
| SQuAD 2.0, answerable? (validation) | en | 1,000 | 83.1% | 90.2% | +7.1 |
| XNLI (test) | de | 1,000 | 72.2% | 82.3% | +10.1 |
| XNLI (test) | en | 1,000 | 79.5% | 88.3% | +8.8 |
| XNLI (test) | es | 1,000 | 74.8% | 83.6% | +8.8 |
| XNLI (test) | fr | 1,000 | 71.3% | 82.5% | +11.2 |
| all 13 sets | 5 languages | 13,000 | 78.8% | 89.6% | +10.8 |
The adapter is better on every one of the 13 sets, in all five languages. It never saw these splits, and the agent questions it was built for are a different task from all of them.
PubMedQA was used in training but has no held-out labeled split, so it is not in this table.
Calibration
The option probabilities are the softmax over the option tokens. On the dev set (1,620 held-out rows) the negative
log-likelihood of the right answer goes from 0.615 (base) to 0.184 (LoRA), and to 0.178 with temperature 1.25 from
calibration.json. Use the temperature if you threshold on confidence.
Use it with llama.cpp
You need the PrismML fork of llama.cpp (ternary kernels) and the base model
from prism-ml/Ternary-Bonsai-2-9B-gguf. I tested with
Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf.
llama-server -m Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf \
--lora-scaled TernaryDecider-v1-Ternary-Bonsai-2-9B-LoRA-F16.gguf:0 \
-ngl 999 -fa on -c 16384
Decision request (adapter on for this request only):
{"prompt": "<rendered prompt, see below>", "n_predict": 1, "n_probs": 400, "temperature": 0,
"lora": [{"id": 0, "scale": 1.0}]}
Read completion_probabilities[0].top_logprobs. Each option is the sum of the probabilities of its token forms
(A and A; yes, Yes, YES against no, No, NO), then normalize over the options and, if you want calibrated
confidence, divide the log-probabilities by 1.25 before the softmax. Send every other request with
"lora": [{"id": 0, "scale": 0}], or give decisions their own slot (id_slot): the scale set by a request stays on
its slot.
On Windows --lora-scaled splits on :, so D:\models\x.gguf:0 breaks: start the server from the adapter's folder
and pass the file name only.
Prompt format
The d1 layout of LiquidAI/d1-2B, rendered with the Qwen ChatML template and the non-thinking lead. Two question types:
<|im_start|>user
{
"user_message": "will it rain in Lisbon tomorrow morning?",
"tool_results": "none"
}
QUESTION:
Which read-only tool must be called NOW, before answering the user?
Options:
A weather: forecast for a named place, next 7 days
B news_search: news about a subject within a window of days
C none: no tool is needed now, answer directly
Reply with the option code only.<|im_end|>
<|im_start|>assistant
<think>
</think>
The state is any JSON object (or plain text), indented with 2 spaces. A yes/no question ends with
Reply with yes or no only. instead of the options. Put a "none" option last when "none" is a valid answer: the
model always picks one of the listed options. training/jev_prompt.py renders it for you.
How I trained it
- No distillation. Gold labels only, cross-entropy computed over the option tokens only (each option's token forms pooled with logsumexp), so the training loss is exactly the readout used at inference.
- Data, 30,916 training rows:
| part | rows | source | license |
|---|---|---|---|
| agent decisions: tool, skill, UI element, page state (EN + IT) | 8,562 | generated from my assistant's real tool, skill and UI catalogs, with the label the generator intended; each question twice with the options shuffled ("none" stays last) | mine, Apache-2.0 |
| BoolQ | 2,394 | train split | CC BY-SA 3.0 |
| SQuAD 2.0, "does the passage answer it?" | 2,385 | train split | CC BY-SA 4.0 |
| Civil Comments, "is it toxic?" | 2,373 | train split, balanced | CC0 1.0 |
| MASSIVE intents, EN + IT, 8 options | 4,757 | train split | CC BY 4.0 |
| PubMedQA | 955 | labeled split | MIT |
| XNLI, EN FR ES DE | 4,754 | train split | CC BY-NC 4.0 |
| PAWS-X, EN FR ES DE | 4,736 | train split | see dataset |
5% of the questions (by question, so the two shuffled copies stay together) form the dev set (1,620 rows); the counts above are the training rows left after that split.
- Recipe: three runs with seeds 0, 1, 2, each rank 16, alpha 32, half an epoch (966 steps of 16), AdamW lr 1e-4, warmup, constant, then a final decay. The three adapters are averaged exactly by concatenation (A stacked, B divided by 3): one rank-48 adapter whose output equals the mean of the three. Then one softmax temperature is fitted on the dev set.
- Model: Ternary-Bonsai-2-9B rebuilt exactly in PyTorch from the PQ2_0 GGUF (2-bit codes plus the Hadamard
rotation of the PrismML format), weights frozen; LoRA on the attention and GatedDeltaNet projections and on the MLP of
all 32 layers. Exported in the GGUF LoRA format the fork reads with
--lora. - Hardware and time: one NVIDIA RTX PRO 6000 Blackwell (96 GB). The frozen ternary weights are unpacked once to BF16; batches are bucketed by length (padding from 2x to 1.03x) and gradient checkpointing is used only for long batches. That took a run from 43 to 16 minutes; the three seeds, merge, calibration and evaluation took about 1.5 hours.
How much to trust this
- The agent sets are small (100 + 62 questions per language): one question is 1 or 1.6 points. The gains on tool choice (+11 to +17 questions) are far outside that noise; the small differences on UI elements are not.
- The agent questions come from the same harness as the training questions, written by the same generator, but the test questions were excluded by id and wording.
- The public numbers are on official held-out splits, 1,000 questions per set, one seed for the sampling.
Limitations
- Format bound. It is trained for this prompt layout and the option-token readout. Free-form answers are not its job.
- Always picks an option. Add a "none" option when that is valid, and use the calibrated confidence to hand hard cases to a slower step.
- Languages. Agent questions in English and Italian, public sets in English, Italian, French, Spanish and German.
- Non-commercial data. XNLI (CC BY-NC 4.0) is part of the training data. Check whether that matters for your use.
License and credits
Adapter weights: Apache-2.0. Base model: prism-ml/Ternary-Bonsai-2-9B-gguf (Apache-2.0), from Qwen3.5-9B. The decision layout and the option-token readout follow LiquidAI/d1-2B; no LiquidAI weights or code are included. Dataset licenses are listed above. Not affiliated with PrismML, Qwen or LiquidAI.
Citation
@misc{iancu2026ternarydecider,
author = {Iancu, Adrian Ciprian},
title = {TernaryDecider v1: a one-token decision LoRA for Ternary-Bonsai-2-9B},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/adriandj3/TernaryDecider-v1-Decision-LoRA}}
}
Its sibling for screens: TernaryPilot v2, computer and browser use for the same base model. If you try it, open a discussion here or find me on GitHub.
Keywords: decision model, System One, multiple choice, classification, tool routing, agent, calibrated probabilities, LoRA, GGUF, llama.cpp, ternary, 1.58-bit, Bonsai, Qwen3.5, BoolQ, XNLI, PAWS-X, MASSIVE, local AI
- Downloads last month
- 122
16-bit
Model tree for KeytechSRL/TernaryDecider-v1-Decision-LoRA
Base model
Qwen/Qwen3.5-9B-Base