karpo-decision-9B / README.md
lionel-mp's picture
Copyright holder: MachinePulse Pte. Ltd.
0fb7298 verified
|
Raw History Blame Contribute Delete
7.43 kB
---
license: apache-2.0
library_name: transformers
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
pipeline_tag: text-classification
tags:
- decision-model
- decision-index
- system-one
- qwen3_5
---
# Karpo Decision 9B
<p>
<a href="https://github.com/machinepulse-ai/karpo-decision"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-machinepulse--ai%2Fkarpo--decision-181717?style=flat-square&logo=github&logoColor=white"></a>
<a href="https://huggingface.co/MachinePulseAI/karpo-decision-31B"><img alt="Karpo Decision 31B: Decision Index #1" src="https://img.shields.io/badge/Karpo_Decision_31B-Decision_Index_%231-FFD21E?style=flat-square"></a>
<a href="https://huggingface.co/MachinePulseAI/karpo-decision-9B/blob/main/LICENSE"><img alt="License: Apache 2.0" src="https://img.shields.io/badge/License-Apache_2.0-4b5563?style=flat-square"></a>
</p>
A decision model of the Karpo Decision family: given a state (a web page, a document, a conversation: any text or
JSON) and typed questions about it, it returns a probability for every option of every question in one forward pass,
without generating text. Fine-tuned from [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), with a linear readout over
the options.
- **Parameters:** 8.4B in bf16, plus a 255 × 4,096 readout in fp32
- **Questions:** `choice` (up to 255 options), `noul` (true or false), `score` (a level on a scale)
- **Inputs:** text or JSON, and a screenshot at `state.viewport_screenshot`
- **Context:** 32,768 tokens per sequence; a longer request is split by questions, each part with the full state
- **Interfaces:** a Python API and an HTTP server in the System One format (`POST /v1/systemone`)
- **Licence:** [Apache-2.0](LICENSE)
## The Karpo Decision family
<img src="assets/karpo-decision-family.png" alt="Score vs. size: Karpo Decision 0.8B to 31B on the Decision Index 0.3 public suite, with the other board models of each size class" width="100%">
*Score vs. size on the Decision Index 0.3 public suite, board of 12 November 2026.*
| Model | Base | Parameters |
|---|---|---|
| [karpo-decision-31B](https://huggingface.co/MachinePulseAI/karpo-decision-31B) | Gemma 4 31B | 31.3B |
| [karpo-decision-12B](https://huggingface.co/MachinePulseAI/karpo-decision-12B) | Gemma 4 12B | 12.0B |
| **karpo-decision-9B** | Qwen3.5 9B | 8.4B |
| [karpo-decision-4B](https://huggingface.co/MachinePulseAI/karpo-decision-4B) | Qwen3.5 4B | 4.5B |
| [karpo-decision-2B](https://huggingface.co/MachinePulseAI/karpo-decision-2B) | Qwen3.5 2B | 2.2B |
| [karpo-decision-0.8B](https://huggingface.co/MachinePulseAI/karpo-decision-0.8B) | Qwen3.5 0.8B | 0.85B |
The family's largest model, [karpo-decision-31B](https://huggingface.co/MachinePulseAI/karpo-decision-31B), is ranked #1 on the
[Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index), with the highest Full score, 73.9, on
the maintainers' private tests (edition 0.4.1, board of 12 November 2026, 300 entries); its public index is 75.0.
## How it works
- **A readout, not generation.** Every option is labelled with an option code. At the readout point that ends each
question, a linear readout scores 255 option codes and a softmax over the question's own options gives the
probabilities. Nothing is decoded, so the answer is always one of the options.
- **Many questions, one forward pass.** A request is one state and any number of questions, encoded as one sequence:
the list of questions, the state, then each question again with its options and a readout point.
- **Hybrid attention.** The full-attention layers read the whole sequence in both directions; the linear-attention
layers read it in order, so the questions come before the state.
The question-list-first layout follows [FlexDM](https://github.com/1000luca/FlexDM) and
[jev-brain](https://github.com/wcole3/jev-brain).
## Use
Inference code: [github.com/machinepulse-ai/karpo-decision](https://github.com/machinepulse-ai/karpo-decision), with the `qwen` extra. Python 3.12 or later and one GPU; the weights
take 16.8 GB, long requests more. Loading the checkpoint as a plain chat model does not reproduce its answers.
```bash
pip install "karpo-decision[qwen] @ git+https://github.com/machinepulse-ai/karpo-decision"
```
```python
from huggingface_hub import snapshot_download
from karpo_decision.decider import Decider
path = snapshot_download("MachinePulseAI/karpo-decision-9B")
decider = Decider.from_pretrained(path, device="cuda")
response = decider.decide({
"state": "Order 1182: the customer asks to return a blender bought 45 days ago. Policy: returns within 30 days.",
"questions": {
"eligible": {"type": "noul", "instructions": "Is the return eligible under the policy?"},
"action": {"type": "choice", "instructions": "What should the agent do?",
"criteria": {"refund": "Issue a full refund", "decline": "Politely decline the return",
"escalate": "Escalate to a supervisor"}},
},
})
print(response["answers"])
# {"eligible": {"type": "noul", "noul": P(true)},
# "action": {"type": "choice", "choice": <key>, "probabilities": {<key>: p, ...}, "confidence": c}}
```
A screenshot goes in the state at `viewport_screenshot`, as `{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}`.
HTTP server (`POST /v1/systemone`, `GET /healthz`):
```bash
CKPT=$(python -c 'from huggingface_hub import snapshot_download; print(snapshot_download("MachinePulseAI/karpo-decision-9B"))')
torchrun --standalone --nproc_per_node 1 -m karpo_decision.serve.server --checkpoint "$CKPT" --port 18440
```
[Decision Index](https://github.com/apolinario/decision-index) kit, installed separately:
```bash
pip install "decision-index @ git+https://github.com/apolinario/decision-index"
python -m decision_index run --engine karpo_decision.serve.engine:DecisionIndexEngine \
--option checkpoint="$CKPT" --out runs/karpo-decision-9B
```
## Lineage
[Qwen/Qwen3.5-9B-Base](https://huggingface.co/Qwen/Qwen3.5-9B-Base) → [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)
→ **karpo-decision-9B** (fine-tuned)
## Limitations
- Images: a screenshot is read through the base model's vision tower; image inputs were not evaluated.
- Not a chat or text-generation model; generation was not evaluated.
- A question that does not fit in 32,768 tokens with its state is refused, never truncated.
- It always picks one of the given options.
## Licence
The weights and model files are licensed under the [Apache License, Version 2.0](LICENSE).
Copyright 2026 MachinePulse Pte. Ltd.
karpo-decision-9B is a modified version of [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) ([Apache-2.0](https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE)):
decoder weights fine-tuned, readout added. Qwen3.5 stays under its own licence, a copy of which is in [licenses/Qwen3.5-LICENSE](licenses/Qwen3.5-LICENSE); see [NOTICE](NOTICE).
The inference code ([github.com/machinepulse-ai/karpo-decision](https://github.com/machinepulse-ai/karpo-decision)) is licensed under Apache-2.0.
### Attribution
Not required by the licence, but appreciated: if a product or service uses this model or a derivative, show
> Powered by karpo-decision (MachinePulse)
with a link to <https://huggingface.co/MachinePulseAI/karpo-decision-9B> where the medium allows one.