| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-9B |
| library_name: transformers |
| tags: |
| - decision-making |
| - self-distillation |
| - anyjev |
| --- |
| |
| # Tacit-9B (preview) |
|
|
| Tacit-9B makes typed decisions (choose an option, answer yes/no, or pick an ordered level) in a |
| single forward pass, returning a probability over the options. It is built on |
| [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) with AnyJev self-distillation: the base model wrote its own |
| decision problems, answered them with its reasoning on, and learned to give those answers in one |
| forward pass. No human labels, external datasets or other models were used. |
|
|
| ## Results |
|
|
| Accuracy in one forward pass on the full test sets, with the same prompt and readout for both |
| models. |
|
|
| | benchmark | Qwen3.5-9B | Tacit-9B | difference | |
| |---|---|---|---| |
| | JevBench (public set, 231) | 0.805 | 0.823 | +1.7 pts | |
| | bev-decision (test split, 46,320) | 0.680 | 0.725 | +4.5 pts | |
|
|
| ## Usage |
|
|
| ```python |
| from transformers.dynamic_module_utils import get_class_from_dynamic_module |
| |
| Tacit = get_class_from_dynamic_module("tacit_decision.Tacit", "morriszjm/Tacit-9B") |
| # one forward per decision |
| tacit = Tacit.from_pretrained("morriszjm/Tacit-9B") |
| # or: send low-confidence decisions to the model's own reasoning |
| tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", adaptive=True, tau=0.5, max_cot_share=0.2) |
| # or: the same on vLLM (pip install vllm), for throughput |
| tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="vllm", adaptive=True) |
| |
| d = tacit.decide(state="Customer: my package was due last Monday and it still has not arrived.", |
| question="What does the customer want?", |
| options=["track_order", "cancel_order", "refund", "change_address"]) |
| print(d["answer"], d["probs"], d["route"]) # route: "one_forward" or "cot" |
| ``` |
|
|
| `kind` is `"choice"` (default), `"yes_no"`, or `"score"` (`options` are ordered levels, lowest |
| first); `decide_batch([...])` takes a list of such dicts. |
|
|
| | argument | default | meaning | |
| |---|---|---| |
| | `adaptive` | `False` | send low-confidence decisions to the model's own reasoning (thinking on) | |
| | `tau` | `0.5` | a decision is low-confidence when the log-probability gap between its top two options is below `tau` | |
| | `max_cot_share` | `0.2` | at most this share of the last `cot_window` decisions goes to reasoning; `None` removes the cap | |
| | `cot_window` | `1000` | how many recent decisions the cap counts; `None` counts every decision since loading | |
| | `cot_max_tokens` | `8192` | reasoning budget of one decision | |
| | `engine` | `"transformers"` | `"vllm"` runs vLLM in this process; `"server"` uses a running `vllm serve` (see below) | |
| | `base_url` | `None` | `engine="server"`: the server's `/v1` URL | |
| | `vllm_kwargs` | `None` | passed to `vllm.LLM`, e.g. `dict(gpu_memory_utilization=0.85, max_model_len=32768)` | |
|
|
| An escalated decision's answer is read from the label distribution after the reasoning, so it also |
| comes with probabilities. Without `adaptive`, nothing is generated. On vLLM the labels are read from |
| the top 20 log-probabilities of the full vocabulary (a label outside them counts as 0). The checkpoint also loads as a |
| plain causal LM (`AutoModelForCausalLM`, vLLM); `tacit_decision.py` holds the prompt it was trained |
| with. |
|
|
| ## Serving with vLLM |
|
|
| ```bash |
| vllm serve morriszjm/Tacit-9B --host 127.0.0.1 --port 8000 |
| ``` |
|
|
| ```python |
| tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="server", |
| base_url="http://127.0.0.1:8000/v1", adaptive=True) |
| ``` |
|
|
| Only the tokenizer loads on the client. For many clients sharing one cap, run the HTTP gateway that |
| ships in this repo in front of the server: |
|
|
| ```bash |
| hf download morriszjm/Tacit-9B tacit_decision.py --local-dir . |
| python tacit_decision.py serve --model morriszjm/Tacit-9B --upstream http://127.0.0.1:8000/v1 --adaptive --port 8100 |
| curl -s 127.0.0.1:8100/v1/decide -H 'Content-Type: application/json' -d '{"state": "Customer: my package has not arrived.", "question": "What does the customer want?", "options": ["track_order", "refund"]}' |
| ``` |
|
|
| On Hopper GPUs vLLM compiles this model's gated-delta-rule kernel with `nvcc` on first use, so the serving machine needs the CUDA toolkit (`engine="vllm"` in Python switches to a Triton kernel by itself). |
|
|
| Apache-2.0, as is the base model by the Qwen team. |
|
|
