minojev general decision model (Qwen3-1.7B + trained head)

Decisions, not tokens. This checkpoint turns Qwen3-1.7B into a typed decision model: it returns calibrated probability distributions for Choice, Boolean, and Score questions in one forward pass, with zero output tokens. It was produced by head training: the backbone stays frozen, candidate-path features are cached once, and only the decision head is trained, followed by dev-fitted temperature calibration.

Results

Test suite: 200 requests, balanced across four sources (banking77, CLINC150, Amazon Polarity, GSM8K verification); every candidate is declared in the request.

Metric	Value
Accuracy	97.5%
Expected calibration error (ECE)	0.014
Mean confidence	0.963
Selective accuracy (confidence >= 0.9)	89.5% coverage at 98.9% accuracy
Output tokens per decision	0

Head-to-head against token generation with the same backbone and prompts (120-decision balanced suite, chat template, thinking disabled):

Metric	minojev	generative baseline
Accuracy	95.8%	80.0%
ECE	0.024	not available
Output tokens / decision	0	3.48
Time to first token (p50)	0 ms	~100 ms
Latency p95	~1.1 s	~5.3 s

Zero-training native-logits readout on the same suite: 88.3% accuracy, ECE 0.097.

Training recipe

• Backbone: Qwen/Qwen3-1.7B (frozen, bfloat16), hidden size 2048.
• Head: shared scalar + set attention, ~0.8M parameters.
• Data: 8000 training requests converted from permissive public sources (banking77, CLINC150, Amazon Polarity, GSM8K verification; OOD evaluation uses MASSIVE and WANLI).
• Cost: ~37 minutes and ~4 GB peak memory on an Apple Silicon laptop; no GPU.
• Calibration: per-primitive temperature scaling fitted on dev outcomes.

Usage

from huggingface_hub import snapshot_download
from minojev import DecisionModel, Request, make_choice_question, ScoreOptions

path = snapshot_download("zeredy879/minojev", allow_patterns=["general/*"])
model = DecisionModel.load(f"{path}/general", device="cpu")

request = Request(
    id="r1",
    state={"text": "How do I add an existing card to the app?"},
    questions=[make_choice_question("q", "Which category applies?",
                                    {"card_linking": "card_linking", "atm": "atm", "transfer": "transfer"})],
)
record = model.score([request], ScoreOptions(mode="reuse"))[0]
print(record["candidate_ids"], record["probabilities"], record["decode_steps"])

Data and licensing

Source	License	Use
mteb/banking77 (upstream PolyAI/banking77)	MIT (upstream CC-BY-4.0)	train
clinc/clinc_oos	CC-BY-3.0	train
fancyzhx/amazon_polarity	Apache-2.0	train
openai/gsm8k	MIT	train
SetFit/amazon_massive_intent_en-US (upstream MASSIVE)	CC-BY-4.0	OOD evaluation
alisawuffles/WANLI	CC-BY-4.0	OOD evaluation

No NC-licensed data was used for training. The base model is Apache-2.0; this checkpoint inherits that license. Code is MIT.

Links

• Code, demos, and reproducible pipeline: https://github.com/zeredy879/minojev
• Live benchmark page: https://zeredy879.github.io/minojev/benchmark.html
• Datasets: https://huggingface.co/datasets/zeredy879/minojev-data