LemonDecider

A typed decision engine derived from Qwen/Qwen3.5-4B: it answers structured choice / noul / score questions in a single forward pass with an explicit unknown (abstention) candidate. There is no autoregressive generation head — the backbone is kept full-weight, and a 255-way single-token readout replaces the LM head.

Model description

Property	Value
Architecture	AutoJevForDecisionClassification (custom code, trust_remote_code=True)
Base model	Qwen/Qwen3.5-4B (Qwen3.5 text + vision backbones)
Parameters	~4B backbone + 255×2560 readout head
Readout	255 single-token answer codes (A … JT), initialized from the Qwen LM-head rows of those tokens
Decision head	Linear(hidden_size=2560, 255, bias=False) over the last prompt position
Dtype / context	bfloat16; max_input_length = 8192 (no silent truncation)
Options per question	1–255 (option_counts required per row; invalid option logits are masked)
Multimodal	Optional images (PIL, decoded) via the Qwen3 vision processor
Serving temperature	1.0 (post-processing softmax only; not used in CE loss)

Each input is rendered as one user message (state + instructions + options + an appended __unknown__ slot), then scored in one forward pass. The forward pass rejects KV-cache decoding and right padding (left padding is required).

Question types and output semantics

decision_utils.py implements the native _known() semantics: probabilities over known candidates are renormalized, and the unknown slot gets its own probability.

question.type	Candidates	Output
choice	1–254 named options (+unknown)	choice (argmax key), per-option probabilities, confidence
noul	true, false (+unknown)	noul = P(true) + 0.5 * P(unknown)
score	2–10 ordered descriptions (+unknown)	score = Σ i · P(i) plus legend and probabilities
—	—	abstained = True iff the raw argmax lands on unknown; unknown_probability is always returned

confidence = concentration · (1 − P(unknown)), where concentration = (n·p_max − 1)/(n − 1) (1 for single-candidate questions).

Quickstart

Standalone CLI (bundled model.py):

python model.py --checkpoint . \
  --input inputs.json \
  --device cuda:0

With transformers:

import torch
from transformers import AutoModel, AutoProcessor
from decision_utils import predict_batch

model = AutoModel.from_pretrained(
    "inclusionAI/LemonDecider",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda:0",
    attn_implementation="sdpa",
).eval()
processor = AutoProcessor.from_pretrained(
    "inclusionAI/LemonDecider", trust_remote_code=True
)
processor.tokenizer.padding_side = "left"

rows = [{
    "state": "我想申请退款。",
    "question": {
        "type": "choice",
        "instructions": "判断主要诉求",
        "criteria": {
            "refund": "退款相关",
            "delivery": "物流相关",
            "account": "账户相关",
        },
    },
}]

result = predict_batch(model, processor, rows)
print(result["answers"][0])
# {"type": "choice", "choice": "refund", "probabilities": {...},
#  "confidence": ..., "unknown_probability": ..., "abstained": ...}

Images are passed as decoded PIL.Image objects in row["images"]; paths/URLs are not fetched.

Training

Distilled via supervised fine-tuning from imajev-4b, a larger Jev-class decision model:

• Checkpoint: step 24 (6,144 examples seen)
• Loss: cross-entropy over the masked 255-way readout (supports hard option indices and soft teacher distributions)
• Dev set (n=574) accuracy@T=1: 0.7735 (best step: 3, 0.7805)

Temperature fit (not applied): an NLL scan over T ∈ [0.30, 5.00] on the dev set found T = 0.60 (NLL 0.56265, Brier 0.28261, vs 0.6016 / 0.29885 at T=1). The checkpoint still serves at T = 1.0; adopting T* is deliberately blocked until the public evaluation suite is re-evaluated (argmax is invariant at T* except at the noul decision boundary).

Intended use and limitations

• Intended for typed decision inputs (choice / noul / score) rendered through decision_utils.decision_messages(); prompt-layout divergence will degrade results.
• It is not a generative chat model: never use it for free-form text generation.
• Results come from a small internal dev split (n=574); validate on your own distribution before production use.
• Long inputs are rejected above 8,192 tokens instead of being silently truncated; handle your own context budget upstream.

Files

File	Purpose
model-00001/2-of-00002.safetensors(+index)	Weights (bfloat16)
configuration_autojev.py / modeling_autojev.py	Custom architecture (decision forward, option masking, hard/soft-target loss)
decision_utils.py	Prompt rendering, _known() scoring, batching (prepare_batch / predict_batch)
model.py	Standalone inference CLI
chat_template.jinja, processor_config.json	Qwen3-style chat template and image processor
sft_receipt.json	Training artifact receipt (step, dev metrics, temperature fit)

License

Apache-2.0 (see LICENSE / LICENSE.code).