Qwen3.5-4B-Router-1M

A 4B open decider for one job: what should this agent do next, and which tool should it call? It returns a calibrated probability for each option, not free text.

Qwen3.5-4B + an Unsloth decision head (LoRA r=16), trained on 17,600 rows with source labels only. No Jev answers were used in training.

Results

Scored on the held-out test split, 480 rows from 4 task families. Jev 1.13's answers (TypeSafe) are the ones stored with those rows. None of these rows shares a group or a state with the 976,219 train and validation rows of the merged 1M set.

Task	Rows	This model	Jev 1.13	Always the most common answer
Next action, agent	120	85.0%	74.2%	50.0%
Next action, customer service	120	83.3%	75.8%	50.0%
Tool choice, agent	120	90.8%	82.5%	14.2%
Tool choice, customer service	120	86.7%	90.0%	23.3%
All 4	480	86.5%	80.6%

Difference: +5.8 points, 95% interval +1.7 to +10.1 (bootstrap over 364 groups). This model alone was right on 58 rows, Jev alone on 30.

Limits

Not a clinical model. Training and test rows are customer-service and software-agent traces. It has not been tested on clinical agents and must not decide anything about a patient.
Home advantage. The labels are what the source trace did next or a tool-call parse, not a human verdict. This model learned from those same labels and Jev did not, so read the table as agreement with the source.
Not a trace judge. On trace_disposition it gave one answer on all 120 rows, 34.2% against a majority class of 43.3%. trace_outcome is 47.5% against 41.7%. Don't use it for those.
Long rows untested. The test covers states up to 13,000 characters, about 70% of these families' test rows, and the model reads at most 4,096 tokens. Jev scores a little higher on the longer rows.
Five more trained families had a single class in this test, so their 100% says nothing and is left out.
Training stopped on a time budget at 0.67 of an epoch, on one A100 in 1.9 hours.

Use

from unsloth import FastDecisionModel

model, tokenizer = FastDecisionModel.from_pretrained("MaziyarPanahi/Qwen3.5-4B-Router-1M", max_seq_length=4096, load_in_4bit=False)
FastDecisionModel.for_inference(model)
question = {"decision": {"type": "choice",
    "instructions": "Which available tool should be called next?",
    "criteria": {"get_order_status": "Call `get_order_status`.",
                 "transfer_to_human_agent": "Call `transfer_to_human_agent`."}}}
state = '{"available_tools": [{"name": "get_order_status"}, {"name": "transfer_to_human_agent"}], '\
        '"conversation": [{"role": "user", "content": "I would like to speak to a person please."}]}'
print(FastDecisionModel.predict(model, tokenizer, state, question))

Data

Trained on the source labels of four public Jev datasets. Jev's answers in those datasets were not used in training.

Credit

Base model: Qwen3.5-4B by the Qwen team, Apache-2.0. Recipe: Unsloth FastDecisionModel.
The rows behind those datasets come from NVIDIA's Nemotron-SFT-Agentic-v2 and Open-SWE-Traces, both CC BY 4.0. They were turned into decision questions with one label each. See those datasets for their terms.