Instructions to use hmntkmr/decision-q1-2b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use hmntkmr/decision-q1-2b with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
decision-q1-2b
A 2B on-device decision model for fast, private security triage and other typed decisions. It does not generate text. Given a state (JSON or text, optionally images) and a list of typed questions, it returns calibrated probabilities for every question in one forward pass.
It was built by converting Qwen3.5-2B into a decision model: the language-model head is replaced by a joint schema head that reads option markers from the final hidden states, the backbone is adapted with LoRA, and the outputs are temperature-calibrated.
What it answers
| Type | Output | Example |
|---|---|---|
choice |
probability over labelled options | "What should the security team do with this alert?" |
score |
distribution over an ordinal scale plus expected score | "How severe is this alert?" (0 to 2) |
noul |
probability of true / false | "Is this account likely compromised?" |
box |
zone, edge distributions and a bounding box (image input) | "Locate the brightness slider" |
Several questions, including several box queries, are answered in the same pass. Every answer carries its probabilities and a confidence value, so a caller can route low-confidence cases elsewhere.
Security use cases
The question types map directly onto security triage, where a fast local decision with a calibrated confidence is more useful than free text:
| Use case | Typical questions |
|---|---|
| Alert and incident triage | choice: ignore / monitor / step-up / isolate; score: severity |
| Phishing and spam screening | noul: is this message malicious? |
| Access and permission requests | choice: approve / deny / escalate; noul: needs human review |
| Action gating for agents | noul: is this tool call safe to run? |
| Content and abuse moderation | choice / noul over policy categories |
Why it suits security work:
- Runs entirely on your own hardware. Logs, alerts and messages are never sent to a hosted API.
- Calibrated probabilities. Set thresholds on confidence and send uncertain cases to an analyst instead of auto-acting.
- One pass, many questions. Verdict, severity and recommended action come from a single forward pass.
- Deterministic and auditable. The output is a probability table, not generated text, so it can be logged and replayed.
Not yet measured: this checkpoint has not been evaluated on dedicated security datasets. The closest public sets in the tables below are sms_spam (98.0%) and civil_comments (93.0%). Validate on your own labelled data before relying on it, and keep a human in the loop for destructive actions.
Model details
| Base model | unsloth/Qwen3.5-2B (hybrid Gated DeltaNet and attention, with vision tower) |
| Adaptation | LoRA, r = 32, alpha = 32, no dropout, on the language-model attention, DeltaNet and MLP projections |
| Decision head | joint schema head: width 512, 4 layers, 8 heads, feed-forward 2048, input hidden size 2048 |
| Layout | clef |
| Max sequence length | 2048 tokens |
| Image input | 280 x 280 |
| Merge | multi-objective weight-space soup, weights [prior, s2a, s2b] = [0.0, 0.7, 0.3] |
| Calibration | per-question-type temperature scaling, stored in unsloth_decision_config.json |
| Files | LoRA adapter (about 128 MB) and joint head (about 58 MB); base weights are downloaded separately |
Usage
Works on Linux and Windows with an NVIDIA CUDA GPU (about 6 GB VRAM with 4-bit). CPU-only and Apple Silicon are not supported by this runner. Requires an unsloth build that ships FastDecisionModel (tested with unsloth 2026.10.6).
huggingface-cli download hmntkmr/decision-q1-2b --local-dir ./decision-q1-2b
pip install -r ./decision-q1-2b/requirements.txt
python ./decision-q1-2b/decision_engine.py --benchmark
From Python:
import sys
sys.path.insert(0, "./decision-q1-2b")
from decision_engine import DecisionEngine
engine = DecisionEngine(model_dir="./decision-q1-2b/qwen3_5_2b_decision_lora", load_in_4bit=True)
result = engine.predict(
state={
"alert": "Impossible-travel sign-in followed by mailbox forwarding rule creation",
"account_role": "finance_admin",
"mfa_used": False,
"source_country": "new",
},
questions={
"action": {
"type": "choice",
"instructions": "What should the security team do?",
"criteria": {
"ignore": "Benign, no action needed",
"monitor": "Watch the account and collect more signals",
"step_up": "Force re-authentication and MFA",
"isolate": "Disable the account and open an incident",
},
},
"compromised": {"type": "noul", "instructions": "Is the account likely compromised?"},
"severity": {
"type": "score",
"instructions": "How severe is this alert?",
"criteria": ["0: Informational", "1: Needs review", "2: Critical, act now"],
},
},
)
print(result["answers"])
The runner can also serve the model over HTTP (--serve, POST /v1/systemone), offload the vision tower to CPU (--text-only), and export a merged 16-bit checkpoint (--merge-16bit DIR).
Memory
The backbone runs in 4-bit NF4 while the joint head and temperatures stay in FP32, which is what allows a 6 GB laptop GPU. --no-4bit loads the backbone at higher precision.
Evaluation
Numbers come from jev_summary.json in this repository, produced by the evaluation run of this checkpoint. Sample sizes are small (150 rows per domain suite, 100 per public set), so differences of a few points are within noise. No confidence intervals were computed.
record accuracy counts a row as correct only if all of its questions are right. ECE is expected calibration error (lower is better).
Domain suites (n = 150 each)
| Suite | Question accuracy | Record accuracy | ECE |
|---|---|---|---|
automation |
98.9 | 95.3 | 0.055 |
mind2web_ood |
98.3 | 96.7 | 0.028 |
mind2web |
96.7 | 94.0 | 0.037 |
typed_decisions |
74.3 | 30.0 | 0.086 |
Public text benchmarks (n = 100 each)
Mean accuracy over the 20 sets that were scored is 67.0%, mean ECE 0.136. clinc150 and ledgar were not scored.
| Set | Acc | ECE | Set | Acc | ECE | |
|---|---|---|---|---|---|---|
sms_spam |
98.0 | 0.040 | massive |
82.0 | 0.108 | |
civil_comments |
93.0 | 0.100 | paws |
79.0 | 0.104 | |
fever_evidence |
91.0 | 0.061 | arc_challenge |
78.0 | 0.082 | |
boolq |
85.0 | 0.084 | measuring_hate_speech |
74.0 | 0.130 | |
strategyqa_grounded |
83.0 | 0.061 | mnli |
68.0 | 0.071 | |
strategyqa_closed |
66.0 | 0.058 | banking77 |
61.0 | 0.172 | |
helpsteer2_verbosity |
59.0 | 0.191 | mmlu |
56.0 | 0.096 | |
yelp5 |
56.0 | 0.146 | go_emotions |
51.0 | 0.214 | |
chaosnli |
49.0 | 0.311 | sst5 |
47.0 | 0.161 | |
stsb |
38.0 | 0.186 | helpsteer2_helpfulness |
26.0 | 0.338 |
Vision
| Set | Acc | ECE | Mean box IoU |
|---|---|---|---|
pope_vision_noul |
82.5 | 0.121 | |
aokvqa_vision_choice |
80.0 | 0.154 | |
ai2d_vision_diagram |
72.5 | 0.136 | |
ocr_charsheet_box |
66.8 | 0.155 | 0.515 |
showui_box_gui |
56.1 | 0.121 | 0.283 |
Limitations
- Strong on agent-style decisions (automation, web actions) and weak on subjective or fine-grained ordinal judgements (
helpsteer2_helpfulness,sst5,stsb,go_emotions) and on disagreement-heavy data (chaosnli). - Multi-question records are much harder than single questions:
typed_decisionsis 74.3% per question but 30.0% per record. - Box localisation is coarse (mean IoU 0.28 to 0.52). It is a 3 x 3 zone plus 11-bin edge decoder, not a detector.
- Calibration is imperfect, notably on
helpsteer2_helpfulnessandchaosnli(ECE above 0.30). - Ordinal and option-count behaviour was tuned on the training distributions; expect drift on very different option counts or schemas.
- English only. Not evaluated on dedicated security data. Do not use it as the only control for blocking, account lockout, or financial, medical or legal actions; use the confidence values to escalate uncertain cases.
Files
| Path | Purpose |
|---|---|
qwen3_5_2b_decision_lora/adapter_model.safetensors, adapter_config.json |
LoRA adapter |
qwen3_5_2b_decision_lora/joint_head.safetensors, joint_head_config.json |
joint decision head |
qwen3_5_2b_decision_lora/unsloth_decision_config.json |
layout, max length, calibrated temperatures |
qwen3_5_2b_decision_lora/soup_manifest.json |
weight-soup recipe |
qwen3_5_2b_decision_lora/calibration_per_config.json |
per-dataset raw ECE and scale used during calibration |
qwen3_5_2b_decision_lora/jev_summary.json |
evaluation results quoted above |
qwen3_5_2b_decision_lora/ tokenizer and processor files |
tokenizer, chat template, image processor |
decision_engine.py |
local runner and HTTP server |
gui_qwen3_5_2b_decision_studio.py |
optional Tk desktop studio (needs decision_engine.py) |
requirements.txt, CHANGELOG.md |
dependencies and release notes |
License
Apache-2.0 for the adapter, head and runner. The base model and its tokenizer keep their own license; see the base model card. unsloth is a separate dependency under its own license.
Citation
@misc{decision_q1_2b,
title = {decision-q1-2b},
author = {Hemant Kumar},
year = {2026},
howpublished = {\url{https://huggingface.co/hmntkmr/decision-q1-2b}}
}
Built by Hemant Kumar.
- Web: bloomeex.org and iamhemant.in
- Follow me on X: x.com/iamhemantindia
- Downloads last month
- -