Update weights and benchmark numbers

README.md

@@ -31,7 +31,6 @@ tags:
 - calibration
 - system-one
 - multimodal
-- decision-model
 ---
 
 <div align="center">
@@ -50,18 +49,19 @@ tags:
 # d1-3B
 
 d1-3B is a 3B parameter **decision model** built on [LFM2.5-VL-3B](https://huggingface.co/LiquidAI/LFM2.5-VL-3B).
-You give it a state (text, JSON, images, or a mix) and a set of questions. It returns calibrated,
-typed answers in **one forward pass with zero output tokens**.
+You give it a state (text, JSON, images, or a mix) and a set of named questions. It returns calibrated,
+typed answers in **one forward pass with zero output tokens**: every answer is read directly from the
+model's distribution over the options, with no generation and no parsing.
 
-- **Best decision model under 10B on the Decision Index 0.2.1**: 48.57, ahead of every 4B and 9B model
-  and of Decider 35B-A3B (47.11).
+- **Best decision model under 10B on the Decision Index 0.2.1** (leaderboard of 2026-09-28): 48.57, ahead
+  of every 4B and 9B model and of Decider 35B-A3B (47.11).
 - **Multimodal**: images and text in the same state. It scores 74.1 on 11 public image benchmarks
   (LFM2.5-VL-3B: 73.9).
 - **Fast**: 8 ms a decision on an NVIDIA RTX 4090, 9 ms on an AMD MI325X, 30 ms on an Apple M5 Pro.
 
-Find more information about open d1 in our [blog post](https://www.liquid.ai/blog/open-d1).
+Find more information about open d1 in our [blog post](www.liquid.ai/blog/open-d1).
 
-![image](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/r2H7UlZ_m48SYAhZWIHiu.png)
+![image](https://cdn-uploads.huggingface.co/production/uploads/644249b08443bce4c9890a0f/ZNvmQHJ0IRhO9Gvg52jPr.png)
 
 > [!NOTE]
 > 💻 **Demos**: Try d1-3B in a Hugging Face space without any setup:
@@ -78,9 +78,11 @@ Find more information about open d1 in our [blog post](https://www.liquid.ai/blo
 d1-3B is a multimodal decision model with the following features:
 
 - **Total parameters**: 3.12B
+- **LM backbone**: LFM2.5-2.6B, 30 layers (22 double-gated short convolution blocks + 8 GQA)
 - **Vision encoder**: SigLIP2 NaFlex shape-optimized 400M
 - **Context length**: 32,768 tokens
 - **Vocabulary size**: 128,000
+- **Output**: no generated tokens. Each answer is a probability distribution over the question's options.
 
 We recommend d1-3B wherever a pipeline needs a yes/no, a pick from named options, or a rating:
 routing and triage, moderation, intent and topic classification, extraction checks, reranking, LLM-as-a-judge
@@ -97,47 +99,36 @@ pip install "transformers>=5.14" torch torchvision pillow
 The model ships its own code, so load it with `trust_remote_code=True`:
 
 ```python
+import io
+import urllib.request
+
 import torch
+from PIL import Image
 from transformers import AutoModel
-from transformers.image_utils import load_image
 
 device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu"
-dtype = torch.float32 if device == "cpu" else torch.bfloat16
-model = AutoModel.from_pretrained("LiquidAI/d1-3B", trust_remote_code=True, dtype=dtype).to(device)
+model = AutoModel.from_pretrained("LiquidAI/d1-3B", trust_remote_code=True,
+                                  dtype=torch.float32 if device == "cpu" else torch.bfloat16).to(device)
 
-# Text: several named questions over one state, answered in one pass
+# Several named questions over one text state, answered in one pass
 questions = {
-    "refund": {
-        "type": "noul",
-        "instructions": "Is the customer asking for a refund?",
-    },
-    "team": {
-        "type": "choice",
-        "instructions": "Which team should handle this?",
-        "criteria": {
-            "billing": "Charges, refunds, invoices",
-            "technical": "App or site faults",
-            "fraud": "Suspected unauthorised use",
-        },
-    },
-    "urgency": {
-        "type": "score",
-        "instructions": "How urgent is this?",
-        "criteria": ["Can wait", "Today", "Blocking the customer now"],
-    },
+    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
+    "team": {"type": "choice", "instructions": "Which team should handle this?",
+             "criteria": {"billing": "Charges, refunds, invoices", "technical": "App or site faults",
+                          "fraud": "Suspected unauthorised use"}},
+    "urgency": {"type": "score", "instructions": "How urgent is this?",
+                "criteria": ["Can wait", "Today", "Blocking the customer now"]},
 }
 print(model.system_one("I was charged twice this month, please refund one of them.", questions))
 
-# Image: the photo is the whole state
-image = load_image("http://images.cocodataset.org/val2017/000000039769.jpg")  # two cats on a sofa
-cats = {
-    "type": "choice",
-    "instructions": "How many cats are there?",
-    "criteria": {"one": "One", "two": "Two", "more": "Three or more"},
-}
-print(model.system_one(None, {"cats": cats}, images=[image]))
+# An image as the whole state
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"  # two cats on a sofa
+photo = Image.open(io.BytesIO(urllib.request.urlopen(url).read()))
+print(model.system_one(None, {"cats": {"type": "choice", "instructions": "How many cats are there?",
+                                       "criteria": {"one": "One", "two": "Two", "more": "Three or more"}}},
+                       images=[photo]))
 
-# Batch: many requests, packed together with no padding
+# Many requests, packed together with no padding
 tickets = ["Where is my parcel? It was due Monday.", "The app crashes when I open settings."]
 print(model.system_one_batch([(t, {"team": questions["team"]}) for t in tickets]))
 ```
@@ -163,32 +154,16 @@ Each call returns `{"answers": {name: answer}, "usage": {"input_tokens": n, "out
 
 ## ⚡ Speed
 
-Warm calls, one request at a time: a single question, three questions over one state, a 3.4k-token
-state and a 384 px image. The last column is throughput with 64 states packed into one pass.
-
-### Edge Inference
-
-We measure latency on an Apple M5 Pro and, in collaboration with NVIDIA, on an NVIDIA Jetson AGX Thor,
-a Jetson AGX Orin 64 GB and a Jetson Orin Nano.
-
-| | one question | 3 questions, one pass | 3.4k-token state | 384 px image | 64 states, packed |
-|---|---:|---:|---:|---:|---:|
-| Apple M5 Pro (`mps`) | 30 ms | 41 ms | 640 ms | 62 ms | 78 / s |
-| NVIDIA Jetson AGX Thor | 16 ms | 20 ms | 220 ms | 35 ms | 262 / s |
-| NVIDIA Jetson AGX Orin 64 GB | 26 ms | 35 ms | 560 ms | 83 ms | 110 / s |
-| NVIDIA Jetson Orin Nano | 50 ms | 73 ms | 1640 ms | 202 ms | 38 / s |
-
-### GPU Inference
-
-We measure latency on an NVIDIA RTX 4090 and an AMD MI325X, in bf16, median of 20 runs.
+Warm calls in bf16, median of 20 runs:
 
 | | one question | 3 questions, one pass | 3.4k-token state | 384 px image | 64 states, packed |
 |---|---:|---:|---:|---:|---:|
-| NVIDIA RTX 4090 | 8 ms | 21 ms | 102 ms | 17 ms | 475 / s |
-| AMD MI325X | 9 ms | 14 ms | 44 ms | 18 ms | 1,106 / s |
+| NVIDIA RTX 4090 | **8.0 ms** | 21 ms | 102 ms | 17 ms | 475 / s |
+| AMD MI325X | 9.2 ms | 14 ms | 44 ms | 18 ms | 1,106 / s |
+| Apple M5 Pro (`mps`) | 30 ms | 41 ms | 0.64 s | 62 ms | 78 / s |
 
-On NVIDIA GPUs, `model.compile(mode="reduce-overhead")` runs single questions as CUDA graphs (the RTX 4090
-row uses it). Without it, a single question takes 16 ms. The first call with a new shape pays for kernel
+On NVIDIA GPUs, `model.compile(mode="reduce-overhead")` runs single questions as CUDA graphs. The RTX 4090
+row uses it; without it a single question takes 16 ms. The first call with a new shape pays for kernel
 selection or compilation, so warm up the shapes you serve.
 
 ## 📊 Performance
@@ -197,7 +172,10 @@ All results are on public benchmarks.
 
 ### Decision Index 0.2.1
 
-We scored d1-3B with the official scorer (not a leaderboard submission). All other rows come from the public leaderboard v0.2.1.
+The [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) casts 38 public
+benchmarks as 120,226 typed decisions, scored as chance-corrected skill. We scored d1-3B with the official
+scorer; it is not a leaderboard submission. All other rows come from the public leaderboard
+(snapshot 2026-09-28). Sizes are served parameters.
 
 | Model | Size | Decision Index | Knowledge | Language | Retrieval | Tools | Arts |
 |---|---:|---:|---:|---:|---:|---:|---:|
@@ -214,7 +192,8 @@ We scored d1-3B with the official scorer (not a leaderboard submission). All oth
 
 ### Benchmarks as decisions
 
-Besides the Decision Index, we added a few other internal evaluations based on public benchmarks.
+Evaluation splits are cast as typed decisions, with the same rows for every model. These scores follow the
+decision protocol, not each benchmark's own.
 
 | Benchmark | d1-3B | Decider 4B | Decider 2B |
 |---|---:|---:|---:|
@@ -255,6 +234,13 @@ compared with the base model:
 
 With the images removed, the same questions score 45.1, so the answers come from the images.
 
+## ⚠️ Limitations
+
+- **Knowledge-heavy questions**: larger models stay ahead on hard knowledge (for example MMLU-Pro and GPQA).
+- **Thresholds**: calibrate any threshold on data from your own deployment before gating on it.
+- **No text output**: d1-3B answers questions over fixed options. Use [LFM2.5-VL-3B](https://huggingface.co/LiquidAI/LFM2.5-VL-3B)
+  for open-ended generation.
+
 ## 📬 Contact
 
 - Got questions or want to connect? [Join our Discord community](https://discord.com/invite/liquid-ai)
@@ -268,7 +254,7 @@ With the images removed, the same questions score 45.1, so the answers come from
   title   = {Open d1: Edge decision models for text, vision, and audio},
   journal = {Liquid AI Blog},
   year    = {2026},
-  note    = {https://www.liquid.ai/blog/d1-open},
+  note    = {www.liquid.ai/blog/open-d1},
 }
 ```
 
@@ -279,4 +265,4 @@ With the images removed, the same questions score 45.1, so the answers come from
   journal = {arXiv preprint arXiv:2511.23404},
   year    = {2025}
 }
-```
+```

assets/d1-3b-smash.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08f69b1a67bf32cd18fad32b5872519b580be9e94bafbe26a13d5cf0aaa49c1f
+size 53062187

assets/ood_tasks.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b491dfa851157e12ba2bee33a394c9565408247cf0004e493730622c62878e42
+size 8931625