Card for FastFlowLM 1.0.11: images, 32,768-token input, current accuracy and speed

MANIFEST.sha256

@@ -1,6 +1,6 @@
 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
 b1f8d5da5bd599029de424b307a979abc0bfc60500e95018c6cd5c8caf3de970  NOTICE
-3075a15638f882899334faee638fd64ca333c3477a45a8810d5381360195266c  README.md
+944edd2a82fb0ec05991f7ae5b7515111f019c64fb9cfc05a8673154a3553f43  README.md
 273d8e0e683b885071fb17e08d71e5f2a5ddfb5309756181681de4f5a1822d80  chat_template.jinja
 7e59d9fda4eb32509e0f306a826e0a64cf4ccb5099f446550f1a58b9f17fb833  config.json
 30b6095b945d5a93697868afdf9be9123ea892085fbaff3078112c48da20dc6b  decision_config.json

README.md

@@ -13,9 +13,10 @@ tags:
 # D3-Lite-NPU2
 
 [vllm-sr/d3-lite](https://huggingface.co/vllm-sr/d3-lite) `v3.1.3` (Decision 3.0, the decision model of vLLM Semantic
-Router) for AMD Ryzen AI NPUs, served by [FastFlowLM](https://github.com/ROCm/FastFlowLM) 1.0.10+.
-It answers *System One* requests -- choice, yes / no and score questions about a state, with a probability for every
-option -- over the same `POST /v1/systemone` API as the upstream package.
+Router) for AMD Ryzen AI NPUs, served by [FastFlowLM](https://github.com/ROCm/FastFlowLM) 1.0.11+.
+It answers *System One* requests -- choice, yes / no and score questions about a state, optionally with images, with a
+probability for every option -- over the same `POST /v1/systemone` API as the upstream package. Up to 32,768 input
+tokens per question.
 
 ```shell
 flm serve --decision d3:lite          # optionally next to a chat model: flm serve qwen3.5:4b --decision d3:lite
@@ -24,38 +25,64 @@ curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json"
        "criteria": {"returns": "Refunds and replacements", "billing": "Payments", "general": null}}}}'
 ```
 
+Images go in an `images` list of base64 data URLs (PNG, JPEG or WebP; each at most 8 MB and 16 MP, resized to at most
+1.6 MP), shared by every question of the request:
+
+```shell
+curl -sLo panda.png https://raw.githubusercontent.com/ROCm/FastFlowLM/main/src/tb_files/panda.png
+printf '{"questions": {"animal": {"type": "choice", "instructions": "Which animal is in the photo?",
+  "criteria": {"dog": "A dog", "panda": "A panda", "cat": "A cat"}}},
+  "images": ["data:image/png;base64,%s"]}' "$(base64 -w0 panda.png)" > image_request.json
+curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" -d @image_request.json
+# ... "choice":"panda", "panda":0.999 ...
+```
+
+More in the [FastFlowLM d3 guide](https://fastflowlm.com/docs/models/d3/).
+
 ## What was changed
 
 - **Weights:** the Qwen3.5 text tower quantized to FastFlowLM's Q4NX format: every projection Q4_K with
   GPTQ-calibrated rounding (calibration: 300 Decision Index prompts, none of them in the evaluation
   sets below), token embedding Q8_0, norms and the Gated DeltaNet parameters unquantized. `vision_weight.q4nx` holds
-  the vision tower for a later FastFlowLM release; FastFlowLM 1.0.10 answers text requests.
+  the vision tower (BF16), used from FastFlowLM 1.0.11 for requests with images.
 - **Unchanged:** `readout.safetensors`, `decision_config.json`, `tokenizer.json`, `tokenizer_config.json`,
   `chat_template.jinja` and `LICENSE` are the upstream files byte for byte (`MANIFEST.sha256`).
 - **Same contract:** prompts, token ids and answer objects are byte-identical to the upstream runtime (checked on every
-  FastFlowLM build against 1,308 upstream-rendered requests).
+  FastFlowLM build against 1,365 upstream-rendered requests, 57 of them with images); image grids and input token
+  counts equal upstream's, and the image pixels are within 2 grey levels of the upstream processor's.
 
 ## Accuracy (against the upstream BF16 model)
 
 | | this package on the NPU | upstream BF16 |
 |---|---|---|
-| Decision Index public-suite subset, gold accuracy (6,824 questions, 35 benchmarks) | 60.6% | 61.2% |
-| Top-1 agreement with BF16 on the same questions | 90.5% | -- |
+| Decision Index public-suite subset, gold accuracy (6,824 questions, 35 benchmarks) | 60.5% | 61.2% |
+| Top-1 agreement with BF16 on the same questions (7,226) | 90.6% | -- |
+| Decision Index public vision suite, gold accuracy (600 questions, 10 benchmarks) | 61.7% | 60.2% |
+| Top-1 agreement with BF16 on the same image questions | 84.5% | -- |
 
-Paired gold accuracy, upstream BF16 minus this package: +0.59 pt (95% CI [-0.00, +1.18] pt).
+Paired gold accuracy, upstream BF16 minus this package: text +0.67 pt (95% CI [+0.08, +1.27] pt); images -1.50 pt
+(95% CI [-3.92, +0.92] pt). The vision suite leaves out Winoground (gated); per-benchmark numbers are on the
+[FastFlowLM d3 benchmark page](https://fastflowlm.com/docs/benchmarks/d3_results/).
 
-## Speed (AMD Ryzen AI MAX+ PRO 395 (Strix Halo, XDNA 2 NPU), 128 GB, Ubuntu 24.04, XRT)
+## Speed (AMD Ryzen AI MAX+ PRO 395 (Strix Halo, XDNA 2 NPU), 128 GB, Ubuntu 24.04, XRT, FastFlowLM 1.0.11)
 
 | request | median latency |
 |---|---|
-| 1 question, 128 tokens | 112 ms |
-| 1 question, 330 tokens | 159 ms |
-| 1 question, 1,024 tokens | 518 ms |
-| 1 question, 4,096 tokens | 3652 ms |
-| 4 questions, 330 tokens each | 574 ms |
-| 16 questions, 330 tokens each | 2198 ms |
-
-Prompts over 4,096 tokens are answered with `max_length_exceeded` (nothing is truncated).
+| 1 question, 128 tokens | 109 ms |
+| 1 question, 330 tokens | 151 ms |
+| 1 question, 1,024 tokens | 430 ms |
+| 1 question, 4,096 tokens | 1,587 ms |
+| 1 question, 16,384 tokens | 8,425 ms |
+| 1 question, 32,768 tokens | 22,696 ms |
+| 4 questions, 330 tokens each | 536 ms |
+| 16 questions, 330 tokens each | 2,067 ms |
+| 1 question + a 640 x 480 image (398 tokens) | 393 ms |
+| 1 question + a 1.6 MP image (1,698 tokens) | 1,529 ms |
+| 8 questions + one 1.6 MP image (one image encoding) | 6,368 ms |
+
+From 1,536 tokens attention runs on the NPU's attention kernel (below that the CPU is faster). Prompts over 32,768
+tokens are answered with `max_length_exceeded` (nothing is truncated). Image rows were measured over HTTP; the others
+at the engine (HTTP adds about a millisecond).
 
 ## License