| --- |
| license: apache-2.0 |
| license_link: LICENSE |
| base_model: vllm-sr/d3-mini |
| pipeline_tag: zero-shot-classification |
| library_name: coreai |
| tags: |
| - vllm-sr |
| - qwen3_5 |
| - decision |
| - classification |
| - system-one |
| - decision-model |
| - apple |
| - coreai |
| - ane |
| - apple-neural-engine |
| - fp16 |
| - tinkerbox |
| --- |
| |
| # d3-mini on Apple Neural Engine |
|
|
| Core AI conversion of [vllm-sr/d3-mini](https://huggingface.co/vllm-sr/d3-mini) for typed yes/no, choice, and score decisions over text and images, using [tinkerbox-ane](https://github.com/WeGoToMars/tinkerbox). |
|
|
| _This model has 4B parameters, for a samller 2B parameter model from the same family see [wegotomars/d3-nano](https://huggingface.co/wegotomars/d3-nano-CoreAI-ANE)._ |
|
|
| d3-mini accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️. |
|
|
| Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13. |
|
|
| ## Usage |
|
|
| Install `tinkerbox-ane>=2.1.0`. Loading the model downloads only this Mac's bundle: its `checkpoint` folder carries the upstream code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses. |
|
|
| ```python |
| import tinkerbox_ane |
| |
| with tinkerbox_ane.load("d3-mini") as model: |
| result = model.system_one("I was charged twice. Please refund me today.", { |
| "refund": {"type": "noul", "instructions": "Is this a refund request?"}, |
| }) |
| print(result) |
| ``` |
|
|
| Images can accompany text, or be used alone: |
|
|
| ```python |
| from PIL import Image |
| import tinkerbox_ane |
| |
| with tinkerbox_ane.load("d3-mini") as model, Image.open("receipt.jpg") as image: |
| result = model.system_one(None, { |
| "receipt": {"type": "noul", "instructions": "Is this a receipt?"}, |
| }, images=[image]) |
| print(result) |
| ``` |
|
|
| d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with `SequentialQuestionsWarning`. Each result's `usage` reports its tokens, Neural Engine time, and counted FLOPs. |
|
|
| ## Supported architectures |
| | Architecure | SoC name | |
| | --- | --- | |
| | `h16g` | Apple M4 | |
| | `h16s` | Apple M4 Pro | |
| | `h16c` | Apple M4 Max | |
| | `h17g` | Apple M5 | |
| | `h17s` | Apple M5 Pro | |
| | `h17c` | Apple M5 Max | |
| | `h17d` | Apple M5 Ultra | |
| | `h18g` | Apple M6 | |
|
|
| ## Benchmarks |
|
|
| Measured October 21, 2026 on Apple M5 Pro (`h17s`, 24 GiB), macOS 27.0.1, one question, warm. |
|
|
| | Metric | Result | |
| |---|---:| |
| | Top-answer agreement with FP32 text reference | 100% | |
| | Absolute probability drift, mean / maximum | 0.0008 / 0.025 | |
| | Official BF16 drift from FP32, mean / maximum | 0.0006 / 0.018 | |
| | 91-token text prompt | 85 ms | |
| | 250-token text prompt | 171 ms | |
| | 506-token text prompt | 354 ms | |
| | 4,090-token text prompt | 3.07 s | |
| | 7,998 token text prompt | 8.23 s | |
| | 26,063-token text prompt | 51.4 s | |
| | 640 × 480 image prompt | 0.64 s | |
| | ANE energy, 91-token prompt | 317 mJ | |
|
|
| Upstream checkpoint: `vllm-sr/d3-mini@bd0090dd758d32ba59ef5c749910a8e87fc4391f`. Weights, upstream code, and converted assets retain the [Apache License 2.0](LICENSE). See [NOTICE](NOTICE). |
|
|
