File size: 3,316 Bytes
5c714e4 e20b23a 5c714e4 e20b23a b40f004 e20b23a 9d28f67 e20b23a b40f004 e20b23a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | ---
license: apache-2.0
license_link: LICENSE
base_model: vllm-sr/d3-mini
pipeline_tag: zero-shot-classification
library_name: coreai
tags:
- vllm-sr
- qwen3_5
- decision
- classification
- system-one
- decision-model
- apple
- coreai
- ane
- apple-neural-engine
- fp16
- tinkerbox
---
# d3-mini on Apple Neural Engine
Core AI conversion of [vllm-sr/d3-mini](https://huggingface.co/vllm-sr/d3-mini) for typed yes/no, choice, and score decisions over text and images, using [tinkerbox-ane](https://github.com/WeGoToMars/tinkerbox).
_This model has 4B parameters, for a samller 2B parameter model from the same family see [wegotomars/d3-nano](https://huggingface.co/wegotomars/d3-nano-CoreAI-ANE)._
d3-mini accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️.
Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13.
## Usage
Install `tinkerbox-ane>=2.1.0`. Loading the model downloads only this Mac's bundle: its `checkpoint` folder carries the upstream code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses.
```python
import tinkerbox_ane
with tinkerbox_ane.load("d3-mini") as model:
result = model.system_one("I was charged twice. Please refund me today.", {
"refund": {"type": "noul", "instructions": "Is this a refund request?"},
})
print(result)
```
Images can accompany text, or be used alone:
```python
from PIL import Image
import tinkerbox_ane
with tinkerbox_ane.load("d3-mini") as model, Image.open("receipt.jpg") as image:
result = model.system_one(None, {
"receipt": {"type": "noul", "instructions": "Is this a receipt?"},
}, images=[image])
print(result)
```
d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with `SequentialQuestionsWarning`. Each result's `usage` reports its tokens, Neural Engine time, and counted FLOPs.
## Supported architectures
| Architecure | SoC name |
| --- | --- |
| `h16g` | Apple M4 |
| `h16s` | Apple M4 Pro |
| `h16c` | Apple M4 Max |
| `h17g` | Apple M5 |
| `h17s` | Apple M5 Pro |
| `h17c` | Apple M5 Max |
| `h17d` | Apple M5 Ultra |
| `h18g` | Apple M6 |
## Benchmarks
Measured October 21, 2026 on Apple M5 Pro (`h17s`, 24 GiB), macOS 27.0.1, one question, warm.
| Metric | Result |
|---|---:|
| Top-answer agreement with FP32 text reference | 100% |
| Absolute probability drift, mean / maximum | 0.0008 / 0.025 |
| Official BF16 drift from FP32, mean / maximum | 0.0006 / 0.018 |
| 91-token text prompt | 85 ms |
| 250-token text prompt | 171 ms |
| 506-token text prompt | 354 ms |
| 4,090-token text prompt | 3.07 s |
| 7,998 token text prompt | 8.23 s |
| 26,063-token text prompt | 51.4 s |
| 640 × 480 image prompt | 0.64 s |
| ANE energy, 91-token prompt | 317 mJ |
Upstream checkpoint: `vllm-sr/d3-mini@bd0090dd758d32ba59ef5c749910a8e87fc4391f`. Weights, upstream code, and converted assets retain the [Apache License 2.0](LICENSE). See [NOTICE](NOTICE).
|