edge/README.md

---
license: apache-2.0
base_model: vllm-sr/d3-edge
library_name: coreml
pipeline_tag: zero-shot-classification
tags:
- coreml
- mlx
- apple-silicon
- decision-model
- vision
- video
---

# d3-edge — Core ML and MLX

[vllm-sr/d3-edge](https://huggingface.co/vllm-sr/d3-edge) (0.6B, the camera-stream model of [Decision 3.0](https://huggingface.co/collections/vllm-sr/decision-30) by the vLLM Semantic Router team) for Apple silicon: text, image and video requests on device, as Core ML packages and as MLX weights. Same contract as upstream: one forward pass per question, the 255-way FP32 readout at the last prompt token, a probability for every option, no text generated.

## Files

| File | What |
|---|---|
| `d3_E256` / `d3_E640` / `d3_E1536` / `d3_E4096.mlpackage` | Core ML text graph (fp16 compute, int8 weights) per prompt-length bucket (prompts up to 4,096 tokens); input = token embeddings with image/video tokens spliced in |
| `vision_P320` / `vision_P1536` / `vision_P3072.mlpackage` | Core ML vision tower (fp32) per patch bucket; video frames go through it one frame pair at a time (block mask) |
| `embeddings.f16`, `pos_embed.f32`, `readout.f32`, `tokenizer.json`, `config.json`, `coreml_config.json` | host-side tables and config |
| `mlx/` | the upstream bf16 weights in the layout [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) loads (`model.safetensors`, `config.json`) |
| `python/` | Python hosts: `d3_coreml_text.py`, `d3_coreml_image.py`, `d3_coreml_video.py` (coremltools) and `d3_mlx_video.py` (MLX, text and video) |

## Checked against upstream

Same weights as upstream (revision `fe71f8e1`). Answers that differ from the upstream PyTorch runtime (bf16, MPS), same requests:

| | Core ML | MLX |
|---|---|---|
| Text (typed-decisions TEST, 20 requests, 100 questions) | 1 | 0 |
| Images (8 head-camera parkour frames × 4 questions) | 0 / 32 | — |
| Video (2-second parkour windows × 8 yes/no questions, 192 answers) | 3 | 3 |

Speed on an M5 Pro (24 GB), one request at a time:

| | Core ML (`CPU_AND_GPU`) | MLX |
|---|---|---|
| per text question | ~20 ms | ~16 ms |
| per 2-second video look, 8 questions | 440 ms | **112 ms** |
| upstream PyTorch (MPS), same look | 651 ms | |

MLX is faster on video because it runs all of a request's questions as one batched forward at their exact lengths; the Core ML graph runs one padded pass per question. Neither places ops on the Neural Engine: with `CPU_AND_NE`, Core ML falls back to the CPU for every op of this architecture.

## Python

```python
# Core ML
from d3_coreml_video import D3CoreMLVideo   # d3_coreml_text.D3CoreMLText, d3_coreml_image.D3CoreMLImage
m = D3CoreMLVideo("<vllm-sr/d3-edge snapshot>", "<this folder>", max_pixels=100_352)
m.system_one(state="A short clip from a head camera.", videos=[frames_2fps], questions={"high": {"type": "noul", "instructions": "Is the camera high above the ground?"}})

# MLX (needs mlx-vlm >= 0.7.8)
from d3_mlx_video import D3MLXVideo
m = D3MLXVideo("<vllm-sr/d3-edge snapshot>", "<this folder>/mlx", max_pixels=100_352)
m.system_one(state="...", questions={...}, videos=[frames_2fps])   # or videos=None for text
```

Both hosts build prompts with upstream's own runtime (`d3_runtime.py` in the upstream snapshot, run without weights), so prompts match upstream exactly. Note for MLX: mlx-vlm's Qwen3.5 builds layer kinds from `full_attention_interval`, but d3-edge's full-attention layers are 2, 5, 8, 9, 10 and 11 (`layer_types` in `config.json`); `d3_mlx_video.py` patches the layer constructor from `layer_types` before loading, and loads strictly. Without that, half the layers load no weights.

License: Apache-2.0, as upstream.