d1-2B gestures on Core AI (GPU and Neural Engine)
A Core AI conversion of LiquidAI/d1-2B by Liquid AI, for the d1 gestures webcam demo: one frame in, hand gesture, facial expression, hand box and face box out, in one forward pass, on the GPU or the Neural Engine through Apple's Core AI (macOS 27, iOS / iPadOS 27).
The questions are inputs, not part of the graph. The functions take the frame plus the prompt (its token embeddings, its
attention mask, the answer positions and the answer tokens' readout rows), so other questions need no re-export, as long as
the prompt fits: 640 tokens with the image, 16 answer positions, 1,152 answer tokens (the demo's four questions use 578, 12 and
1,021). code/d1_common.py (prompt_inputs) builds them from code/questions.py and embed_tokens_fp16.npy.
| File | What it is |
|---|---|
d1_gpu_prompt_fp16.aimodel/ |
the function for the GPU, fp16, 3.1 GB |
d1_ane_prompt_fp16.aimodel/ |
the same function re-authored for the Neural Engine, fp16, 3.1 GB (Mac) |
d1_ane_prompt_split_fp16/ |
the Neural Engine function in 4 pieces (part0β¦3.aimodel, run in order), for iPhone and iPad |
embed_tokens_fp16.npy |
d1's token embedding table, fp16 [128000, 2048], to build the prompt inputs (500 MB) |
calibration.json |
Liquid AI's calibration, unchanged from LiquidAI/d1-2B |
code/ |
the export (d1_torch.py, d1_ane.py, export_*.py), the parity and speed checks, the host readout, the Swift worker |
LICENSE |
the LFM Open License v1.0, as distributed with the original weights |
d1_gpu_fp16.aimodel/, d1_ane_fp16.aimodel/, prompt.json |
the first version, with the four questions baked in; superseded, kept for old checkouts |
Function
image float32 [1, 3, 352, 640] the 640x360 frame after the d1 image processor (resize to 640x352, (x/255 - 0.5)/0.5)
text fp16 [420, 2048] the prompt's token embeddings, without the 220 image positions (tokens 5β¦224)
attend fp16 [640, 640] 1 where query token i may attend to token j (d1's decision mask, padding masked out)
slots fp16 [16, 640] one-hot: which token each answer position reads
head fp16 [1152, 2048] the embedding rows of the answer tokens, the readout
logits [16, 1152] one row per answer position, one column per answer token (pad rows/columns ignored)
The demo's 12 rows are the answer positions: hand gesture (1), facial expression (1), hand box (present + 4 edges), face box
(present + 4 edges). Turning them into answers (softmax over each option's tokens, Liquid AI's calibration, box medians) is
d1's own readout; code/d1_common.py runs it through mlx-vlm's d1 code, so the answers
have exactly the format of model.system_one(...). The iOS app has the same readout in Swift.
Run it with the Swift worker (cd code/swift && swift build -c release), the prompt inputs written as raw fp16 files plus a
shapes.json (code/coreai_backend.py does this):
.build/release/d1worker --model ../../d1_ane_prompt_fp16.aimodel --compute ane --prompt /path/prompt # or the GPU model, --compute gpu
# stdin, one JSON a line: {"pixels": "/path/frame.f32"} (raw float32 [1,3,352,640]) -> {"logits": [...], "ms": ...}
or from Python with coreai.runtime (code/coreai_backend.py is the backend the demo uses).
Why 4 pieces on iOS
iOS's Neural Engine compiler fails on the whole 3.1 GB function ("Could not retrieve data from weight file") and Core AI then
quietly runs it on the GPU. The split runs layers 0β6 with the vision tower, 7β12, 13β19, and 20β23 with the readout, passing the
LM stream from one to the next; each piece takes attend, the first also image and text, the last also slots and head.
On the Mac the pieces are as fast as the whole.
Measured (MacBook Pro M5 Max, macOS 27.3)
Parity with the MLX reference (LiquidAI/d1-2B-MLX-8bit, system_one) on 9 frames; a frame passes with the same top choice
for hand and face, that choice's probability within 0.05, and every box edge within 3% of the frame:
| GPU fp16 | Neural Engine fp16 | Neural Engine, 4 pieces | |
|---|---|---|---|
| frames passing | 9 / 9 | 8 / 9 | 8 / 9 |
The Neural Engine's one miss is a synthetic uniform mid-gray frame: same answers, a probability about 0.05 off. Every real frame passes.
Speed (per decision, preprocessing + model + readout): GPU about 50 ms, Neural Engine about 180 ms (whole or in pieces). Measured on the first, baked-prompt export (the prompt-input one runs at the same speed), with power for the whole Mac (macmon, 20 s of back-to-back decisions):
| MLX 8-bit | Core AI GPU | Core AI Neural Engine | |
|---|---|---|---|
| per decision | 59 ms | 55 ms | 182 ms |
| first load (specialization, once) | 12 s | 56 s | |
| cached load | 0.6 s | 0.3 s | |
| Mac power above idle | 131 W | 98 W | 17 W |
| energy per decision | 8.4 J | 6.0 J | 3.2 J |
The Neural Engine build runs entirely on the Neural Engine: its compiled plan is marked mps.fullyPlacedOnANE and
mps.noGPUActivity. It is about 3Γ slower than the GPU for this 2B model and uses about half the energy per decision.
Changes from the original
Modified by Ivan Fioravanti (notice under section 4(b) of the LFM Open License). The .aimodel files and
embed_tokens_fp16.npy are derived from the LiquidAI/d1-2B weights:
- re-implemented d1's decision pass as one fixed-shape PyTorch function (the prompt as inputs) and exported it with Apple's
Core AI tools (
coreai-torch0.4.2,coreai-modelsexport path); weights cast from bf16 to fp16; - the Neural Engine variant is re-authored for the Neural Engine (1Γ1 convolutions, per-head attention, fp16 constants)
and, to keep fp16 precision there, carries exact rescalings (the vision and text residual streams, the patch embedding and
the branch outputs before scale-invariant norms are carried at larger scale), computes
sigmoidthroughtanh, and computes each MLP in 8 chunks of its hidden channels (1.4Γ faster there). The function is mathematically the same; numbers differ only by fp16 rounding; the split variant is the same function cut between decoder layers; - the token embedding table is shipped separately (
embed_tokens_fp16.npy) to build the prompt inputs; d1's LM head is tied to it, so the readout rows come from the same table.
No retraining or fine-tuning. Int8 weights were tried and are not shipped: no GPU speedup; on the Neural Engine, per-channel int8 brought no speedup and per-block int8 does not compile for it (Core AI then silently runs it on the GPU).
License
Same license as the original weights: LFM Open License v1.0 (LICENSE, also at
LiquidAI/d1-2B). As there, commercial use is licensed only for
entities below US$10 million in annual revenue (section 5). "d1", "LFM" and "Liquid AI" are Liquid AI's names, used here
only to describe where this comes from.
Citation
Cite the original model:
@article{liquidAI2026opend1,
author = {Liquid AI},
title = {Open d1: Edge decision models for text, vision, and audio},
journal = {Liquid AI Blog},
year = {2026},
note = {https://www.liquid.ai/blog/open-d1},
}
@article{liquidAI2026LFM3,
author = {Liquid AI},
title = {Introducing LFM3: Every Modality for Specialized Intelligence},
journal = {Liquid AI Blog},
year = {2026},
note = {https://www.liquid.ai/blog/lfm3},
}