| --- |
| title: ANEForge |
| sdk: static |
| pinned: false |
| --- |
| |
| # ANEForge |
|
|
| **Run computation on the Apple Neural Engine (ANE) directly, without CoreML.** |
|
|
| ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it |
| through the same daemon and kernel-driver stack Apple's own frameworks use. In production |
| the ANE is reachable only through CoreML, which treats it as a schedulable option that can |
| silently fall back to CPU/GPU; ANEForge targets the engine directly and deterministically. |
|
|
| - **Code:** https://github.com/sbryngelson/ANEForge |
| - **Install:** `pip install aneforge` · [PyPI](https://pypi.org/project/aneforge/) |
| - **Paper:** https://arxiv.org/abs/2606.17090 |
| - **Docs:** https://aneforge.readthedocs.io |
|
|
| ## What runs on the engine |
|
|
| - **LLM decode & prefill** — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding. |
| - **Training on the ANE** — the forward pass, backward pass, and Adam update all compile to ANE programs. |
| - **ONNX frontend** — import ONNX graphs and run them on the engine. |
| - **Vision** — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE. |
| - **Scientific computing** — FFT, linear algebra (solve / LU / SVD / expm), DSP. |
| - **Native fused attention**, and **int8 / int4-LUT / sparse** weight streaming from the engine's dequant path (~4x smaller for int4, accuracy-gated). |
|
|
| ## Performance |
|
|
| A small fused program completes a call in ~90 us, near the engine's ~70 us per-program |
| dispatch floor; a pretrained ResNet-18 forward runs end-to-end in ~0.33 ms. Apple Silicon, |
| macOS 14 and later; each release is verified against a recorded macOS and ANE-compiler version. |
|
|
| ## On the Hub |
|
|
| - [aneforge/sentence-embeddings](https://huggingface.co/aneforge/sentence-embeddings) — run any sentence-transformers model's encoder on the ANE (drop-in for `sentence_transformers`). |
|
|
| More cards for the LLM, vision, and ONNX paths are on the way. |
|
|
| ## Cite |
|
|
| > Bryngelson, S. H. *ANEForge: Python for direct computation on the Apple Neural Engine.* arXiv:2606.17090 (2026). |
|
|
