title: ANEForge
sdk: static
pinned: false
ANEForge
Run computation on the Apple Neural Engine (ANE) directly, without CoreML.
ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it through the same daemon and kernel-driver stack Apple's own frameworks use. In production the ANE is reachable only through CoreML, which treats it as a schedulable option that can silently fall back to CPU/GPU; ANEForge targets the engine directly and deterministically.
- Code: https://github.com/sbryngelson/ANEForge
- Install:
pip install aneforge· PyPI - Paper: https://arxiv.org/abs/2606.17090
- Docs: https://aneforge.readthedocs.io
What runs on the engine
- LLM decode & prefill — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding.
- Training on the ANE — the forward pass, backward pass, and Adam update all compile to ANE programs.
- ONNX frontend — import ONNX graphs and run them on the engine.
- Vision — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE.
- Scientific computing — FFT, linear algebra (solve / LU / SVD / expm), DSP.
- Native fused attention, and int8 / int4-LUT / sparse weight streaming from the engine's dequant path (~4x smaller for int4, accuracy-gated).
Performance
A small fused program completes a call in ~90 us, near the engine's ~70 us per-program dispatch floor; a pretrained ResNet-18 forward runs end-to-end in ~0.33 ms. Apple Silicon, macOS 14 and later; each release is verified against a recorded macOS and ANE-compiler version.
On the Hub
- aneforge/sentence-embeddings — run any sentence-transformers model's encoder on the ANE (drop-in for
sentence_transformers).
More cards for the LLM, vision, and ONNX paths are on the way.
Cite
Bryngelson, S. H. ANEForge: Python for direct computation on the Apple Neural Engine. arXiv:2606.17090 (2026).