ANEForge

Run computation on the Apple Neural Engine (ANE) directly — without CoreML.

ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it through the same daemon and kernel-driver stack Apple's own frameworks use. In production the ANE is reachable only through CoreML, which can silently fall back to CPU/GPU; ANEForge targets the engine directly.

GitHub PyPI Docs Paper

What runs on the engine

• LLM decode & prefill — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding.
• Training on the ANE — the forward pass, backward pass, and Adam update all compile to ANE programs.
• ONNX frontend — import ONNX graphs and run them on the engine.
• Vision — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE.
• Scientific computing — FFT, linear algebra (solve / LU / SVD / expm), DSP.
• Native fused attention, and int8 / int4-LUT / sparse weight streaming (~4x smaller for int4, accuracy-gated).

Performance

A small fused program completes a call in ~90 us, near the engine's ~70 us per-program dispatch floor; a pretrained ResNet-18 forward runs end to end in ~0.33 ms. Apple Silicon, macOS 14+.

On the Hub

• sentence-embeddings — run any sentence-transformers model's encoder on the ANE.
• llm-text-generation — run a Llama/Qwen-family causal LM's decode on the ANE.
• reranker — run a BERT-family cross-encoder reranker on the ANE (~0.8 ms/pair on M5 Pro).
• vit-image-classification — run a HF ViT image classifier on the ANE.
• ane-leaderboard — how fast is the Neural Engine on your Mac? Peak perf and correctness cliffs across Apple Silicon.
• ane-rooflines — cross-Apple-Silicon roofline & fp16-correctness data.
• demos — a gallery of fluid, reaction-diffusion, and on-engine training runs.

Install: pip install aneforge. The ANE only exists on Apple Silicon, so ANEForge runs on your own Mac.

Cite: Bryngelson, S. H. ANEForge: Python for direct computation on the Apple Neural Engine. arXiv:2606.17090 (2026).