README.md

---
title: ANEForge
sdk: static
pinned: false
---

# ANEForge

**Run computation on the Apple Neural Engine (ANE) directly, without CoreML.**

ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it
through the same daemon and kernel-driver stack Apple's own frameworks use. In production
the ANE is reachable only through CoreML, which treats it as a schedulable option that can
silently fall back to CPU/GPU; ANEForge targets the engine directly and deterministically.

- **Code:** https://github.com/sbryngelson/ANEForge
- **Install:** `pip install aneforge` · [PyPI](https://pypi.org/project/aneforge/)
- **Paper:** https://arxiv.org/abs/2606.17090
- **Docs:** https://aneforge.readthedocs.io

## What runs on the engine

- **LLM decode & prefill** — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding.
- **Training on the ANE** — the forward pass, backward pass, and Adam update all compile to ANE programs.
- **ONNX frontend** — import ONNX graphs and run them on the engine.
- **Vision** — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE.
- **Scientific computing** — FFT, linear algebra (solve / LU / SVD / expm), DSP.
- **Native fused attention**, and **int8 / int4-LUT / sparse** weight streaming from the engine's dequant path (~4x smaller for int4, accuracy-gated).

## Performance

A small fused program completes a call in ~90 us, near the engine's ~70 us per-program
dispatch floor; a pretrained ResNet-18 forward runs end-to-end in ~0.33 ms. Apple Silicon,
macOS 14 and later; each release is verified against a recorded macOS and ANE-compiler version.

## On the Hub

- [aneforge/sentence-embeddings](https://huggingface.co/aneforge/sentence-embeddings) — run any sentence-transformers model's encoder on the ANE (drop-in for `sentence_transformers`).

More cards for the LLM, vision, and ONNX paths are on the way.

## Cite

> Bryngelson, S. H. *ANEForge: Python for direct computation on the Apple Neural Engine.* arXiv:2606.17090 (2026).