| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1" /> |
| <title>ANEForge</title> |
| <style> |
| :root { color-scheme: light dark; } |
| * { box-sizing: border-box; } |
| body { |
| margin: 0; padding: 40px 20px 64px; |
| font: 16px/1.65 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; |
| background: #0b0d12; color: #e7e9ee; |
| } |
| @media (prefers-color-scheme: light) { body { background: #fbfbfd; color: #1c1e26; } } |
| .wrap { max-width: 860px; margin: 0 auto; } |
| h1 { font-size: 2rem; margin: 0 0 6px; letter-spacing: -0.02em; } |
| .lede { font-size: 1.1rem; color: #9aa2b2; margin: 0 0 22px; } |
| @media (prefers-color-scheme: light) { .lede { color: #5a6072; } } |
| p { margin: 0 0 16px; } |
| h2 { font-size: 1.15rem; margin: 28px 0 10px; } |
| ul { margin: 0 0 8px; padding-left: 20px; } |
| li { margin: 4px 0; } |
| a { color: #7f9cff; text-decoration: none; } |
| a:hover { text-decoration: underline; } |
| .pills a { |
| display: inline-block; border: 1px solid #2a3040; border-radius: 999px; |
| padding: 6px 14px; margin: 4px 6px 4px 0; font-size: 0.9rem; color: #c9d2ff; |
| } |
| @media (prefers-color-scheme: light) { .pills a { border-color: #d0d4e0; color: #3a52cc; } } |
| .pills a:hover { border-color: #5566aa; text-decoration: none; } |
| code { background: #1a1f2b; padding: 2px 6px; border-radius: 6px; font-size: 0.85em; } |
| @media (prefers-color-scheme: light) { code { background: #eceef3; } } |
| .muted { color: #7f8797; font-size: 0.9rem; } |
| </style> |
| </head> |
| <body> |
| <div class="wrap"> |
| <h1>ANEForge</h1> |
| <p class="lede">Run computation on the Apple Neural Engine (ANE) directly — without CoreML.</p> |
|
|
| <p>ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it through the |
| same daemon and kernel-driver stack Apple's own frameworks use. In production the ANE is reachable |
| only through CoreML, which can silently fall back to CPU/GPU; ANEForge targets the engine directly.</p> |
|
|
| <p class="pills"> |
| <a target="_blank" rel="noopener" href="https://github.com/sbryngelson/ANEForge">GitHub</a> |
| <a target="_blank" rel="noopener" href="https://pypi.org/project/aneforge/">PyPI</a> |
| <a target="_blank" rel="noopener" href="https://aneforge.readthedocs.io">Docs</a> |
| <a target="_blank" rel="noopener" href="https://arxiv.org/abs/2606.17090">Paper</a> |
| </p> |
|
|
| <h2>What runs on the engine</h2> |
| <ul> |
| <li><b>LLM decode & prefill</b> — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding.</li> |
| <li><b>Training on the ANE</b> — the forward pass, backward pass, and Adam update all compile to ANE programs.</li> |
| <li><b>ONNX frontend</b> — import ONNX graphs and run them on the engine.</li> |
| <li><b>Vision</b> — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE.</li> |
| <li><b>Scientific computing</b> — FFT, linear algebra (solve / LU / SVD / expm), DSP.</li> |
| <li><b>Native fused attention</b>, and <b>int8 / int4-LUT / sparse</b> weight streaming (~4x smaller for int4, accuracy-gated).</li> |
| </ul> |
|
|
| <h2>Performance</h2> |
| <p>A small fused program completes a call in ~90 us, near the engine's ~70 us per-program |
| dispatch floor; a pretrained ResNet-18 forward runs end to end in ~0.33 ms. Apple Silicon, macOS 14+.</p> |
|
|
| <h2>On the Hub</h2> |
| <ul> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/sentence-embeddings">sentence-embeddings</a> — run any sentence-transformers model's encoder on the ANE.</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/llm-text-generation">llm-text-generation</a> — run a Llama/Qwen-family causal LM's decode on the ANE.</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/reranker">reranker</a> — run a BERT-family cross-encoder reranker on the ANE (~0.8 ms/pair on M5 Pro).</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/vit-image-classification">vit-image-classification</a> — run a HF ViT image classifier on the ANE.</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/spaces/aneforge/ane-leaderboard">ane-leaderboard</a> — how fast is the Neural Engine on your Mac? Peak perf and correctness cliffs across Apple Silicon.</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/datasets/aneforge/ane-rooflines">ane-rooflines</a> — cross-Apple-Silicon roofline & fp16-correctness data.</li> |
| <li><a target="_blank" rel="noopener" href="https://huggingface.co/spaces/aneforge/demos">demos</a> — a gallery of fluid, reaction-diffusion, and on-engine training runs.</li> |
| </ul> |
|
|
| <p class="muted">Install: <code>pip install aneforge</code>. The ANE only exists on Apple Silicon, so |
| ANEForge runs on your own Mac.</p> |
|
|
| <p class="muted">Cite: Bryngelson, S. H. <i>ANEForge: Python for direct computation on the Apple Neural Engine.</i> arXiv:2606.17090 (2026).</p> |
| </div> |
| </body> |
| </html> |
|
|
