File size: 5,138 Bytes
f211032 c89575e 078e7d6 c89575e 078e7d6 89d9a9f 078e7d6 c89575e f211032 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | <!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ANEForge</title>
<style>
:root { color-scheme: light dark; }
* { box-sizing: border-box; }
body {
margin: 0; padding: 40px 20px 64px;
font: 16px/1.65 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
background: #0b0d12; color: #e7e9ee;
}
@media (prefers-color-scheme: light) { body { background: #fbfbfd; color: #1c1e26; } }
.wrap { max-width: 860px; margin: 0 auto; }
h1 { font-size: 2rem; margin: 0 0 6px; letter-spacing: -0.02em; }
.lede { font-size: 1.1rem; color: #9aa2b2; margin: 0 0 22px; }
@media (prefers-color-scheme: light) { .lede { color: #5a6072; } }
p { margin: 0 0 16px; }
h2 { font-size: 1.15rem; margin: 28px 0 10px; }
ul { margin: 0 0 8px; padding-left: 20px; }
li { margin: 4px 0; }
a { color: #7f9cff; text-decoration: none; }
a:hover { text-decoration: underline; }
.pills a {
display: inline-block; border: 1px solid #2a3040; border-radius: 999px;
padding: 6px 14px; margin: 4px 6px 4px 0; font-size: 0.9rem; color: #c9d2ff;
}
@media (prefers-color-scheme: light) { .pills a { border-color: #d0d4e0; color: #3a52cc; } }
.pills a:hover { border-color: #5566aa; text-decoration: none; }
code { background: #1a1f2b; padding: 2px 6px; border-radius: 6px; font-size: 0.85em; }
@media (prefers-color-scheme: light) { code { background: #eceef3; } }
.muted { color: #7f8797; font-size: 0.9rem; }
</style>
</head>
<body>
<div class="wrap">
<h1>ANEForge</h1>
<p class="lede">Run computation on the Apple Neural Engine (ANE) directly — without CoreML.</p>
<p>ANEForge compiles a lazy tensor graph into a single fused ANE program and dispatches it through the
same daemon and kernel-driver stack Apple's own frameworks use. In production the ANE is reachable
only through CoreML, which can silently fall back to CPU/GPU; ANEForge targets the engine directly.</p>
<p class="pills">
<a target="_blank" rel="noopener" href="https://github.com/sbryngelson/ANEForge">GitHub</a>
<a target="_blank" rel="noopener" href="https://pypi.org/project/aneforge/">PyPI</a>
<a target="_blank" rel="noopener" href="https://aneforge.readthedocs.io">Docs</a>
<a target="_blank" rel="noopener" href="https://arxiv.org/abs/2606.17090">Paper</a>
</p>
<h2>What runs on the engine</h2>
<ul>
<li><b>LLM decode & prefill</b> — Llama / Qwen / MoE blocks, KV cache resident across steps, speculative decoding.</li>
<li><b>Training on the ANE</b> — the forward pass, backward pass, and Adam update all compile to ANE programs.</li>
<li><b>ONNX frontend</b> — import ONNX graphs and run them on the engine.</li>
<li><b>Vision</b> — ResNet, Vision Transformer, Stable Diffusion U-Net / VAE.</li>
<li><b>Scientific computing</b> — FFT, linear algebra (solve / LU / SVD / expm), DSP.</li>
<li><b>Native fused attention</b>, and <b>int8 / int4-LUT / sparse</b> weight streaming (~4x smaller for int4, accuracy-gated).</li>
</ul>
<h2>Performance</h2>
<p>A small fused program completes a call in ~90 us, near the engine's ~70 us per-program
dispatch floor; a pretrained ResNet-18 forward runs end to end in ~0.33 ms. Apple Silicon, macOS 14+.</p>
<h2>On the Hub</h2>
<ul>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/sentence-embeddings">sentence-embeddings</a> — run any sentence-transformers model's encoder on the ANE.</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/llm-text-generation">llm-text-generation</a> — run a Llama/Qwen-family causal LM's decode on the ANE.</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/reranker">reranker</a> — run a BERT-family cross-encoder reranker on the ANE (~0.8 ms/pair on M5 Pro).</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/aneforge/vit-image-classification">vit-image-classification</a> — run a HF ViT image classifier on the ANE.</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/spaces/aneforge/ane-leaderboard">ane-leaderboard</a> — how fast is the Neural Engine on your Mac? Peak perf and correctness cliffs across Apple Silicon.</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/datasets/aneforge/ane-rooflines">ane-rooflines</a> — cross-Apple-Silicon roofline & fp16-correctness data.</li>
<li><a target="_blank" rel="noopener" href="https://huggingface.co/spaces/aneforge/demos">demos</a> — a gallery of fluid, reaction-diffusion, and on-engine training runs.</li>
</ul>
<p class="muted">Install: <code>pip install aneforge</code>. The ANE only exists on Apple Silicon, so
ANEForge runs on your own Mac.</p>
<p class="muted">Cite: Bryngelson, S. H. <i>ANEForge: Python for direct computation on the Apple Neural Engine.</i> arXiv:2606.17090 (2026).</p>
</div>
</body>
</html>
|