File size: 2,375 Bytes
4f9b650 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
library_name: aneforge
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- apple-neural-engine
- ane
- coreml-free
- on-device
- apple-silicon
- embeddings
license: mit
---
# Sentence embeddings on the Apple Neural Engine (via ANEForge)
[ANEForge](https://github.com/sbryngelson/ANEForge) runs computation on the Apple
Neural Engine (ANE) directly, without CoreML. Its `SentenceTransformer` drop-in loads
**any** sentence-transformers model from the Hub by repo id and runs the encoder on the
engine, matching the `sentence_transformers` API.
This is a usage card, not a re-hosted model: it points at the upstream weights and shows
how to run them on the ANE.
## Install
```sh
pip install aneforge # numpy-only core; the dispatch shim builds on first use
```
Requires Apple Silicon under macOS 14+. `import aneforge` works anywhere; compiling and
dispatching to the ANE needs the hardware.
## Use (drop-in for `sentence_transformers.SentenceTransformer`)
```python
from aneforge.sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") # any Hub repo id
emb = model.encode(["a sentence on the Neural Engine", "another one"],
normalize_embeddings=True)
print(emb.shape) # (2, 384)
```
`int8=True` streams int8 weights (cosine ~0.9999 vs fp16). A model whose `modules.json`
declares a Normalize module is always L2-normalised, as with upstream sentence-transformers.
## Why the ANE
The ANE is the fixed-function accelerator on every recent Apple device. In production it
is reachable only through CoreML, which treats it as a schedulable option that can
silently fall back to CPU/GPU. ANEForge compiles a lazy tensor graph into a single ANE
program and dispatches it through the same daemon and kernel-driver stack Apple's own
frameworks use, so the encoder runs on the engine deterministically and off the CPU/GPU.
## Links
- Code: https://github.com/sbryngelson/ANEForge
- Package: https://pypi.org/project/aneforge/
- Paper: https://arxiv.org/abs/2606.17090
## Citation
```bibtex
@article{bryngelson2026aneforge,
title = {ANEForge: Python for direct computation on the Apple Neural Engine},
author = {Bryngelson, Spencer H.},
journal = {arXiv preprint arXiv:2606.17090},
year = {2026},
doi = {10.48550/arXiv.2606.17090}
}
```
|