README.md

---
library_name: aneforge
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- apple-neural-engine
- ane
- coreml-free
- on-device
- apple-silicon
- embeddings
license: mit
---

# Sentence embeddings on the Apple Neural Engine (via ANEForge)

[ANEForge](https://github.com/sbryngelson/ANEForge) runs computation on the Apple
Neural Engine (ANE) directly, without CoreML. Its `SentenceTransformer` drop-in loads
**any** sentence-transformers model from the Hub by repo id and runs the encoder on the
engine, matching the `sentence_transformers` API.

This is a usage card, not a re-hosted model: it points at the upstream weights and shows
how to run them on the ANE.

## Install

```sh
pip install aneforge          # numpy-only core; the dispatch shim builds on first use
```

Requires Apple Silicon under macOS 14+. `import aneforge` works anywhere; compiling and
dispatching to the ANE needs the hardware.

## Use (drop-in for `sentence_transformers.SentenceTransformer`)

```python
from aneforge.sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")   # any Hub repo id
emb = model.encode(["a sentence on the Neural Engine", "another one"],
                   normalize_embeddings=True)
print(emb.shape)   # (2, 384)
```

`int8=True` streams int8 weights (cosine ~0.9999 vs fp16). A model whose `modules.json`
declares a Normalize module is always L2-normalised, as with upstream sentence-transformers.

## Why the ANE

The ANE is the fixed-function accelerator on every recent Apple device. In production it
is reachable only through CoreML, which treats it as a schedulable option that can
silently fall back to CPU/GPU. ANEForge compiles a lazy tensor graph into a single ANE
program and dispatches it through the same daemon and kernel-driver stack Apple's own
frameworks use, so the encoder runs on the engine deterministically and off the CPU/GPU.

## Links

- Code: https://github.com/sbryngelson/ANEForge
- Package: https://pypi.org/project/aneforge/
- Paper: https://arxiv.org/abs/2606.17090

## Citation

```bibtex
@article{bryngelson2026aneforge,
  title   = {ANEForge: Python for direct computation on the Apple Neural Engine},
  author  = {Bryngelson, Spencer H.},
  journal = {arXiv preprint arXiv:2606.17090},
  year    = {2026},
  doi     = {10.48550/arXiv.2606.17090}
}
```