File size: 2,201 Bytes
ddf5b84 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
library_name: aneforge
pipeline_tag: text-ranking
tags:
- apple-neural-engine
- ane
- coreml-free
- on-device
- apple-silicon
- reranker
- cross-encoder
- rag
license: mit
---
# Reranking on the Apple Neural Engine (via ANEForge)
[ANEForge](https://github.com/sbryngelson/ANEForge) runs computation on the Apple Neural
Engine (ANE) directly, without CoreML. Its `CrossEncoder` loads a BERT-family cross-encoder /
reranker from the Hub by repo id and runs the transformer on the engine, matching the
`sentence_transformers.CrossEncoder` API.
This is a usage card, not a re-hosted model: it points at the upstream weights and shows how
to run them on the ANE.
## Install
```sh
pip install aneforge
```
Apple Silicon, macOS 14+.
## Use
```python
from aneforge.sentence_transformers import CrossEncoder
ce = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") # any BERT-family cross-encoder
query = "How many people live in Berlin?"
passages = [
"Berlin has about 3.85 million residents.",
"Paris is the capital of France.",
]
scores = ce.predict([(query, p) for p in passages]) # higher = more relevant; transformer on the ANE
ranked = sorted(zip(scores, passages), reverse=True)
```
## Measured
On an M5 Pro, `cross-encoder/ms-marco-MiniLM-L-6-v2` scores a (query, passage) pair in
**~0.8 ms**, matching the Hugging Face reference ranking (relerr 5e-4).
## Scope
BERT-family cross-encoders with a pooler + classifier head (e.g. `cross-encoder/ms-marco-MiniLM-L-6-v2`,
`-L-12-v2`). RoBERTa/XLM-R rerankers (bge-reranker) are a work in progress -- see the repo issues.
## Why the ANE
The ANE is the fixed-function accelerator on every recent Apple device. In production it is
reachable only through CoreML, which can silently fall back to CPU/GPU; ANEForge compiles the
transformer to a single ANE program and dispatches it through the same daemon and kernel-driver
stack Apple's own frameworks use.
## Links
- Code: https://github.com/sbryngelson/ANEForge
- Package: https://pypi.org/project/aneforge/
- Paper: https://arxiv.org/abs/2606.17090
## Cite
> Bryngelson, S. H. *ANEForge: Python for direct computation on the Apple Neural Engine.* arXiv:2606.17090 (2026).
|