---
language: multilingual
license: apache-2.0
base_model: perplexity-ai/pplx-embed-v1-0.6b
tags:
  - coreml
  - apple-neural-engine
  - qwen3
  - sentence-embedding
  - on-device
library_name: coreml
---

# pplx-embed for Apple CoreML (ANE-optimized)

CoreML conversion of Perplexity's
[`pplx-embed-v1-0.6b`](https://huggingface.co/perplexity-ai/pplx-embed-v1-0.6b)
(a bidirectional Qwen3-0.6B encoder → masked-mean pool → tanh-int8 head) produced with
the [CoreML-LLM](https://github.com/john-rocky/CoreML-LLM) pipeline. Targets macOS 26.

Each subfolder is a **fixed-shape sequence-length bucket** that stays resident on the
Apple Neural Engine (flexible shapes force CPU fallback). At runtime the Swift package
pads each input to the smallest bucket that fits; inputs longer than the largest fixed
bucket fall through to the `dyn*-int8/` flexible GPU catch-all. The encoder uses native
RMSNorm and a single fixed RoPE table — the ANE-fastest path on M4 Max / macOS 26.

## Buckets in this repo

| Subfolder | Variant | Bucket (L) | Kind | Size |
|---|---|---|---|---|
| `L1024-int8/` | plain | 1024 | fixed ANE bucket | 2.44 GB |
| `L2048-int8/` | plain | 2048 | fixed ANE bucket | 2.44 GB |
| `L4096-int8/` | plain | 4096 | fixed ANE bucket | 2.44 GB |
| `L512-int8/` | plain | 512 | fixed ANE bucket | 2.44 GB |
| `dyn8192-int8/` | plain | 1..8192 | dynamic GPU catch-all | 2.44 GB |
| `context/L512-int8/` | context | 512 | fixed ANE bucket | 2.44 GB |

The encoder `weight.bin` is **byte-identical across every bucket** (a single fixed-size
RoPE table makes the weights independent of bucket length). So HF stores the weight blob
**once**, and the HF content-addressed cache fetches it **once by etag** on download —
pulling several buckets costs ~1.15 GB total, not ~1.15 GB × N.

## Use it

Via the [CoreML-LLM Swift package](https://github.com/john-rocky/CoreML-LLM). It uses the
HF Swift Hub client, so only the buckets you request are downloaded and the shared weight
is fetched once into the content-addressed cache:

```swift
import CoreMLLLM
let embedder = try await PplxEmbed.load(
    repo: "dokterbob/pplx-embed-coreml",
    buckets: [512, 1024, 2048])       // shared HF cache; weight fetched once by etag
let vecs = try embedder.embed(["On-device embeddings", "Bonjour le monde"])  // [[Int8]]
```

Each bucket is published in both `.mlpackage` and precompiled `.mlmodelc`; pass
`preferCompiled: false` for the portable package. Or download the bundle directory
yourself and load it with `load(bundleDir:)`.

## I/O contract (per bucket `model_config.json`)

- `input_ids (1, L) int32`, `attention_mask (1, L) fp16` (1.0 valid, 0.0 pad)
- `embedding (1, 1024) int8` — `clamp(round(tanh(x)*127), -128, 127)`; derive
  `binary`/`ubinary` from the int8 sign (see `PplxEmbed`).

## License

Inherits the base model's [license](https://huggingface.co/perplexity-ai/pplx-embed-v1-0.6b).