---
license: apache-2.0
base_model: openai/privacy-filter
pipeline_tag: token-classification
library_name: mlx
language:
  - en
tags:
  - mlx
  - apple-silicon
  - metal
  - token-classification
  - ner
  - pii
  - privacy
  - redaction
---

# privacy-filter-mlx

On-device **PII / secret detection** for Apple Silicon (MLX / Metal). A repackaging of
[`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) (a gpt-oss-style
128-expert MoE token classifier) for the [MLX](https://github.com/ml-explore/mlx) runtime,
used by the native Swift redactor `pf`.

It tags every token with one of **33 BIOES labels** across 8 categories
(`account_number`, `private_address`, `private_date`, `private_email`, `private_person`,
`private_phone`, `private_url`, `secret`) so a downstream redactor can mask spans it has
never seen before — names, emails, phone numbers, API keys, URLs — that exact-match
maskers miss.

## Variants

| Path        | Format                         | Size     | Labels intact¹ | Use                                              |
|-------------|--------------------------------|----------|----------------|--------------------------------------------------|
| `/` (root)  | bf16, upstream tensor names    | ~2.6 GB  | 100% (ceiling) | Full fidelity; quantize at runtime to any config |
| `q4-8emb/`  | 4-bit MoE + 8-bit embeddings   | ~0.87 GB | 99.4%          | Pre-quantized — smaller download, no runtime quant |

¹ Argmax labels matching the fp32 reference, measured on a 200-text / 40k-token synthetic
PII eval set.

### Quantization frontier (measured)

The MoE experts are ~90% of the weights, so they dominate the size/quality trade-off:

| Config                          | Size    | Labels | Cosine  |
|---------------------------------|---------|--------|---------|
| fp16                            | 2799 MB | 99.9%  | 0.99982 |
| 8-bit MoE                       | 1619 MB | 99.8%  | —       |
| **4-bit MoE + 8-bit embed** (`q4-8emb/`) | **870 MB** | **99.4%** | **0.998** |
| 3-bit/128 + 8-bit embed         | 670 MB  | 98.9%  | —       |
| 2-bit                           | 642 MB  | 97.9%  | —       |

`q4-8emb` is the recommended default: a 69% size cut for ~0.5% label drift. 3-/2-bit are
certified but risky for a redactor (label drift on a fail-closed task).

## Usage

### Download

```bash
# bf16 (root) — full fidelity, runtime-quantizable
hf download beshkenadze/privacy-filter-mlx --local-dir ./privacy-filter

# pre-quantized 870 MB only
hf download beshkenadze/privacy-filter-mlx --include "q4-8emb/*" --local-dir ./privacy-filter
```

### Python (MLX)

The reference forward (`pf_mlx.py`) loads the bf16 weights and quantizes in-memory via a
runtime knob:

```bash
PF_QBITS=4 PF_QEMBED=8 python pf_mlx.py        # 870 MB path
PF_QBITS=0 python pf_mlx.py                     # fp16, full size
```

### Swift CLI (`pf`)

`pf` streams `stdin → stdout` and replaces detected spans with stable `<CATEGORY_n>`
tokens, **fail-closed** (no raw value ever reaches stdout unless `--fail-open`):

```bash
cat app.log | pf --model ./privacy-filter
# Contact <PRIVATE_PERSON_0> at <PRIVATE_EMAIL_0>, key <SECRET_0>
```

## Architecture

gpt-oss-style MoE token classifier: 8 layers, `hidden=640`, 14/2 attention heads × 64,
`intermediate=640`, **128 experts top-4**, attention sinks, bidirectional sliding-window
attention (radius 128), interleaved YaRN RoPE (θ=150000, factor=32, truncate=false),
o200k tokenizer, 33-label BIOES head. See `config.json`.

## Why MLX, not Core ML / ANE

A 128-expert top-4 MoE needs a **sparse gather**, which the Apple Neural Engine cannot do
— Core ML evicts the MoE to the GPU and runs it dense (32× redundant compute, ~336 ms).
MLX's sparse `gather_mm` path on Metal is both exact and ~44× faster (~7.6 ms). For MoE on
Apple Silicon, use MLX.

## License & attribution

Apache-2.0, inherited from the upstream model. This repo is a format repackaging of
[`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) — all model credit
to OpenAI. The bf16 weights at the root are byte-equivalent to the upstream `safetensors`;
`q4-8emb/` is a quantization of those weights.