| --- |
| license: apache-2.0 |
| base_model: openai/privacy-filter |
| pipeline_tag: token-classification |
| library_name: mlx |
| language: |
| - en |
| tags: |
| - mlx |
| - apple-silicon |
| - metal |
| - token-classification |
| - ner |
| - pii |
| - privacy |
| - redaction |
| --- |
| |
| # privacy-filter-mlx |
|
|
| On-device **PII / secret detection** for Apple Silicon (MLX / Metal). A repackaging of |
| [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) (a gpt-oss-style |
| 128-expert MoE token classifier) for the [MLX](https://github.com/ml-explore/mlx) runtime, |
| used by the native Swift redactor `pf`. |
|
|
| It tags every token with one of **33 BIOES labels** across 8 categories |
| (`account_number`, `private_address`, `private_date`, `private_email`, `private_person`, |
| `private_phone`, `private_url`, `secret`) so a downstream redactor can mask spans it has |
| never seen before — names, emails, phone numbers, API keys, URLs — that exact-match |
| maskers miss. |
|
|
| ## Variants |
|
|
| | Path | Format | Size | Labels intact¹ | Use | |
| |-------------|--------------------------------|----------|----------------|--------------------------------------------------| |
| | `/` (root) | bf16, upstream tensor names | ~2.6 GB | 100% (ceiling) | Full fidelity; quantize at runtime to any config | |
| | `q4-8emb/` | 4-bit MoE + 8-bit embeddings | ~0.87 GB | 99.4% | Pre-quantized — smaller download, no runtime quant | |
|
|
| ¹ Argmax labels matching the fp32 reference, measured on a 200-text / 40k-token synthetic |
| PII eval set. |
|
|
| ### Quantization frontier (measured) |
|
|
| The MoE experts are ~90% of the weights, so they dominate the size/quality trade-off: |
|
|
| | Config | Size | Labels | Cosine | |
| |---------------------------------|---------|--------|---------| |
| | fp16 | 2799 MB | 99.9% | 0.99982 | |
| | 8-bit MoE | 1619 MB | 99.8% | — | |
| | **4-bit MoE + 8-bit embed** (`q4-8emb/`) | **870 MB** | **99.4%** | **0.998** | |
| | 3-bit/128 + 8-bit embed | 670 MB | 98.9% | — | |
| | 2-bit | 642 MB | 97.9% | — | |
|
|
| `q4-8emb` is the recommended default: a 69% size cut for ~0.5% label drift. 3-/2-bit are |
| certified but risky for a redactor (label drift on a fail-closed task). |
|
|
| ## Usage |
|
|
| ### Download |
|
|
| ```bash |
| # bf16 (root) — full fidelity, runtime-quantizable |
| hf download beshkenadze/privacy-filter-mlx --local-dir ./privacy-filter |
| |
| # pre-quantized 870 MB only |
| hf download beshkenadze/privacy-filter-mlx --include "q4-8emb/*" --local-dir ./privacy-filter |
| ``` |
|
|
| ### Python (MLX) |
|
|
| The reference forward (`pf_mlx.py`) loads the bf16 weights and quantizes in-memory via a |
| runtime knob: |
|
|
| ```bash |
| PF_QBITS=4 PF_QEMBED=8 python pf_mlx.py # 870 MB path |
| PF_QBITS=0 python pf_mlx.py # fp16, full size |
| ``` |
|
|
| ### Swift CLI (`pf`) |
|
|
| `pf` streams `stdin → stdout` and replaces detected spans with stable `<CATEGORY_n>` |
| tokens, **fail-closed** (no raw value ever reaches stdout unless `--fail-open`): |
|
|
| ```bash |
| cat app.log | pf --model ./privacy-filter |
| # Contact <PRIVATE_PERSON_0> at <PRIVATE_EMAIL_0>, key <SECRET_0> |
| ``` |
|
|
| ## Architecture |
|
|
| gpt-oss-style MoE token classifier: 8 layers, `hidden=640`, 14/2 attention heads × 64, |
| `intermediate=640`, **128 experts top-4**, attention sinks, bidirectional sliding-window |
| attention (radius 128), interleaved YaRN RoPE (θ=150000, factor=32, truncate=false), |
| o200k tokenizer, 33-label BIOES head. See `config.json`. |
|
|
| ## Why MLX, not Core ML / ANE |
|
|
| A 128-expert top-4 MoE needs a **sparse gather**, which the Apple Neural Engine cannot do |
| — Core ML evicts the MoE to the GPU and runs it dense (32× redundant compute, ~336 ms). |
| MLX's sparse `gather_mm` path on Metal is both exact and ~44× faster (~7.6 ms). For MoE on |
| Apple Silicon, use MLX. |
|
|
| ## License & attribution |
|
|
| Apache-2.0, inherited from the upstream model. This repo is a format repackaging of |
| [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) — all model credit |
| to OpenAI. The bf16 weights at the root are byte-equivalent to the upstream `safetensors`; |
| `q4-8emb/` is a quantization of those weights. |
|
|
