Add privacy-filter README.md

privacy/README.md

@@ -0,0 +1,152 @@
+---
+library_name: cellm
+license: apache-2.0
+base_model: openai/privacy-filter
+pipeline_tag: token-classification
+tags:
+- pii
+- privacy
+- token-classification
+- rust
+- quantized
+- mixture-of-experts
+- on-device
+---
+
+# privacy-filter (cellm)
+
+[`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) converted
+to the cellm format and quantized for on-device inference. Runs in Rust on CPU
+with no Python, no ONNX runtime, and no network access — the point of a PII
+filter is that the text never leaves the machine.
+
+Three quantization recipes are published. **`int4-g32` is the recommended
+build**; the smaller two are included because the size/accuracy trade is a real
+decision for mobile bundles and the numbers below are measured, not estimated.
+
+## Files
+
+| File | Size | Recommended |
+| --- | --- | --- |
+| `privacy-filter-int4-g32-f16s.cellm` | 899 MB | **yes** |
+| `privacy-filter-int3-g128.cellm` | 667 MB | size-constrained only |
+| `privacy-filter-int2-g32.cellm` | 599 MB | not for compliance use |
+| `tokenizer.json` | 27.9 MB | required by all three |
+
+## Model details
+
+- **Base**: `openai/privacy-filter` (2.6 GB fp32 safetensors)
+- **Architecture**: sparse MoE token classifier, 8 layers, 128 experts, top-4
+- **Hidden size**: 640 · **Heads**: 14 query / 2 key-value · **Head dim**: 64
+- **Vocab**: 200,064 · **Attention**: bidirectional, sliding window 128
+- **RoPE**: YaRN, theta 150,000, factor 32
+- **Labels**: 33 BIOES tags over 8 entity types
+
+Experts are 84% of the file, so quantization targets them: each expert is stored
+transposed (`[E, K, N]` → `[N, K]`) and group-quantized along `K` with affine
+per-group scale and bias in f16. Embeddings are int8 with per-row f16 scales — a
+fixed 122.5 MB floor, since 200,064 × 640 cannot be compressed further without
+hurting rare-token recall. Everything else stays f16.
+
+### Entity types
+
+`account_number`, `private_address`, `private_date`, `private_email`,
+`private_person`, `private_phone`, `private_url`, `secret`
+
+## Usage
+
+```bash
+./target/release/pii privacy-filter-int4-g32-f16s.cellm \
+  --tokenizer tokenizer.json \
+  --text "Contact Bob Smith at bob.smith@example.com or (555) 123-4567."
+```
+
+```
+=== Contact Bob Smith at bob.smith@example.com or (555) 123-4567.
+    private_person   [   7:  17]  " Bob Smith"
+    private_email    [  20:  42]  " bob.smith@example.com"
+    private_phone    [  45:  60]  " (555) 123-4567"
+```
+
+`--redact` replaces each span with its label; `--dump-logits` prints per-token
+scores. Spans are byte offsets into the input, so redaction is exact.
+
+## Benchmark
+
+140 texts / 12,740 chars / 366 gold spans. Ground truth is the **fp32 HF model's
+own output** (transformers 5.6.2, torch 2.10.0) — this measures quantization
+damage, not the base model's absolute accuracy. A span counts as correct only if
+both its label and its exact boundaries match.
+
+| Build | Size | Recall | Precision | F1 | Leak |
+| --- | --- | --- | --- | --- | --- |
+| **int4 g32** | **899 MB** | **0.978** | **0.975** | **0.977** | **2.2%** |
+| int3 g128 | 667 MB | 0.954 | 0.923 | 0.938 | 4.6% |
+| int2 g32 | 599 MB | 0.934 | 0.919 | 0.927 | 6.6% |
+
+"Leak" is the fraction of fp32-detected PII that the quantized build misses.
+
+### Recall by entity type
+
+| Entity type | Gold | int4 g32 | int3 g128 | int2 g32 |
+| --- | --- | --- | --- | --- |
+| `private_person` | 96 | 1.000 | 1.000 | 1.000 |
+| `private_phone` | 22 | 1.000 | 1.000 | 1.000 |
+| `private_email` | 87 | 1.000 | 1.000 | 0.954 |
+| `private_url` | 13 | 1.000 | 0.923 | 0.769 |
+| `secret` | 26 | 0.962 | 0.923 | 1.000 |
+| `private_address` | 42 | 0.952 | 0.833 | 0.881 |
+| `account_number` | 35 | 0.943 | 0.914 | 0.829 |
+| `private_date` | 45 | 0.933 | 0.911 | 0.867 |
+
+People, phones and emails survive aggressive quantization; **structured
+identifiers and addresses do not**. int2 drops 23% of URLs and 17% of account
+numbers, and its failure mode is worse than a clean miss — it *fragments* spans,
+returning `.com` for a full email address. A truncated redaction looks like a
+success to the calling code while still leaking the local part.
+
+### Resources
+
+Apple M4, CPU backend, cold start on a 30-char input:
+
+| Build | Load + infer | Peak RSS |
+| --- | --- | --- |
+| int4 g32 | 0.92 s | 479 MB |
+| int3 g128 | 0.78 s | 440 MB |
+| int2 g32 | 0.75 s | 432 MB |
+
+Peak RSS is roughly half the file size: weights are mmapped and only 4 of 128
+experts are touched per token, so the resident set never approaches the
+on-disk footprint. **If your constraint is memory rather than bundle size, the
+899 MB build already fits in ~479 MB** and the smaller variants buy you little.
+
+## Known limitations
+
+These are properties of the base model and are present in all three builds:
+
+- **CVV codes and SWIFT/BIC codes are not detected.**
+- Bank sort codes (`11-01-04`) are labelled `private_date`.
+- IP addresses are labelled `private_url`.
+- Organization names in a PII-dense context are over-flagged as
+  `private_address` — expect over-redaction of company names.
+- Documentation placeholders are correctly ignored (`AKIAIOSFODNN7EXAMPLE`
+  scores `O` at p≈1.0) while real-looking keys are caught. Do not benchmark this
+  model with placeholder secrets; you will conclude it is broken.
+- Boundary handling is left-inclusive of a preceding space. Trim before
+  comparing spans against another tagger.
+
+## Reproducing
+
+```bash
+python tools/convert_privacy_filter_hf.py <hf_dir> out.cellm \
+  --quant int4 --group-size 32 --quant-embedding
+```
+
+`--quant` accepts `int8`, `int4`, `int3`, `int2`; `--group-size` sets the
+quantization group. Note that int3 packs 10 values per `u32` and wastes the top
+2 bits, so a 3-bit file is ~6.7% larger than the ideal bit count implies — which
+is why `int3 g128` is 667 MB and not the 630 MB a naive calculation gives.
+
+## License
+
+Apache-2.0, inherited from `openai/privacy-filter`.