docs: trim internal training-provenance from the model card
Browse files
README.md
CHANGED
|
@@ -15,52 +15,31 @@ tags:
|
|
| 15 |
|
| 16 |
# onnx-email-gate β multilingual KEEP/DROP email prefilter
|
| 17 |
|
| 18 |
-
A tiny
|
| 19 |
-
|
| 20 |
-
front of
|
| 21 |
-
|
| 22 |
|
| 23 |
- **Labels:** `{0: DROP, 1: KEEP}`
|
| 24 |
- **Architecture:** `paraphrase-multilingual-MiniLM-L12-v2` sentence embedder (mean-pool +
|
| 25 |
-
L2-normalize) with a logistic-regression head
|
| 26 |
-
|
| 27 |
-
- **Quantization:** dynamic **INT8** (`model_int8.onnx`, ~119 MB
|
| 28 |
-
- **Runtime:** `onnxruntime` on CPU
|
| 29 |
|
| 30 |
## Why multilingual
|
| 31 |
|
| 32 |
-
The embedder
|
| 33 |
-
|
| 34 |
-
**Spanish** applications and junk (5/5).
|
| 35 |
|
| 36 |
-
##
|
| 37 |
|
| 38 |
-
The model is **
|
| 39 |
-
|
| 40 |
|
| 41 |
-
1. **KEEP overrides** β rΓ©sumΓ© attachment /
|
| 42 |
-
recruiting-forward β **KEEP** (bypass, straight to the GPU).
|
| 43 |
2. **regex junk** β noreply / notifications / OTP / bounce / job-board-alert / newsletter β **DROP**.
|
| 44 |
-
3. **this model** β KEEP/DROP on the
|
| 45 |
-
|
| 46 |
-
`DROP` rows are labelled `OTHER` and the GPU is skipped; `KEEP` rows go to the GPU.
|
| 47 |
-
Ships in **shadow mode** first (log would-drops, still call the GPU), then flip to hard-drop.
|
| 48 |
-
|
| 49 |
-
## Training
|
| 50 |
-
|
| 51 |
-
- **3,953** real inbound emails (dev + a small prod read-only slice), deduped, **per-sender
|
| 52 |
-
capped at 30** to stop a few high-volume senders dominating the KEEP class.
|
| 53 |
-
- Labelled by a **local 35B** model on the ambiguous residual; clear cases auto-labelled by rule.
|
| 54 |
-
- Head: `LogisticRegression(class_weight="balanced")` on frozen, L2-normalized embeddings.
|
| 55 |
-
|
| 56 |
-
## Metrics
|
| 57 |
-
|
| 58 |
-
- **Model-only** on a 303-row hand-checked gold set: accuracy **0.868**, KEEP recall 0.80.
|
| 59 |
-
Most model-only misses are *forwarded* applications β rescued by the KEEP override ladder.
|
| 60 |
-
- **With the ladder (production):** the overrides take the raw model to a high-precision gate;
|
| 61 |
-
bake-off showed β 0β3 real candidates lost in-distribution.
|
| 62 |
-
- **INT8 vs fp32 parity:** 97.4% on gold (flips lean toward keeping candidates); multilingual
|
| 63 |
-
concrete cases unchanged (5/5).
|
| 64 |
|
| 65 |
## Usage
|
| 66 |
|
|
@@ -89,15 +68,14 @@ Input text = `from_address\nfrom_name\nsubject\nbody`, truncated to 128 tokens.
|
|
| 89 |
|
| 90 |
| File | Purpose |
|
| 91 |
|------|---------|
|
| 92 |
-
| `model_int8.onnx` | the model β embedder + folded
|
| 93 |
| `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json` | tokenizer |
|
| 94 |
| `config.json` | model config + `id2label` |
|
| 95 |
-
| `logreg_head.npz`, `gate_meta.json` |
|
| 96 |
|
| 97 |
## Limitations
|
| 98 |
|
| 99 |
-
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
- INT8 shifts ~2.6% of borderline predictions vs fp32; ship fp32 if a use case needs exact parity.
|
|
|
|
| 15 |
|
| 16 |
# onnx-email-gate β multilingual KEEP/DROP email prefilter
|
| 17 |
|
| 18 |
+
A tiny **multilingual** CPU classifier that labels an inbound email **KEEP** (looks like a real
|
| 19 |
+
job application β worth further processing) or **DROP** (junk β skip). It's meant as a cheap
|
| 20 |
+
pre-filter in front of a more expensive downstream model, discarding obvious junk β newsletters,
|
| 21 |
+
notifications, one-time codes, job-board alerts, bounces β at roughly a millisecond per email.
|
| 22 |
|
| 23 |
- **Labels:** `{0: DROP, 1: KEEP}`
|
| 24 |
- **Architecture:** `paraphrase-multilingual-MiniLM-L12-v2` sentence embedder (mean-pool +
|
| 25 |
+
L2-normalize) with a logistic-regression head folded into the ONNX graph as a final linear
|
| 26 |
+
layer β so the whole thing is one classifier ONNX: `tokens β 2 logits β argmax`.
|
| 27 |
+
- **Quantization:** dynamic **INT8** (`model_int8.onnx`, ~119 MB).
|
| 28 |
+
- **Runtime:** `onnxruntime` on CPU, ~ms per email.
|
| 29 |
|
| 30 |
## Why multilingual
|
| 31 |
|
| 32 |
+
The embedder covers ~50 languages, so the gate reads non-English application emails directly
|
| 33 |
+
instead of wrong-dropping them (verified on English, Hindi, and Spanish).
|
|
|
|
| 34 |
|
| 35 |
+
## Recommended use β an override ladder
|
| 36 |
|
| 37 |
+
The model is best used as the **last rung** of a cheap rule ladder, so a genuine application is
|
| 38 |
+
never dropped by the model alone:
|
| 39 |
|
| 40 |
+
1. **KEEP overrides** β rΓ©sumΓ© attachment / forwarded application / recruiter-style sender β **KEEP**.
|
|
|
|
| 41 |
2. **regex junk** β noreply / notifications / OTP / bounce / job-board-alert / newsletter β **DROP**.
|
| 42 |
+
3. **this model** β KEEP/DROP on the remaining ambiguous mail.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
## Usage
|
| 45 |
|
|
|
|
| 68 |
|
| 69 |
| File | Purpose |
|
| 70 |
|------|---------|
|
| 71 |
+
| `model_int8.onnx` | the model β embedder + folded logistic-regression head, INT8 |
|
| 72 |
| `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json` | tokenizer |
|
| 73 |
| `config.json` | model config + `id2label` |
|
| 74 |
+
| `logreg_head.npz`, `gate_meta.json` | raw head weights + fold record (reproducibility) |
|
| 75 |
|
| 76 |
## Limitations
|
| 77 |
|
| 78 |
+
- The model alone has modest recall β **use it behind the override ladder**, not standalone.
|
| 79 |
+
- Dynamic INT8 shifts a small fraction of borderline predictions vs fp32; ship fp32 if you need
|
| 80 |
+
exact parity.
|
| 81 |
+
- Very low-resource languages may be weaker than the ~50 the embedder covers well.
|
|
|