pranav0494 commited on
Commit
c4516f7
Β·
verified Β·
1 Parent(s): a3fd3de

docs: trim internal training-provenance from the model card

Browse files
Files changed (1) hide show
  1. README.md +21 -43
README.md CHANGED
@@ -15,52 +15,31 @@ tags:
15
 
16
  # onnx-email-gate β€” multilingual KEEP/DROP email prefilter
17
 
18
- A tiny, **multilingual** CPU classifier that decides **KEEP** (looks like a real
19
- candidate email β†’ send to the GPU model) or **DROP** (junk β†’ skip the GPU). It sits in
20
- front of the expensive GPU email-intent classifier (Qwen) in Curriculo's pipeline and
21
- exists purely to **save GPU calls** by discarding obvious junk cheaply.
22
 
23
  - **Labels:** `{0: DROP, 1: KEEP}`
24
  - **Architecture:** `paraphrase-multilingual-MiniLM-L12-v2` sentence embedder (mean-pool +
25
- L2-normalize) with a logistic-regression head **folded into the ONNX graph** as a final
26
- linear layer, so the whole thing is one classifier ONNX: `tokens β†’ 2 logits β†’ argmax`.
27
- - **Quantization:** dynamic **INT8** (`model_int8.onnx`, ~119 MB, down 4Γ— from ~471 MB fp32).
28
- - **Runtime:** `onnxruntime` on CPU (AVX512-VNNI). ~ms per email.
29
 
30
  ## Why multilingual
31
 
32
- The embedder is trained on ~50 languages, so the gate reads non-English candidate emails
33
- directly instead of wrong-dropping them. Verified KEEP/DROP on English, **Hindi**, and
34
- **Spanish** applications and junk (5/5).
35
 
36
- ## How it's used (the override ladder)
37
 
38
- The model is **rung 3** of a cheap ladder that runs in the cron β€” the overrides are the
39
- real safety net and run FIRST so a genuine candidate is never dropped by the model:
40
 
41
- 1. **KEEP overrides** β€” rΓ©sumΓ© attachment / ZipRecruiter forward / forwarded application /
42
- recruiting-forward β†’ **KEEP** (bypass, straight to the GPU).
43
  2. **regex junk** β€” noreply / notifications / OTP / bounce / job-board-alert / newsletter β†’ **DROP**.
44
- 3. **this model** β€” KEEP/DROP on the fuzzy residual.
45
-
46
- `DROP` rows are labelled `OTHER` and the GPU is skipped; `KEEP` rows go to the GPU.
47
- Ships in **shadow mode** first (log would-drops, still call the GPU), then flip to hard-drop.
48
-
49
- ## Training
50
-
51
- - **3,953** real inbound emails (dev + a small prod read-only slice), deduped, **per-sender
52
- capped at 30** to stop a few high-volume senders dominating the KEEP class.
53
- - Labelled by a **local 35B** model on the ambiguous residual; clear cases auto-labelled by rule.
54
- - Head: `LogisticRegression(class_weight="balanced")` on frozen, L2-normalized embeddings.
55
-
56
- ## Metrics
57
-
58
- - **Model-only** on a 303-row hand-checked gold set: accuracy **0.868**, KEEP recall 0.80.
59
- Most model-only misses are *forwarded* applications β€” rescued by the KEEP override ladder.
60
- - **With the ladder (production):** the overrides take the raw model to a high-precision gate;
61
- bake-off showed β‰ˆ 0–3 real candidates lost in-distribution.
62
- - **INT8 vs fp32 parity:** 97.4% on gold (flips lean toward keeping candidates); multilingual
63
- concrete cases unchanged (5/5).
64
 
65
  ## Usage
66
 
@@ -89,15 +68,14 @@ Input text = `from_address\nfrom_name\nsubject\nbody`, truncated to 128 tokens.
89
 
90
  | File | Purpose |
91
  |------|---------|
92
- | `model_int8.onnx` | the model β€” embedder + folded LogReg head, INT8 |
93
  | `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json` | tokenizer |
94
  | `config.json` | model config + `id2label` |
95
- | `logreg_head.npz`, `gate_meta.json` | reproducibility β€” raw head weights + fold record |
96
 
97
  ## Limitations
98
 
99
- - English-only vocabulary would wrong-drop non-English candidates; this multilingual variant
100
- is chosen precisely to avoid that, but very low-resource languages may be weaker.
101
- - The model alone has modest recall; **do not ship it without the override ladder** β€” the
102
- ladder, not the model, is what makes the gate safe for real candidates.
103
- - INT8 shifts ~2.6% of borderline predictions vs fp32; ship fp32 if a use case needs exact parity.
 
15
 
16
  # onnx-email-gate β€” multilingual KEEP/DROP email prefilter
17
 
18
+ A tiny **multilingual** CPU classifier that labels an inbound email **KEEP** (looks like a real
19
+ job application β†’ worth further processing) or **DROP** (junk β†’ skip). It's meant as a cheap
20
+ pre-filter in front of a more expensive downstream model, discarding obvious junk β€” newsletters,
21
+ notifications, one-time codes, job-board alerts, bounces β€” at roughly a millisecond per email.
22
 
23
  - **Labels:** `{0: DROP, 1: KEEP}`
24
  - **Architecture:** `paraphrase-multilingual-MiniLM-L12-v2` sentence embedder (mean-pool +
25
+ L2-normalize) with a logistic-regression head folded into the ONNX graph as a final linear
26
+ layer β€” so the whole thing is one classifier ONNX: `tokens β†’ 2 logits β†’ argmax`.
27
+ - **Quantization:** dynamic **INT8** (`model_int8.onnx`, ~119 MB).
28
+ - **Runtime:** `onnxruntime` on CPU, ~ms per email.
29
 
30
  ## Why multilingual
31
 
32
+ The embedder covers ~50 languages, so the gate reads non-English application emails directly
33
+ instead of wrong-dropping them (verified on English, Hindi, and Spanish).
 
34
 
35
+ ## Recommended use β€” an override ladder
36
 
37
+ The model is best used as the **last rung** of a cheap rule ladder, so a genuine application is
38
+ never dropped by the model alone:
39
 
40
+ 1. **KEEP overrides** β€” rΓ©sumΓ© attachment / forwarded application / recruiter-style sender β†’ **KEEP**.
 
41
  2. **regex junk** β€” noreply / notifications / OTP / bounce / job-board-alert / newsletter β†’ **DROP**.
42
+ 3. **this model** β€” KEEP/DROP on the remaining ambiguous mail.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
  ## Usage
45
 
 
68
 
69
  | File | Purpose |
70
  |------|---------|
71
+ | `model_int8.onnx` | the model β€” embedder + folded logistic-regression head, INT8 |
72
  | `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json` | tokenizer |
73
  | `config.json` | model config + `id2label` |
74
+ | `logreg_head.npz`, `gate_meta.json` | raw head weights + fold record (reproducibility) |
75
 
76
  ## Limitations
77
 
78
+ - The model alone has modest recall β€” **use it behind the override ladder**, not standalone.
79
+ - Dynamic INT8 shifts a small fraction of borderline predictions vs fp32; ship fp32 if you need
80
+ exact parity.
81
+ - Very low-resource languages may be weaker than the ~50 the embedder covers well.