Update

README.md

@@ -26,9 +26,7 @@ metrics:
 ---
 
 # 1. Overview
-A Korean/English **PII detection model for the finance domain**, built by full fine-tuning
-[`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) (1.4B MoE, 50M active) on synthetic finance-domain PII data. It tags **18 PII entity types** (73 BIOES classes) at token level and is intended as the **NER layer of a multi-layer PII-masking gateway** in front of LLM services —
-behind a regex backstop for fully structured identifiers, never as a standalone compliance guarantee.
+A Korean/English **PII detection model for the finance domain**, built by full fine-tuning [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) (1.4B MoE, 50M active) on synthetic finance-domain PII data. It tags **18 PII entity types** (73 BIOES classes) at token level and is intended as the **NER layer of a multi-layer PII-masking gateway** in front of LLM services, as well as a PII detection component for offline privacy review and audit workflows.
 
 On held-out validation it reaches **strict span-F1 0.956 (ko) / 0.969 (en)**. On an independent, adversarially-hardened Golden Set it holds **0.944 (ko) / 0.907 (en)** with **masking coverage 0.996 (ko) / 0.998 (en)** — i.e. ≥99.5% of gold PII characters are covered by predicted spans.
 
@@ -36,14 +34,14 @@ On held-out validation it reaches **strict span-F1 0.956 (ko) / 0.969 (en)**. On
 * **Base model**: [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) — 1.4B-parameter MoE (128 experts, 50M active), 8 layers, hidden 640, bidirectional banded attention (±128), o200k tokenizer
 * **Domain / Language**: Finance (BC Card — cards, accounts, national IDs, customer service text) / Korean + English
 * **Task**: Token classification (BIOES) → character-offset PII spans → masking
-* **Labels (18)**: `PERSON, RRN, FRN, CARD_NUMBER, ACCOUNT_NUMBER, SECRET, USER_ID, EMAIL, PHONE, PASSPORT, DRIVER_LICENSE, GENERIC_ID, ADDRESS, ZIPCODE, DATE, CARD_EXPIRY, CVC, IPIN`
+* **Labels (18)**: `PERSON`, `RRN`, `FRN`, `CARD_NUMBER`, `ACCOUNT_NUMBER`, `SECRET`, `USER_ID`, `EMAIL`, `PHONE`, `PASSPORT`, `DRIVER_LICENSE`, `GENERIC_ID`, `ADDRESS`, `ZIPCODE`, `DATE`, `CARD_EXPIRY`, `CVC`, `IPIN`
 * **Method**: Full fine-tuning (all parameters incl. experts & router) with a re-initialized 73-class head (rows copied from the base head by taxonomy mapping)
-* **Decoding**: **constrained BIOES Viterbi** (not per-token argmax) + whitespace span refinement — the bundled `viterbi_calibration.json` exposes precision↔recall operating-point biases without retraining
+* **Decoding**: constrained BIOES Viterbi (not per-token argmax) + whitespace span refinement — the bundled `viterbi_calibration.json` exposes precision↔recall operating-point biases without retraining
 * **Format**: BF16 (attention `sinks` kept FP32), single safetensors + tokenizer + label taxonomy + Viterbi calibration sidecar
-* **Sequence length**: trained on sequences ≤768 tokens — chunk longer inputs
-* **Intended use**
-  - In-house **PII masking gateway** (detect → mask before text reaches an LLM)
-  - Korean-centric finance text with mixed English (IDs, e-mails, card numbers)
+* **Sequence length**: trained on sequences ≤ 768 tokens — chunk longer inputs
+* **Intended use**:
+  1. In-house PII masking gateway (detect → mask before text reaches an LLM)
+  2. Offline privacy review and audit support (PII discovery in stored text, logs and documents)
 
 ## 1.2. Label Taxonomy (N=18)
 The 18 labels re-map the upstream ai4privacy source labels to the granularity a Korean financial masking policy needs - merging fragments into single spans (`GIVENNAME`/`SURNAME` → `PERSON`, `CITY`/`STREET`/`BUILDINGNUM` → `ADDRESS`) and adding Korea-specific classes absent upstream (`RRN`, `FRN`, `IPIN`, `CARD_EXPIRY`, `CVC`, `SECRET`). `data source` records the row-source buckets in which each label occurs: `ko` means `openpii-1.5m-ko`, `en` means `openpii-1.5m-en`, and `domain` means locally synthesized rows.
@@ -52,7 +50,7 @@ The 18 labels re-map the upstream ai4privacy source labels to the granularity a
 |-------|-------------|-------------|
 | `PERSON` | full name (surname + given, single span) | ko, en, domain |
 | `RRN` | resident registration number (Korea) | ko, domain |
-| `FRN` | foreign registration number | domain |
+| `FRN` | foreign registration number (Korea) | domain |
 | `CARD_NUMBER` | credit/debit card PAN | ko, en, domain |
 | `ACCOUNT_NUMBER` | bank account number | ko, domain |
 | `SECRET` | auth secret (password / API key / token) | ko, domain |
@@ -61,15 +59,15 @@ The 18 labels re-map the upstream ai4privacy source labels to the granularity a
 | `PHONE` | phone number (mobile / landline) | ko, en, domain |
 | `PASSPORT` | passport number | ko, en, domain |
 | `DRIVER_LICENSE` | driver's license number | ko, en, domain |
-| `GENERIC_ID` | generic identifier (no KO counterpart) | ko, en, domain |
+| `GENERIC_ID` | generic identifier without a more specific taxonomy class | ko, en, domain |
 | `ADDRESS` | address (city / street / building, single span) | ko, en, domain |
 | `ZIPCODE` | postal code | ko, en, domain |
 | `DATE` | date / time | ko, en, domain |
 | `CARD_EXPIRY` | card expiry date | domain |
 | `CVC` | card verification code | domain |
-| `IPIN` | I-PIN number | domain |
+| `IPIN` | I-PIN number (Korea only) | domain |
 
-Each entity type has `B-`, `I-`, `E-` and `S-` boundary classes, plus the background class `O`. This yields 73 output classes. The bundled `label-taxonomy.yaml` and `config.json` must remain in the same label order.
+Each entity type has `B-`, `I-`, `E-` and `S-` boundary classes, plus the background class `O`. This yields 73 output classes.
 
 ## 1.3. Usage
 
@@ -87,31 +85,40 @@ enc = tokenizer(text, return_offsets_mapping=True, add_special_tokens=False, ret
 offsets = enc.pop("offset_mapping")[0].tolist()
 
 with torch.no_grad():
-    logits = model(**enc).logits[0]          # [T, 73] — raw output
-
-# 1) Decode the logits with constrained BIOES Viterbi (recommended; see note below)
-#    and map token paths to character spans via `offsets` — the detector's output:
-# -> [{"start": 3, "end": 6, "label": "PERSON"},   '모아이'
-#     {"start": 8, "end": 22, "label": "RRN"},     '000000-0000000'
-#     {"start": 26, "end": 39, "label": "PHONE"}]  '010-0000-0000'
-# 2) Masking is downstream application logic — replace each span according to
-#    your masking policy, e.g.:
-# -> "고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다."
+    logits = model(**enc).logits.float()  # [B, T, 73]
+
+# Decode logits[0] with constrained BIOES Viterbi and map token tags through offsets.
+print(tuple(logits.shape))
+```
+
+Raw logits shape and decoded spans:
+
+```text
+(1, 30, 73)
+[
+  {'start': 3, 'end': 6, 'label': 'PERSON'},
+  {'start': 8, 'end': 22, 'label': 'RRN'},
+  {'start': 26, 'end': 39, 'label': 'PHONE'}
+]
+```
+
+The offsets use Python's half-open character interval `[start, end)`. Masking is downstream policy logic. For example, the spans above can produce:
+
+```text
+고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다.
 ```
 
-> **Decoding note** — this model (like its base) is post-trained for **constrained Viterbi decoding**
-> over the BIOES transition grammar, *not* independent per-token argmax. Argmax can emit invalid tag
-> sequences (span splits / orphan tags) and measurably lowers span-F1. The bundled
-> `viterbi_calibration.json` follows the upstream operating-point schema: its six transition biases
-> shift the precision↔recall trade-off at deploy time without retraining (all `0.0` = neutral).
+For batches, enable right padding and pass only `input_ids` and `attention_mask` to the model. `offset_mapping` stays outside the model and is used only to map decoded token tags back to the original text. Convert logits to FP32 before constrained Viterbi decoding, as shown above.
+
+> **Decoding note** - this model, like its base, is trained with a supervised token-level BIOES classification objective and is intended to be decoded with **constrained Viterbi** over the BIOES transition grammar, not independent per-token argmax. Independent argmax can emit invalid BIOES sequences and is not the decoding path used for the reported metrics. The bundled `viterbi_calibration.json` follows the upstream operating-point schema. Its six transition biases allow users to adjust the precision-recall trade-off without retraining. All-zero biases mean no additive operating-point adjustment; BIOES transition constraints remain active.
 
 ## 1.4. Training Data
 | Dataset | Role | Size |
 |---------|------|------|
-| (Public) [BCCard/pii-masking-openpii-finance](https://huggingface.co/datasets/BCCard/pii-masking-openpii-finance) (v2) | Training / validation | ~58.5k train rows · ~14.5k validation rows |
-| (Private) BCCard/pii-masking-openpii-finance-test (v2) | Golden Set (release gate; never used for training/tuning) | 2,000 rows (ko 1,460 / en 540) |
+| (Public) [BCCard/pii-masking-openpii-finance](https://huggingface.co/datasets/BCCard/pii-masking-openpii-finance) (v2) | Training / Validation | ~58.5k train rows · ~14.5k validation rows |
+| (Private) BCCard/pii-masking-openpii-finance-test (v2) | Golden Set (release evaluation; not used for training/tuning) | 2,000 rows (ko 1,460 / en 540) |
 
-* Sources: curated Korean subset of `ai4privacy/pii-masking-openpii-1.5m` (label taxonomy remapped, name spans merged & naturalized) + finance-domain synthetic templates + **~30% English replay** (forgetting guard)
+* Sources: curated Korean subset of `ai4privacy/pii-masking-openpii-1.5m` (label taxonomy remapped, name spans merged & naturalized) + finance-domain synthetic templates + **~30% English replay** (catastrophic forgetting guard)
 * Hard-example design baked into v2: surface-similar non-PII decoys (FP suppression), label-confusion pairs in one sentence (RRN↔FRN, DRIVER_LICENSE↔GENERIC_ID), weak-context true PII (FN suppression), long-span address boundary variants
 * All values are synthetic; validity-pattern collisions with real identifiers are removed at generation time (e.g. card numbers are forced to fail Luhn)
 
@@ -124,7 +131,7 @@ with torch.no_grad():
 | Batch | effective 16 (per-device × world × accum), fixed across hardware layouts |
 | LR / scheduler | 1e-4 / linear decay, warmup 3% |
 | Optimizer | AdamW (fused), weight decay 0.0, max_grad_norm 1.0 |
-| Epochs | 5 — best checkpoint by validation span micro-F1, **decoded with the same constrained Viterbi as deployment** |
+| Epochs | 5 — best checkpoint by validation span micro-F1, decoded with the same constrained Viterbi as deployment |
 | Precision | FP32 master weights + BF16 autocast; MoE router/experts explicitly kept FP32 during compute |
 | Hardware | 1× NVIDIA H100 (~5h) |
 
@@ -146,7 +153,6 @@ with torch.no_grad():
 
 ## 2.2. Results
 ### `validation` dataset
-
 | Metric | v2 model / v2 validation |
 |---|---:|
 | micro F1 | 0.9599 |
@@ -159,6 +165,8 @@ with torch.no_grad():
 | **ko masking coverage** | **0.9987** |
 | **en masking coverage** | **0.9965** |
 
+**Observed masking coverage is 99.8% overall and at least 99.6% in both the Korean and English validation slices.** This leakage-oriented metric is reported as a diagnostic rather than a release gate.
+
 * These values were measured post-hoc by running the exported `final-bf16` artifact over all 14,543 v2 validation rows (ko 10,460 / en 4,083) through the deployment-equivalent chain: constrained Viterbi, actual tokenizer character offsets and whitespace refinement.
 * Overall micro F1 and masking coverage pool all ko/en spans or characters before scoring. Overall macro F1 pools per-label TP/FP/FN across both languages and then averages the 18 label F1 values.
 * Character coverage counts are ko **482,861 / 483,488** and en **264,979 / 265,898** gold PII characters.
@@ -186,7 +194,7 @@ Independently generated Golden Set — deliberately harder than validation: weak
 * Label-swap errors (e.g. en `ACCOUNT_NUMBER` predicted as `GENERIC_ID`/`CARD_NUMBER`) keep **coverage 1.0** — the value is still masked; only the label name is wrong
 
 ## 2.3. Reading the numbers
-Strict exact-match span-F1 on an adversarial test is a deliberately harsh score: a one-character boundary miss or a swapped label counts as a full error. For the product question — *"how much PII text leaks through?"* — masking coverage is the operative metric: **0.44% (ko) / 0.16% (en) of gold PII characters uncovered**, concentrated in weak-context person names.
+Strict exact-match span-F1 on an adversarial test is a deliberately harsh score: a one-character boundary miss or a swapped label counts as a full error. For diagnosing character-level exposure, masking coverage is the direct diagnostic metric: **0.44% (ko) / 0.16% (en) of gold PII characters uncovered**, concentrated in weak-context person names.
 
 <br>
 
@@ -201,7 +209,7 @@ Strict exact-match span-F1 on an adversarial test is a deliberately harsh score:
 <br>
 
 # 3. Future Work
-* **v3 data** — weak-context person-name hard positives, cue-word diversification for the alphanumeric ID group, privacy-safe failure-collection loop from shadow operation
+* **v3 data enhancements** - weak-context person-name hard positives, more diverse cue words for alphanumeric IDs, and privacy-safe failure collection from shadow-mode operation
 * **Operating point** - recall-leaning Viterbi transition biases tuned on validation or a dedicated calibration set without Golden Set feedback
 * **Serving** - target-hardware latency, throughput and memory benchmarks for the separately published INT8 weight-only ONNX artifact