2.0.6: a model span that stops inside a number grows over the whole number

CHANGELOG.md

@@ -8,6 +8,24 @@ Semver for this island:
 
 Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
 
+## 2.0.6
+
+Same NERGAL weights, threshold, union, rules (SHA `4d054723…`), names layer and API. One change in `nergal.py` `model_keep`: a kept model span that starts or ends inside a run of digits grows over the whole run, before the phone cut. Rule spans and person spans are unchanged.
+
+- **A model span no longer stops mid-number.** XLM-R scores every digit as its own unit, so a kept span could start or end inside a number and leave the rest visible: `NIP [PII]7890`. 2.0.6 grows it to both ends of the run (`NIP [PII]`). No gold span cuts a digit run (0 of the bench's 790 `pii`/`phone` spans, 0 of name tests v1/v2's 73), so the policy is the whole run at every length and for both labels.
+- **It never creates a span.** It only lengthens a span the model kept at ≥ 0.95, and only over digits. Groups split by a space (`28 1020 5558`) are separate runs and are not joined. A number the model wrongly flagged is now masked whole instead of in part.
+- **Phone policy v3 applies to the grown span.** A phone span holding one full number and the head of a second (`601 234 567, 6012…`) stayed one span; grown, both numbers are full and it becomes one span per number, so the separator between them is unmasked. A cut phone under 7 digits was dropped; grown to 7+ digits it is kept.
+- **Measured on stored spans.** 2.0.5 left a cut edge in 131 of 34,715 UZP rulings, 4 of 2,237 bench rows, 2 of 1,000 Dynaword documents and no name-test passage. 2.0.5 → 2.0.6: UZP 92 rulings change output and 408 characters gain a mask (10–13-digit `pii` runs completed: 51, 37 of them after `nr`); FineWeb miss hunt 47 of 41,204 passages, 166 characters; Dynaword 1 document, 1 phone digit.
+
+Still missed: the space-separated groups of a number the model did not flag; a number the model never flagged. The UZP runs completed after `nr` have no gold; most are 12 digits, and whether they are identifiers or notice and case references is unmeasured. If they are references, 2.0.5 already masked them in part, so 2.0.6 adds false characters there, not false passages.
+
+841-dev, names off: unchanged from 2.0.5. Release gates (`artifacts/nergal_2_0_6_digit_tail/`, frozen before scoring, stored spans, no new inference): rule spans identical, no two 2.0.6 spans overlap, and the 2.0.5 side rebuilds the stored Dynaword spans. Two gates failed as frozen: 7 characters lose a mask and 5 non-digit characters gain one, on four documents (UZP 1, miss hunt 3). All are the phone policy above (the separator between two numbers; a short phone's `+`, `-` and spaces), and no digit loses its mask; the gates assumed growth moves digits only. `test_nergal.py` gained the two-number case after the score. Label pool v3 `bench`, names off: 4 of 22 cohorts move, all up, with fp and false-redaction passages unchanged everywhere. `human_test_v2/bare_id` character recall 0.528 → 0.717 (fn 25 → 15); `human_test_v2/nip_regon` union exact precision 0.911 → 0.932; `human_test_v2/email` union exact precision 0.851 → 0.859; `mc4_negatives_v1/email` whole 99 → 100. `--names` and name tests v1/v2: no change. Role bench: as 2.0.5. `hybrid.json` `eval` and `eval_names` are unchanged.
+
+| Version | Whole /315 | Residual | Rules FP | Union FP | Char P | Char R | What changed |
+|---|---:|---:|---:|---:|---:|---:|---|
+| 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
+| 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
+
 ## 2.0.5
 
 Same NERGAL weights, threshold, union, rules (SHA `4d054723…`) and API. One change in `nergal.py` `person_spans`: the names layer (`names=True`) leaves more role nouns, and a number right after one, out of a person span. Names off is unchanged.

README.md

@@ -16,7 +16,7 @@ tags:
 - hybrid
 ---
 
-# NERGAL 2.0.5
+# NERGAL 2.0.6
 
 **Named Entity Recognition with Grounded Additive Labels**
 
@@ -66,9 +66,9 @@ NERGAL is not designed to remove:
 
 These are intended exclusions; false positives can still mask some of this content.
 
-### Known gaps in 2.0.5
+### Known gaps in 2.0.6
 
-Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.5 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. Names-on gaps are under [Names](#names).
+Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.6 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. Names-on gaps are under [Names](#names).
 
 ## Versions
 
@@ -83,7 +83,8 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
 | 2.0.2, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A rule's label wins on its characters |
 | 2.0.3, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Values cut or preceded by a placeholder |
 | 2.0.4, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves procedural role nouns out |
-| **2.0.5, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | Names layer leaves more role nouns, and a number right after one, out |
+| 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
+| **2.0.6, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | A model span that stops inside a number grows over the whole number |
 
 1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
 
@@ -100,7 +101,7 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
 | Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
 | Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
 
-Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 gives the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
+Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 and 2.0.6 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
 
 **Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
 
@@ -112,7 +113,7 @@ Names off hides no name. A component is the independent unit, covered when every
 | Skarżący | 50 | 3 | 3 | 0 | **0** |
 | Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
 
-The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). The texts hold real names and are not released.
+The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The texts hold real names and are not released.
 
 **Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
 
@@ -126,7 +127,7 @@ The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodn
 
 ## Compared with other systems
 
-Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.5 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
+Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.6 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
 
 | System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
 |---|---|---:|---:|---:|---:|---:|
@@ -136,9 +137,9 @@ Same split and gold. Every model runs through the NERGAL recipe with names off:
 | Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
 | Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
 | XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
-| **NERGAL 2.0.5, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
+| **NERGAL 2.0.6, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
 
-Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.5 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
+Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.6 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
 
 XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
 

hybrid.json

@@ -1,7 +1,7 @@
 {
   "full_name": "Named Entity Recognition with Grounded Additive Labels",
   "hub_id": "SlayerLab/NERGAL",
-  "version": "2.0.5",
+  "version": "2.0.6",
   "mode": "rules_union",
   "epoch": 5,
   "seed": 202609160,

nergal.py

@@ -17,7 +17,7 @@ import scrub_pii
 from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
 
 HUB_ID = 'SlayerLab/NERGAL'
-VERSION = '2.0.5'
+VERSION = '2.0.6'
 GAPS = ['[PII_SPACE]', '[PII_BREAK]']
 GAP_IDS = [250002, 250003]
 BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
@@ -212,6 +212,18 @@ def expand(text, start, end):
     return start, end
 
 
+def whole_digits(text, start, end):
+    """Grow a span that starts or ends inside a run of digits over the whole run (2.0.6). No gold span cuts a digit
+    run, so a model span stopping mid-number leaves the rest of an identifier open."""
+    if text[start].isdecimal():
+        while start > 0 and text[start - 1].isdecimal():
+            start -= 1
+    if text[end - 1].isdecimal():
+        while end < len(text) and text[end].isdecimal():
+            end += 1
+    return start, end
+
+
 def person_spans(text, ranges):
     """Person spans from FastPDN person-token ranges: whole words, joined across spaces, tabs and no-break spaces
     (not punctuation or line breaks); words touching a placeholder, role nouns (ROLE_NOUNS) and a number
@@ -272,11 +284,14 @@ def apply_union(text, spans, firm=()):
 
 
 def model_keep(text, model_spans, *, threshold=THRESHOLD, module=scrub_pii):
-    """Decoded model spans, each phone span cut by the rules' phone policy: one span per number of 7+ digits; short
-    and emergency numbers alone are dropped. Other labels pass unchanged."""
-    return [dict(s, start=a, end=b) for s in decode(model_spans, threshold)
-            for a, b in (module._phone_spans(text, s['start'], s['end']) if s['label'] == 'phone'
-                         else [(s['start'], s['end'])])]
+    """Decoded model spans grown over whole runs of digits (whole_digits), then each phone span cut by the rules'
+    phone policy: one span per number of 7+ digits; short and emergency numbers alone are dropped."""
+    keep = []
+    for s in decode(model_spans, threshold):
+        start, end = whole_digits(text, s['start'], s['end'])
+        parts = module._phone_spans(text, start, end) if s['label'] == 'phone' else [(start, end)]
+        keep += [dict(s, start=a, end=b) for a, b in parts]
+    return keep
 
 
 def scrub_spans(text, rule_spans, model_spans, *, threshold=THRESHOLD, module=scrub_pii, name_spans=()):

test_nergal.py

@@ -43,7 +43,7 @@ class NergalTests(unittest.TestCase):
         from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
         card = json.loads((HERE / 'hybrid.json').read_text())
         self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
-        self.assertEqual(VERSION, '2.0.5')
+        self.assertEqual(VERSION, '2.0.6')
         self.assertEqual(card['version'], VERSION)
         self.assertEqual(card['eval']['union_fp'], 80)
         self.assertEqual(card['eval']['rules_fp'], 24)
@@ -349,6 +349,31 @@ class NergalTests(unittest.TestCase):
         self.assertEqual(scrub_spans(text, [], [span(text, '112')])[0], text)
         self.assertEqual(model_keep(text, [span(text, '112', score=0.9)], threshold=0.95), [])
 
+    def test_model_spans_cover_whole_digit_runs(self):
+        from nergal import model_keep, scrub_spans, whole_digits
+        span = lambda text, start, end, label='pii', score=0.99: dict(start=start, end=end, label=label, score=score)
+        kept = lambda text, *spans: [text[s['start']:s['end']] for s in model_keep(text, list(spans))]
+        text = 'NIP 1234567890, tel. 601234567'
+        self.assertEqual(kept(text, span(text, 4, 10)), ['1234567890'])             # the tail was left
+        self.assertEqual(kept(text, span(text, 8, 14)), ['1234567890'])             # the head was left
+        self.assertEqual(kept(text, span(text, 0, 7)), ['NIP 1234567890'])          # a word before stays in
+        self.assertEqual(kept(text, span(text, 4, 14)), ['1234567890'])             # whole already
+        self.assertEqual(kept(text, span(text, 21, 28, 'phone')), ['601234567'])    # phone, then the phone policy
+        self.assertEqual(kept(text, span(text, 21, 25, 'phone')), ['601234567'])    # 4 digits alone would drop
+        two = 'tel. 601 234 567, 601234567.'
+        # 2.0.5 kept '601 234 567, 6012' whole (one full number); grown, the second is full and the policy splits them
+        self.assertEqual(kept(two, span(two, 5, 22, 'phone')), ['601 234 567', '601234567'])
+        self.assertEqual(kept(text, span(text, 0, 3)), ['NIP'])                     # no digit at an edge
+        text = 'nip:1234567890abc 28 1020 5558'
+        self.assertEqual(kept(text, span(text, 4, 9)), ['1234567890'])              # stops at the glued letters
+        self.assertEqual(kept(text, span(text, 18, 23)), ['28 1020'])               # one run, not the next group
+        self.assertEqual(whole_digits('12 34', 0, 5), (0, 5))
+        self.assertEqual(whole_digits('a1b', 1, 2), (1, 2))
+        text = 'NIP 1234567890.'
+        self.assertEqual(scrub_spans(text, [], [span(text, 4, 10)])[0], 'NIP [PII].')
+        self.assertEqual(scrub_spans(text, [], [span(text, 4, 9), span(text, 9, 14, score=0.98)])[0], 'NIP [PII].')
+        self.assertEqual(scrub_spans(text, [], [span(text, 4, 10, score=0.9)])[0], text)  # under threshold: nothing
+
     def test_published_weights_label_contact_lines(self):
         from nergal import Nergal
         nergal = Nergal(HERE, device='cpu')