2.0.9: comment and test edits; same spans as 2.0.8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -8,6 +8,10 @@ Semver for this island:
 
 Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
 
+## 2.0.9
+
+Same NERGAL weights, threshold, rules, union, names layer and API: comment and test edits only. Rules SHA `7db3f739…` → `08bcaf19…` from comments alone (the rules module parses to the same syntax tree), so every span equals 2.0.8's.
+
 ## 2.0.8
 
 Same NERGAL weights, threshold, union, names layer and API. Rules SHA `4d054723…` → `7db3f739…`, two forms seen in notarial deeds:

README.md

@@ -16,7 +16,7 @@ tags:
 - hybrid
 ---
 
-# NERGAL 2.0.8
+# NERGAL 2.0.9
 
 **Named Entity Recognition with Grounded Additive Labels**
 
@@ -66,9 +66,9 @@ NERGAL is not designed to remove:
 
 These are intended exclusions; false positives can still mask some of this content.
 
-### Known gaps in 2.0.8
+### Known gaps in 2.0.9
 
-Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.8 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). A number neither the rules nor the model flag is left as text: on the label-pool benchmark, 46 of 344 gold phones and 27 of 160 gold identifiers holding a digit have no character masked. Some 10–13-digit runs after `nr` in procurement rulings are masked as identifiers with no gold to say whether they are identifiers or notice and case references. Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. An ID card whose series stands one or more words after its label, other than `serii`, `seria`, `numer` or `nr`, is left to the model; a KW number read out in words with no label nearby is masked only with a valid check digit. Names-on gaps are under [Names](#names).
+Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.9 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). A number neither the rules nor the model flag is left as text: on the label-pool benchmark, 46 of 344 gold phones and 27 of 160 gold identifiers holding a digit have no character masked. Some 10–13-digit runs after `nr` in procurement rulings are masked as identifiers with no gold to say whether they are identifiers or notice and case references. Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. An ID card whose series stands one or more words after its label, other than `serii`, `seria`, `numer` or `nr`, is left to the model; a KW number read out in words with no label nearby is masked only with a valid check digit. Names-on gaps are under [Names](#names).
 
 ## Versions
 
@@ -86,7 +86,8 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
 | 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
 | 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
 | 2.0.7, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves role nouns cut short or glued to a word out |
-| **2.0.8, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | ID-card series and number named apart; KW number read out in words |
+| 2.0.8, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | ID-card series and number named apart; KW number read out in words |
+| **2.0.9, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | Comment and test edits; same spans as 2.0.8 |
 
 1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
 
@@ -103,7 +104,7 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
 | Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
 | Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
 
-Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 through 2.0.8 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
+Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 through 2.0.9 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
 
 **Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
 
@@ -115,7 +116,7 @@ Names off hides no name. A component is the independent unit, covered when every
 | Skarżący | 50 | 3 | 3 | 0 | **0** |
 | Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
 
-The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The bench holds no damaged role noun, so 2.0.7 equals 2.0.5 there too; its words were checked by hand on UZP rulings (`CHANGELOG.md`). 2.0.8 changes only rules; its names layer is 2.0.7's. The texts hold real names and are not released.
+The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The bench holds no damaged role noun, so 2.0.7 equals 2.0.5 there too; its words were checked by hand on UZP rulings (`CHANGELOG.md`). 2.0.8 changes only rules; its names layer is 2.0.7's. 2.0.9 changes no span. The texts hold real names and are not released.
 
 **Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
 
@@ -129,7 +130,7 @@ The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodn
 
 ## Compared with other systems
 
-Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.8 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
+Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.9 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
 
 | System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
 |---|---|---:|---:|---:|---:|---:|
@@ -139,9 +140,9 @@ Same split and gold. Every model runs through the NERGAL recipe with names off:
 | Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
 | Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
 | XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
-| **NERGAL 2.0.8, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
+| **NERGAL 2.0.9, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
 
-Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.8 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
+Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.9 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
 
 XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
 

hybrid.json

@@ -1,12 +1,12 @@
 {
   "full_name": "Named Entity Recognition with Grounded Additive Labels",
   "hub_id": "SlayerLab/NERGAL",
-  "version": "2.0.8",
+  "version": "2.0.9",
   "mode": "rules_union",
   "epoch": 5,
   "seed": 202609160,
   "threshold": 0.95,
-  "rules_sha256": "7db3f7392cc16e22934d9d418be54074bf6c87fb3ef996bfe91ea232fb1d978d",
+  "rules_sha256": "08bcaf1909653a8632df41cc517798a39053a8d89ae34178cb4a1f99e3ce96eb",
   "eval": {
     "split": "841-dev",
     "gold_amendments": [

nergal.py

@@ -17,13 +17,13 @@ import scrub_pii
 from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
 
 HUB_ID = 'SlayerLab/NERGAL'
-VERSION = '2.0.8'
+VERSION = '2.0.9'
 GAPS = ['[PII_SPACE]', '[PII_BREAK]']
 GAP_IDS = [250002, 250003]
 BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
 LABELS = ['phone', 'pii']
 THRESHOLD = 0.95
-RULES_SHA = '7db3f7392cc16e22934d9d418be54074bf6c87fb3ef996bfe91ea232fb1d978d'
+RULES_SHA = '08bcaf1909653a8632df41cc517798a39053a8d89ae34178cb4a1f99e3ce96eb'
 DTYPES = ('float32', 'float16')
 MARKERS = (PHONE_TAG, PII_TAG, PERSON_TAG, LEGACY_PHONE_TAG)
 TAGS = {'person': PERSON_TAG, 'pii': PII_TAG, 'phone': PHONE_TAG}  # ascending rank; a non-bare rule span outranks every model and person span
@@ -34,7 +34,7 @@ _PLACEHOLDER = re.compile('|'.join(map(re.escape, MARKERS)))
 # Procedural roles the names model tags as persons (artifacts/nergal_2_0_4_role_nouns,
 # artifacts/nergal_2_0_5_role_nouns): parties, representatives and offices, the noun in any inflection, the bare
 # participle stem (`Odwołując` cut short, or the adverbial participle), and a number glued to it (`Odwołujący3`,
-# `wykonawcy_1`). Names policy keeps roles outside a person span (`prezes Jan Wróbel` → `Jan Wróbel`).
+# `wykonawcy_1`). Names policy keeps roles outside a person span (`prezes Jan Przykładowski` → `Jan Przykładowski`).
 _PARTICIPLE = '(?:y|ego|emu|ym|ych|ymi|a|ej|ą|e)?'
 _CA = '(?:a|y|ę|ą|o|ów|om|ami|ach)'  # wykonawca
 _NIK = '(?:k|ka|kowi|kiem|ku|cy|ków|kom|kami|kach|czk(?:a|i|ę|ą|o|om|ami|ach)|czce|czek)'  # uczestnik, -niczka

scrub_pii.py

@@ -870,9 +870,9 @@ def _replace_phones(text: str, *, mask=_tag) -> tuple[str, int]:
         # postal address. It needs its own phone label to override that ambiguity.
         postal = re.fullmatch(r'\d{5}', raw) and re.match(r'[ \t]+[^\W\d_]', text[end:])
         # So is a 5-6 digit number after a complete phone and a bare line break: a table
-        # cell, "Fax (32-2) 287 25 24\n28 648,00". After a comma a list goes on,
-        # "95-1-511098,\n514262"; after a fragment the next line is its wrapped rest,
-        # "tel. +48 609 \n953 709", or the next local number, "(30) 25 10 22 33 25,\n22 33 28".
+        # cell, "Fax (32-2) 123 45 67\n28 648,00". After a comma a list goes on,
+        # "95-1-234567,\n345678"; after a fragment the next line is its wrapped rest,
+        # "tel. +48 609 \n123 456", or the next local number, "(30) 12 34 56 78 90,\n56 78 91".
         cell = (last_phone_complete and 5 <= len(_digits(raw)) <= 6
                 and re.fullmatch(r'[ \t]*\n[ \t\n]*', text[last_phone_end:m.start()]))
         continues = last_phone_cued or (last_phone_end is not None and _national_head(raw))

test_nergal.py

@@ -8,7 +8,7 @@ import unittest
 from pathlib import Path
 
 HERE = Path(__file__).resolve().parent
-RULES_SHA = '7db3f7392cc16e22934d9d418be54074bf6c87fb3ef996bfe91ea232fb1d978d'
+RULES_SHA = '08bcaf1909653a8632df41cc517798a39053a8d89ae34178cb4a1f99e3ce96eb'
 # Party and office role nouns in every inflection the names layer leaves out, built here, not from ROLE_NOUNS; the
 # empty participle ending is the bare stem (2.0.5)
 PARTICIPLE_ENDINGS = ('', 'y', 'ego', 'emu', 'ym', 'ych', 'ymi', 'a', 'ej', 'ą', 'e')
@@ -43,7 +43,7 @@ class NergalTests(unittest.TestCase):
         from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
         card = json.loads((HERE / 'hybrid.json').read_text())
         self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
-        self.assertEqual(VERSION, '2.0.8')
+        self.assertEqual(VERSION, '2.0.9')
         self.assertEqual(card['version'], VERSION)
         self.assertEqual(card['eval']['union_fp'], 80)
         self.assertEqual(card['eval']['rules_fp'], 24)
@@ -482,13 +482,13 @@ class NergalTests(unittest.TestCase):
         texts = ['W toku rozprawy Przystępujący podtrzymał swoje stanowisko. Odwołujący wniósł o uwzględnienie odwołania, '
                  'a Zamawiający o jego oddalenie.',
                  'Izba oddaliła odwołanie. Odwołujący nie wykazał, że Przystępującemu należało odrzucić ofertę.',
-                 'Odwołujący Jan Wrzos wniósł odwołanie. Przystępujący Tomasz Kmieć zgłosił przystąpienie po stronie '
+                 'Odwołujący Jan Testowski wniósł odwołanie. Przystępujący Tomasz Przykładowski zgłosił przystąpienie po stronie '
                  'Zamawiającego.',
                  'Protestująca wniosła protest. Zamawiający oddalił protest Protestującej, a Protestująca wniosła odwołanie.',
                  'Skarżąca wniosła skargę na decyzję. Organ wniósł o oddalenie skargi Skarżącej.']
         found = [[t[s['start']:s['end']] for s in spans] for t, spans in zip(texts, names.spans_many(texts))]
         # 2.0.3: 'Przystępujący', 'Przystępującemu', 'Protestującej', 'Protestująca', 'Skarżącej'
-        self.assertEqual(found, [[], [], ['Jan Wrzos', 'Tomasz Kmieć'], [], []])
+        self.assertEqual(found, [[], [], ['Jan Testowski', 'Tomasz Przykładowski'], [], []])
 
     def test_names_find_a_person_past_the_first_window_and_are_idempotent(self):
         from nergal import apply_union
@@ -502,7 +502,7 @@ class NergalTests(unittest.TestCase):
     def test_names_batched_equal_one_at_a_time(self):
         names = self._names()
         texts = ['Wniosek złożyła Anna Nowakowska z Radomia.', '',
-                 'Zdanie bez nazwisk o pogodzie. ' * 200 + 'Podpisał Tomasz Wrzos.', 'Ala ma kota.']
+                 'Zdanie bez nazwisk o pogodzie. ' * 200 + 'Podpisał Tomasz Przykładowski.', 'Ala ma kota.']
         self.assertEqual(names.spans_many(texts, max_batch=2), [names.spans(t) for t in texts])
 
     def test_names_are_opt_in_and_verified(self):