[NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline

Adds the term-filter stage, the last free stage before the paid extraction call.

Dependencies (approved): gliner>=0.2.13 (resolved 0.2.28), torch>=2.6 (resolved
2.11.0+cpu). Install verified clean — `import torchgen` succeeds, so not the
corrupt-wheel failure recorded in the calibration doc.

filters/span_filter.py runs the model on EVERY chunk, unconditionally. There is
deliberately no relevance gate in front of it: a pre-filter's false negatives
drop content before extraction sees it, and dropped content never reaches expert
review to be caught. "Zero candidate spans" is the relevance signal instead.

Two mechanical traps handled, both of which silently cost recall:
- the model truncates long inputs and WARNS rather than failing, so chunks are
fed as overlapping 130-word windows with character offsets remapped. Offsets
are walked over the original spacing, never re-joined — re-joining would shift
every offset in a chunk containing newlines and break every span check.
- the ~12-token span cap is flagged per mention, never silently truncated, so a
recall shortfall can be attributed to it.

Never-throw: a model failure degrades to "no candidates" rather than aborting a
corpus-scale ingest that has already paid for parsing. Model load is lazy and
cached so importing the package does not pay for the model stack.

CLI wires it in and gains --no-span-filter for a fast wiring check.

MEASURED against the migrated gold set:
E1 term-filter recall 0.8537 baseline 0.8537 delta +0.0000 PASS
true positives 35 / 41 gold terms
clusters 66 baseline 66
compression 2.47x baseline 2.56x
runtime 19.8s for 13 chunks
Result committed: eval/knowledge/results/v2_term_filter_2026-08-19_144318.json

The rebuild reproduces the prototype's recall exactly, which is what migrating
the yardstick before the pipeline was for.

Verification: ruff clean on touched paths; full suite 473 passed, 7 skipped
(was 470 + 3 new local tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

KNOWLEDGE_PIPELINE_TODO.md

@@ -146,8 +146,8 @@ envelope shape proposed in §3 — so S1 can settle either way without touching
 |---|---|---|---|---|
 | **X1** | Candidate-entry schema | ✅ | ✅ | `models.py`. All three invariants encoded: content fields Optional, `SubdomainEnum`, `Provenance.span` mandatory. Adds `source_wording` for R1. Pydantic v2 |
 | **X2** | Offline runner over a fixture | ✅ | ✅ | `scripts/run_knowledge_extraction.py`. Takes an artifact, **never a PDF**; each stage writes its own JSON. Runs green on the 9-page BUMA chunks and on Sofhia's draft |
-| **X3** | GLiNER span filter | ✅ | ⬜ | **Next.** Runs on all chunks unconditionally; "zero candidate spans" is the relevance signal. Deferred with the `gliner`+`torch` dependency (D1); the runner has a legend-only stand-in meanwhile, explicitly not a recall measurement |
-| **X4** | Indonesian performance | ✅ **0.854** | n/a | Answered by E1. Re-measure once on v2 to confirm no regression |
+| **X3** | GLiNER span filter | ✅ | ✅ | `filters/span_filter.py`. Runs on all chunks unconditionally; overlapping 130-word windows with character offsets remapped; span-cap hits flagged not truncated; never-throw. **Scored: E1 recall 0.8537 — exactly the frozen baseline**, 35/41 gold terms, 66 clusters (baseline 66), compression 2.47× (baseline 2.56×). Result: `eval/knowledge/results/v2_term_filter_2026-08-19_144318.json` |
+| **X4** | Indonesian performance | ✅ **0.854** | ✅ | Re-measured on v2: **0.8537, zero regression** against the prototype |
 | **X5** | Discourse-cue filter → rule-of-thumb | ✅ | ✅ | `filters/cue_filter.py`, cues in `config/cues.yaml`. 9 rule candidates on the BUMA standard. Also supplies `definitional_hits` for the ranker's +5.0 signal |
 | **X6** | Legend-block filter → formula vars | ✅ | ✅ | `filters/legend_filter.py`. Extracts the p.2 legend block **exactly** — MOHH · Qty · PA · UA · Pty, 5/5 — and rejects equation rows. LaTeX normalisation (§4) still pending for MinerU input |
 | **X7** | Section pass → summary units | ✅ | ⬜ | The quiet cost risk: few calls but ~¼ of all input tokens, because summarisation cannot be filtered — it needs whole documents |
@@ -168,7 +168,7 @@ envelope shape proposed in §3 — so S1 can settle either way without touching
 
 | # | Decision | Owner | Status | Description |
 |---|---|---|---|---|
-| **D1** | New dependencies | Rifqi | 🔄 | **`rapidfuzz>=3.14.5` added 2026-08-19** (approved). `gliner` + `torch` still **deferred** — not needed until X3, and the heavy tail is worth delaying. `PyYAML` was already present transitively. Prototype needed **torch ≥ 2.6** and hit a corrupt-wheel failure worth not re-debugging |
+| **D1** | New dependencies | Rifqi | ✅ | **All approved and added 2026-08-19:** `rapidfuzz>=3.14.5`, `gliner>=0.2.13` (resolved 0.2.28), `torch>=2.6` (resolved 2.11.0+cpu). Install verified clean — `import torchgen` OK, so not the corrupt-wheel failure the calibration doc warns about. `PyYAML` was already present transitively |
 | **D2** | New tables (DDL handoff) | Rifqi → Harry | ⬜ | Parsed artifacts, candidate entries, glossary versions + approval audit trail. Go owns the dedorch schema — Python never executes DDL. One consolidated Harry-ready handoff beats three |
 | **D3** | Model tier per branch | Rifqi | ✅ | **Decided 2026-08-19: stay on nano for all four branches.** The `.env` carries the quad `azureai__{api_key,endpoint__url,deployment__name,api__version}__54n`, mirroring the existing `__54m` scheme. Accepted knowingly: nano measured 0.75 against a 0.80 line, and `rule`/`summary` run on the tier whose failure mode is least detectable. Revisit if extraction quality blocks the demo. **`src/config/settings.py` does not expose the `__54n` quad yet** — four `Field(alias=…)` entries, first build step |
 | **D4** | Admin ingest surface | Rifqi | ⬜ | Whether it is HTTP at all, and if so its auth posture. A write surface triggered by an admin is a different risk class from the current unauthenticated read-only chat surface — raise it, don't inherit the posture by default. Only after the offline path works |

eval/knowledge/results/v2_term_filter_2026-08-19_144318.json

@@ -0,0 +1,53 @@
+{
+  "_note": "First v2 run. Term-filter (E1) only — the paid extraction stage is not built, so E3 is not scoreable yet.",
+  "run": {
+    "created_at": "2026-08-19_144318",
+    "implementation": "src/knowledge_extraction (v2)",
+    "doc_id": "STD_2026_006_MNO"
+  },
+  "config": {
+    "model": "urchade/gliner_multi-v2.1",
+    "labels_variant": "broad",
+    "threshold": 0.25,
+    "window_words": 130,
+    "window_overlap": 30
+  },
+  "gold": {
+    "path": "eval/knowledge/knowledge_gold.yaml",
+    "status": "provisional — not expert-reviewed",
+    "n_terms": 41
+  },
+  "funnel": {
+    "chunks": 13,
+    "mentions_raw": 188,
+    "mentions_after_noise": 163,
+    "clusters": 66,
+    "compression_ratio": 2.47
+  },
+  "E1": {
+    "metric": "term_filter_recall",
+    "value": 0.8537,
+    "kill_line": 0.7,
+    "verdict": "PASS",
+    "true_positives": 35,
+    "precision": 0.4118,
+    "precision_note": "reported, NOT the E1 criterion — the filter is deliberately over-inclusive",
+    "misses": [
+      "Overburden removal",
+      "Grouping (Composite)",
+      "Weight average",
+      "Plan",
+      "Fleet management",
+      "Mineplan"
+    ]
+  },
+  "baseline_comparison": {
+    "baseline_file": "results/baseline_prototype_2026-08-13_145132.json",
+    "E1_baseline": 0.8537,
+    "E1_v2": 0.8537,
+    "delta": 0.0,
+    "clusters_baseline": 66,
+    "clusters_v2": 66,
+    "verdict": "MATCH — v2 reproduces the prototype recall exactly"
+  }
+}

pyproject.toml

@@ -91,6 +91,8 @@ dependencies = [
     "pypdf2>=3.0.1",
     "pyarrow>=24.0.0",
     "rapidfuzz>=3.14.5",
+    "gliner>=0.2.13",
+    "torch>=2.6",
 ]
 
 [project.optional-dependencies]

src/knowledge_extraction/cli.py

@@ -33,7 +33,7 @@ from pathlib import Path
 
 from .adapter import parsed_doc_from_artifact
 from .cluster import cluster_mentions
-from .filters import abbrev_pairs, rule_candidates
+from .filters import abbrev_pairs, extract_mentions, rule_candidates
 from .models import Mention
 from .rank import rank_evidence
 from .settings import EVIDENCE_K
@@ -43,6 +43,11 @@ def main(argv: list[str] | None = None) -> int:
     parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
     parser.add_argument("artifact", type=Path, help="parsed-document artifact JSON")
     parser.add_argument("--mentions", type=Path, help="span-NER mentions JSON")
+    parser.add_argument(
+        "--no-span-filter",
+        action="store_true",
+        help="skip the span model; use legend terms only (fast wiring check, NOT a recall run)",
+    )
     parser.add_argument("--out-dir", type=Path, default=Path("out/knowledge"))
     parser.add_argument("--doc-id", help="override the artifact's doc_id")
     args = parser.parse_args(argv)
@@ -66,8 +71,18 @@ def main(argv: list[str] | None = None) -> int:
     for pair in pairs[:8]:
         print(f"         {pair.abbrev} -> {pair.expansion}")
 
-    mentions = _load_mentions(args.mentions) if args.mentions else _from_pairs(doc, pairs)
-    print(f"[filter] {len(mentions)} mentions in")
+    if args.mentions:
+        mentions = _load_mentions(args.mentions)
+        source = "file"
+    elif args.no_span_filter:
+        mentions = _from_pairs(doc, pairs)
+        source = "legend stand-in (NOT a recall run)"
+    else:
+        mentions = extract_mentions(doc.chunks)
+        source = "span filter"
+    capped = sum(m.hit_span_cap for m in mentions)
+    cap_note = f", {capped} hit the span cap" if capped else ""
+    print(f"[filter] {len(mentions)} mentions from {source}{cap_note}")
 
     clustered = cluster_mentions(mentions, pairs, doc.doc_id)
     print(
@@ -94,6 +109,10 @@ def main(argv: list[str] | None = None) -> int:
             "rule_candidates": [r.model_dump(mode="json") for r in rules],
         },
     )
+    _dump(
+        args.out_dir / f"{doc.doc_id}.mentions.json",
+        {"doc_id": doc.doc_id, "mentions": [m.model_dump(mode="json") for m in mentions]},
+    )
     _dump(args.out_dir / f"{doc.doc_id}.clusters.json", clustered.model_dump(mode="json"))
     print(f"[write ] {args.out_dir}")
     return 0

src/knowledge_extraction/filters/__init__.py

@@ -1,9 +1,12 @@
 from .cue_filter import definitional_hits, rule_candidates
 from .legend_filter import abbrev_pairs, find_legend_lines
+from .span_filter import extract_mentions, load_model
 
 __all__ = [
     "abbrev_pairs",
     "definitional_hits",
+    "extract_mentions",
     "find_legend_lines",
+    "load_model",
     "rule_candidates",
 ]

src/knowledge_extraction/filters/span_filter.py

@@ -0,0 +1,169 @@
+"""Span NER over ALL chunks → Mention[]. Free stage: zero API calls, CPU only.
+
+**Runs unconditionally on every chunk.** There is deliberately no relevance gate
+in front of it: a binary pre-filter's false negatives drop content *before*
+extraction ever sees it, and dropped content never reaches expert review to be
+caught. The model is cheap enough (CPU encoder, ~400 MB) to just look at
+everything, which turns "zero candidate spans" into the relevance signal as a
+byproduct of looking rather than a judgment made before looking.
+
+Two mechanical traps, both of which silently cost recall if reintroduced:
+
+1. **The model truncates long inputs and warns rather than failing.** Feeding a
+   whole chunk drops its tail without erroring. Chunks are therefore fed as
+   overlapping word windows with offsets remapped back to chunk coordinates.
+2. **There is a ~12-token span cap.** Compound terms ("UA plan grouping
+   (composite) actual") exceed it. Candidates that hit the cap are *flagged*,
+   never silently truncated, so a recall shortfall can be attributed to it.
+
+The model is loaded lazily and cached: it is heavy, and only this stage needs
+it, so importing the package must not pay for it.
+"""
+
+from __future__ import annotations
+
+import time
+from functools import lru_cache
+
+from ...middlewares.logging import get_logger
+from ..models import Chunk, Mention
+from ..settings import (
+    SPAN_TOKEN_CAP,
+    WINDOW_OVERLAP,
+    WINDOW_WORDS,
+    labels_for,
+)
+
+logger = get_logger("knowledge_span_filter")
+
+MODEL_NAME = "urchade/gliner_multi-v2.1"
+
+
+@lru_cache(maxsize=2)
+def load_model(model_name: str = MODEL_NAME):
+    """Lazy, cached. Imported inside the function so the package stays importable
+    (and the app stays fast to boot) without the model stack."""
+    from gliner import GLiNER
+
+    logger.info("loading span model", model=model_name)
+    return GLiNER.from_pretrained(model_name)
+
+
+def extract_mentions(
+    chunks: list[Chunk],
+    variant: str | None = None,
+    threshold: float | None = None,
+    model=None,
+) -> list[Mention]:
+    """Candidate term mentions across every chunk.
+
+    Never raises: the span filter is the widest net in the pipeline, and a model
+    failure must degrade to "no candidates from this chunk" rather than abort a
+    corpus-scale ingest that has already paid for parsing. Failures are logged
+    with `repr(e)` — an empty error string is how a real cause once went missing
+    for a day elsewhere in this repo.
+    """
+    labels, configured_threshold = labels_for(variant) if variant else labels_for()
+    score_floor = threshold if threshold is not None else configured_threshold
+
+    try:
+        model = model or load_model()
+    except Exception as exc:  # pragma: no cover - depends on the model stack
+        logger.error("span model unavailable", error=repr(exc))
+        return []
+
+    mentions: list[Mention] = []
+    capped = 0
+    started = time.time()
+
+    for chunk in chunks:
+        for window_text, offset in _windows(chunk.text):
+            try:
+                found = model.predict_entities(
+                    window_text, labels, threshold=score_floor
+                )
+            except Exception as exc:
+                logger.warning(
+                    "span prediction failed", chunk_id=chunk.chunk_id, error=repr(exc)
+                )
+                continue
+
+            for entity in found:
+                surface = entity.get("text", "")
+                if not surface.strip():
+                    continue
+                hit_cap = len(surface.split()) >= SPAN_TOKEN_CAP
+                capped += hit_cap
+                mentions.append(
+                    Mention(
+                        surface=surface,
+                        chunk_id=chunk.chunk_id,
+                        char_start=offset + int(entity.get("start", 0)),
+                        char_end=offset + int(entity.get("end", 0)),
+                        label=str(entity.get("label", "")),
+                        score=float(entity.get("score", 0.0)),
+                        hit_span_cap=hit_cap,
+                    )
+                )
+
+    deduped = _dedupe(mentions)
+    logger.info(
+        "span filter complete",
+        chunks=len(chunks),
+        mentions=len(deduped),
+        dropped_overlapping=len(mentions) - len(deduped),
+        hit_span_cap=capped,
+        seconds=round(time.time() - started, 1),
+    )
+    return deduped
+
+
+def _windows(text: str) -> list[tuple[str, int]]:
+    """Overlapping word windows plus each window's character offset.
+
+    Windowing is on WORDS but offsets must come back in CHARACTERS, so the
+    original spacing is walked rather than re-joined — re-joining would shift
+    every offset in a chunk containing newlines or double spaces, and every
+    span check downstream would then fail.
+    """
+    if not text.strip():
+        return []
+
+    positions: list[tuple[int, int]] = []
+    cursor = 0
+    for word in text.split():
+        start = text.index(word, cursor)
+        positions.append((start, start + len(word)))
+        cursor = start + len(word)
+
+    if len(positions) <= WINDOW_WORDS:
+        return [(text, 0)]
+
+    step = max(1, WINDOW_WORDS - WINDOW_OVERLAP)
+    windows: list[tuple[str, int]] = []
+    for begin in range(0, len(positions), step):
+        chunk_words = positions[begin : begin + WINDOW_WORDS]
+        if not chunk_words:
+            break
+        lo, hi = chunk_words[0][0], chunk_words[-1][1]
+        windows.append((text[lo:hi], lo))
+        if begin + WINDOW_WORDS >= len(positions):
+            break
+    return windows
+
+
+def _dedupe(mentions: list[Mention]) -> list[Mention]:
+    """Drop duplicates produced by window overlap, keeping the highest score.
+
+    Overlap is required for recall (a term straddling a window boundary would
+    otherwise be missed), and it necessarily re-finds terms in the overlap
+    region. Keyed on (chunk, span) so the same surface at a different position
+    stays a separate mention — mention COUNT drives the review queue ordering.
+    """
+    best: dict[tuple[str, int, int], Mention] = {}
+    for mention in mentions:
+        key = (mention.chunk_id, mention.char_start, mention.char_end)
+        current = best.get(key)
+        if current is None or mention.score > current.score:
+            best[key] = mention
+    return sorted(best.values(), key=lambda m: (m.chunk_id, m.char_start))

uv.lock

@@ -21,6 +21,7 @@ dependencies = [
     { name = "cachetools" },
     { name = "cryptography" },
     { name = "fastapi", extra = ["standard"] },
+    { name = "gliner" },
     { name = "httpx" },
     { name = "jsonpatch" },
     { name = "kaleido" },
@@ -73,6 +74,7 @@ dependencies = [
     { name = "structlog" },
     { name = "tenacity" },
     { name = "tiktoken" },
+    { name = "torch" },
     { name = "uvicorn", extra = ["standard"] },
 ]
 
@@ -106,6 +108,7 @@ requires-dist = [
     { name = "cachetools", specifier = "==5.5.0" },
     { name = "cryptography", specifier = "==44.0.0" },
     { name = "fastapi", extras = ["standard"], specifier = "==0.115.6" },
+    { name = "gliner", specifier = ">=0.2.13" },
     { name = "httpx", specifier = "==0.28.1" },
     { name = "httpx", marker = "extra == 'dev'", specifier = "==0.28.1" },
     { name = "jsonpatch", specifier = ">=1.33" },
@@ -165,6 +168,7 @@ requires-dist = [
     { name = "structlog", specifier = "==24.4.0" },
     { name = "tenacity", specifier = "==9.0.0" },
     { name = "tiktoken", specifier = "==0.8.0" },
+    { name = "torch", specifier = ">=2.6" },
     { name = "uvicorn", extras = ["standard"], specifier = "==0.32.1" },
 ]
 provides-extras = ["dev"]
@@ -939,6 +943,14 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/a4/a5/842ae8f0c08b61d6484b52f99a03510a3a72d23141942d216ebe81fefbce/filelock-3.25.2-py3-none-any.whl", hash = "sha256:ca8afb0da15f229774c9ad1b455ed96e85a81373065fb10446672f64444ddf70", size = 26759, upload-time = "2026-03-11T20:45:37.437Z" },
 ]
 
+[[package]]
+name = "flatbuffers"
+version = "25.12.19"
+source = { registry = "https://pypi.org/simple" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" },
+]
+
 [[package]]
 name = "fonttools"
 version = "4.62.1"
@@ -990,6 +1002,23 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/d5/1f/5f4a3cd9e4440e9d9bc78ad0a91a1c8d46b4d429d5239ebe6793c9fe5c41/fsspec-2026.3.0-py3-none-any.whl", hash = "sha256:d2ceafaad1b3457968ed14efa28798162f1638dbb5d2a6868a2db002a5ee39a4", size = 202595, upload-time = "2026-03-27T19:11:13.595Z" },
 ]
 
+[[package]]
+name = "gliner"
+version = "0.2.28"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "huggingface-hub" },
+    { name = "onnxruntime" },
+    { name = "sentencepiece" },
+    { name = "torch" },
+    { name = "tqdm" },
+    { name = "transformers" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f4/b7/0f3e24ff0b8c1c95121532a44e09b5bb87bd771c6bed0d387d51480645c5/gliner-0.2.28.tar.gz", hash = "sha256:b1637afb5cf4235fc871f1e21498831775b7bd19cefbda6dc5fe08ee88cb07a0", size = 263394, upload-time = "2026-07-24T14:03:49.833Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/3e/24/cf6a9eb70bd8eb78a74f90104180c38ae0f93bc213fdd3224ea93c2fe74a/gliner-0.2.28-py3-none-any.whl", hash = "sha256:734e333ebf8a48c135aac5c05599f51051037513030127cbaf676ae71ca501c5", size = 245603, upload-time = "2026-07-24T14:03:48.337Z" },
+]
+
 [[package]]
 name = "google-api-core"
 version = "2.30.3"
@@ -2037,6 +2066,24 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/a8/64/3708a90d1ebe202ffdeb7185f878a3c84d15c2b2c31858da2ce0583e2def/nvidia_nvtx-13.0.85-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb7780edb6b14107373c835bf8b72e7a178bac7367e23da7acb108f973f157a6", size = 148878, upload-time = "2025-09-04T08:28:53.627Z" },
 ]
 
+[[package]]
+name = "onnxruntime"
+version = "1.29.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "flatbuffers" },
+    { name = "numpy" },
+    { name = "packaging" },
+    { name = "protobuf" },
+]
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/d4/80/381c1e9efed9cc32d00aa7cab0547dc84116cec906c3ffe3613686d6963a/onnxruntime-1.29.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3a3814c041251d6a77fdf513fb282056538ee826d2f1178a0df3c549d3fff6ba", size = 21430049, upload-time = "2026-08-17T22:53:48.286Z" },
+    { url = "https://files.pythonhosted.org/packages/30/12/4be0e345d38fe707a701ca07e8f63c05b152a2e6285d1e43a7faf63fedd2/onnxruntime-1.29.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:d2fb19e848f7c33ed8d3182b52504aaa11c5e8da438bbb47296f85b133cbcf6b", size = 20816870, upload-time = "2026-08-17T22:53:51.169Z" },
+    { url = "https://files.pythonhosted.org/packages/96/eb/e6968f5e41aac3125f2ff5708855f09cb0b70d85ed3115b625b0b58305ba/onnxruntime-1.29.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:2b80d8c7ec2cc7438e4da3760b88c24568cba72c9ace96d668800a6c79419acb", size = 23136745, upload-time = "2026-08-17T22:53:53.92Z" },
+    { url = "https://files.pythonhosted.org/packages/b4/80/5b28f1f1111210fc4a336ddbc6950f468ebf9a6a265420568f4f43fa33ce/onnxruntime-1.29.0-cp312-cp312-win_amd64.whl", hash = "sha256:4acf2b4948b7ede87221ca6332344b8facdc8059d6ac751a7d367d04532b02dd", size = 14001407, upload-time = "2026-08-17T22:53:56.486Z" },
+    { url = "https://files.pythonhosted.org/packages/6f/d6/6883f89ea4b044e6e8447ebfaf9bcecdf457b7d80a683635e130b25498e0/onnxruntime-1.29.0-cp312-cp312-win_arm64.whl", hash = "sha256:dc61a79cb39afd66ab3f01fd2c23591a7f01de89c1668e1fb6315067fc279164", size = 13746981, upload-time = "2026-08-17T22:53:58.977Z" },
+]
+
 [[package]]
 name = "openai"
 version = "1.58.1"
@@ -3031,6 +3078,21 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/8b/c8/990e22a465e4771338da434d799578865d6d7ef1fdb50bd844b7ecdcfa19/sentence_transformers-3.3.1-py3-none-any.whl", hash = "sha256:abffcc79dab37b7d18d21a26d5914223dd42239cfe18cb5e111c66c54b658ae7", size = 268797, upload-time = "2024-11-18T14:37:38.579Z" },
 ]
 
+[[package]]
+name = "sentencepiece"
+version = "0.2.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/cc/33/ea3cb3839607eb175da835244a798f797f478c5ddf0e8ecdf57ea85a4c70/sentencepiece-0.2.2.tar.gz", hash = "sha256:3d2b5e824b5622038dc7b490897efe05ebbbb9e7350fc142f3ecc8789ef9bdf6", size = 8218435, upload-time = "2026-07-12T08:39:34.701Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/b8/13/7a562289c8d5b49ebdf3f9c1e8ab67cf14a8743b1d90c8f406bfdec36b72/sentencepiece-0.2.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:1edb10e520e4bddf74d85b0f5ae74cc2d60c2b448885080bfb618bc2b3a49f6b", size = 2188384, upload-time = "2026-07-12T08:38:28.486Z" },
+    { url = "https://files.pythonhosted.org/packages/85/d1/912f14fd5eae168aba726ffb6a9a2dc1c71fe7676c53da6f5c442b886d4a/sentencepiece-0.2.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f7c06c751c19d923435a54bff4f7e66e728fad160e8da28254f133abc9725820", size = 1441553, upload-time = "2026-07-12T08:38:30.552Z" },
+    { url = "https://files.pythonhosted.org/packages/bd/44/caa9cab5f261a019e2808bc5046152775dc57352ba9cbae7525e9e7a1ed4/sentencepiece-0.2.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:38111ed1f79268f399c505028023d5eaaf0ab4e5eafceb709468b0d3323e7838", size = 1347176, upload-time = "2026-07-12T08:38:32.211Z" },
+    { url = "https://files.pythonhosted.org/packages/19/90/cd798935668cff71d309d8ff10385844ecf216b1fe454f1993ed8bf2cb91/sentencepiece-0.2.2-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cbce24284f51f71d10a42b7b9c964dcb9048b28f1c8e5db40bcbcb6f428cba6a", size = 1325200, upload-time = "2026-07-12T08:38:33.689Z" },
+    { url = "https://files.pythonhosted.org/packages/b6/2d/37e3da037318a70066ded0d51bc2a7f35491ae6338dd993d5eb1503fc3b5/sentencepiece-0.2.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c8a168b040bc61681293f79a949b5d911c8e25086f4260285b8d97ab5f1195da", size = 1397736, upload-time = "2026-07-12T08:38:35.771Z" },
+    { url = "https://files.pythonhosted.org/packages/8d/11/753fca2e6b109be3ab7867abf357dfe48677fe726ae5a5363d0b54ca9450/sentencepiece-0.2.2-cp312-cp312-win_amd64.whl", hash = "sha256:7c6e7bf684dc12145bfa685d3060beaea55139134ba848289bee514ed42e7383", size = 1248030, upload-time = "2026-07-12T08:38:37.604Z" },
+    { url = "https://files.pythonhosted.org/packages/e2/0a/70efbe861ca182d7d4b6e1a20f58e043400848fa9f2915229f082e221648/sentencepiece-0.2.2-cp312-cp312-win_arm64.whl", hash = "sha256:76ff5814db72e7462dece042d7593cdf102b8ec82c2b1cc201a2add34ee3050d", size = 1187325, upload-time = "2026-07-12T08:38:39.348Z" },
+]
+
 [[package]]
 name = "sentry-sdk"
 version = "2.57.0"