/feat knowledge management

.gitignore

@@ -54,4 +54,8 @@ docs/specs/tabular_parquet_contract.md
 docs/specs/tabular_parquet.md
 
 # Personal / local working docs (not for the shared repo) — archived out of root
-docs/_archive/
+docs/_archive/
+# Knowledge pipeline output — data, not code (parse cache + run artifacts)
+data/knowledge_cache/
+data/knowledge_runs/
+data/knowledge_docs/

DEV_PLAN.md

@@ -273,6 +273,51 @@ by design** until #43 (Go-forwarded identity). Not a gap to re-raise — it is a
 CORS was left at `["*"]` on purpose (tightening it needs the FE origin as config, which we chose
 not to set for now).
 
+## 0.8. Knowledge pipeline — test integration (planning, 2026-08-19)
+
+**Nothing has landed in this repo.** This section exists so the in-flight work is tracked in the
+usual place; no code, dependency, table or endpoint has been added here yet. Do not read any of it
+into REPO_STATUS.
+
+Context and intent: [knowledge_pipeline_context.md](knowledge_pipeline_context.md). Task breakdown,
+owners and verified statuses: [KNOWLEDGE_PIPELINE_TODO.md](KNOWLEDGE_PIPELINE_TODO.md).
+
+**Shape.** A six-stage pipeline (parse → free CPU/regex filter → one LLM call per *term cluster* →
+deterministic span validation → diff vs. the active glossary → frequency-sorted expert review) that
+turns client documents into reviewable knowledge entries, ultimately served over MCP. It splits into
+two independently runnable halves — **parsing (Sofhia, MinerU)** and **knowledge extraction
+(Rifqi)** — meeting only at a **persisted, versioned parsed-document artifact**. Extraction never
+takes a file path; that constraint is the whole point of the split.
+
+**Where it stands (audited 2026-08-19).** A working **prototype** (`kex`, ~2,763 LOC, 21 passing
+tests, 8-command CLI, four experiments against real Azure calls) exists outside version control at
+`Documents/Dataeyond_catalog/knowledge-pipeline/`. It is **not the product and will not be ported** —
+it answered the design questions; extraction v2 gets rebuilt in this repo. What carries forward is
+the *evidence* (gold sets, experiment results) and the *tuned constants*, not the code.
+Measured on the BUMA standard (9 pages): GLiNER Indonesian recall **0.854** (kill line 0.70, PASS —
+this closes the context doc's headline open risk), clustering compression **2.56×** (kill 2.0, PASS),
+nano schema-fill precision **0.75** (kill 0.80, **FAIL**), span-validation rejection **1.00** with
+0.00 false rejections (kill 0.90, PASS). Cost **$0.0069** for the document.
+
+| # | Task | Owner | Status | Note |
+|---|---|---|---|---|
+| P1 | Freeze the `kex` prototype (read-only snapshot) | Rifqi | ⬜ | No `.git`. ~2,763 LOC + both gold sets + all four experiment results on one machine. It is an archive, **not** a codebase to develop — v2 is a rebuild here, not a port |
+| P2 | Harvest the prototype's tuned constants | Rifqi | ⬜ | Every number that took runs to find, with its reason, **before** v2 code: rapidfuzz 92, fuzzy off below 5 chars (`PA`/`UA` collide), K=3, the evidence weight table, the 1024-token prefix pad (below it, caching does not engage — worth the measured 54% hit rate), label + cue sets, the 4 prompts |
+| P3 | Migrate the evidence into `eval/knowledge/` | Rifqi | ⬜ | Gold sets (41 terms + 15 rules) + scorer + experiment runners, following the existing `eval/{intent,help,readiness,planner}/` convention. **Before any v2 pipeline code** — without a standing baseline, "improved" is unfalsifiable |
+| S1 | Agree the parsed-artifact schema | Sofhia + Rifqi | 🔄 | Sofhia's draft `contracts.py` (2026-08-19) adopted as the base. Open: document-level envelope (no home for `version`), the section-structure claim, `source_wording` |
+| S5 | Keep the seam off the file path | Sofhia + Rifqi | ⬜ | The prototype's boundary was `Parser.parse(path) -> list[Page]` — the shape the plan forbids. Settle **before** the MinerU parser is built against it |
+| X* | Build extraction v2 in this repo | Rifqi | ⬜ | 17 stages, `src/query/`-shaped. Design is validated by the prototype; **none of it exists here yet** — see the two-column table in the checklist |
+| D2 | Artifact + candidate-entry persistence | Rifqi → Harry | ⬜ | Parsed artifacts, candidate entries, glossary versions + approval audit trail. Go owns dedorch DDL → one consolidated Harry-ready handoff, never DDL from Python (§2.2) |
+| R1 | Fix the literal-source-wording violation | Rifqi | ⬜ | A locked decision is violated: the BUMA heading reads "Physical **of** Availability (PA)"; the extracted entry carries "Physical Availability". The discrepancy is meant to reach the expert |
+| D3 | Provision a mini deployment | Rifqi | ⛔ | Only nano is provisioned, so all four branches route there — including `rule` and `summary`, whose failure mode is least detectable. Blocks the nano-vs-mini A/B |
+| D6 | Expert review of the gold set | Rifqi → Mas Beta | ⛔ | The 41-term / 15-rule gold set is self-bootstrapped and provisional. Now the baseline v2 is measured against |
+| D5 | v2 module name | Rifqi | ⬜ | `src/knowledge/` is taken by the existing OCR→pgvector service. Pick a non-colliding name before the first commit |
+
+**Not decided, do not assume:** the v2 module name (D5); whether the admin ingest surface is HTTP
+at all (the offline script path is the honest first milestone); and GLiNER + LangExtract ensembling
+stays ⏸️ flagged as scope creep. **`kex` is not being ported** �� it is a prototype that answered the
+design questions; v2 is built here against this repo's patterns.
+
 ## 5. Critical path & sequencing
 
 - **Critical path:** ~~#22 (send Harry the `report_inputs` schema)~~ **✅ resolved 2026-07-22** — now **#32** (`message_traceability` + `message_charts` DDL to Harry) and **#31** (non-convergent migration set). HF deploy (#13) for the playground. (#4 ✅, #21 ✅; Harry's #3 no longer blocks us — Python is getattr-tolerant.)
@@ -297,3 +342,8 @@ not to set for now).
 - **W3 re-open timing** (deferred `analyze_*` activation) — Rifqi (deferred at the 2026-07-13 approval).
 - **INV-6 relaxation for S1b targeted repair** — team, only after S1a `repair_candidate` telemetry
   shows a meaningful hit-rate (SPINE_V2_PLAN §6).
+- **Knowledge pipeline — v2 module name** (§0.8 D5): `src/knowledge/` is already the OCR→pgvector
+  service, so extraction v2 needs its own module name before the first commit.
+- **Knowledge pipeline — admin ingest surface** (§0.8 D4): whether it is HTTP at all, and if so its
+  auth posture. A write surface triggered by an admin is a different risk class from the current
+  unauthenticated read-only chat surface (§0.7) — raise it, do not inherit the posture by default.

KNOWLEDGE_PIPELINE_CALIBRATION.md

@@ -0,0 +1,257 @@
+# Knowledge Pipeline — Calibration Reference
+
+**Harvested from the `kex` prototype, 2026-08-19.** Every number below cost a run, a bug, or a
+measurement to find, and **none of it is in the spec** — it lived only as constants and comments in
+a codebase that is not being ported.
+
+This file exists because clean rewrites drop accidental knowledge and then rediscover it as
+production defects. When building extraction v2, take the value **and** the reason. If you change
+one, change it deliberately and record why.
+
+Companions: [KNOWLEDGE_PIPELINE_TODO.md](KNOWLEDGE_PIPELINE_TODO.md) (plan) ·
+[eval/knowledge/](eval/knowledge/) (gold set, scorer, frozen baseline) ·
+[knowledge_pipeline_context.md](knowledge_pipeline_context.md) (intent).
+
+---
+
+## 1. Term filter — GLiNER
+
+| Constant | Value | Why this value |
+|---|---|---|
+| Model | `urchade/gliner_multi-v2.1` | CPU, ~400 MB. Cheap enough to run unconditionally on every chunk, which is what lets us reject a standalone relevance gate |
+| Score threshold | **0.25** | Not 0.35. The sweep measured recall **0.854 @ 0.25** vs **0.658 @ 0.35** on the `broad` label set. Precision falls (0.41 vs 0.50) and that is accepted: the filter is meant to be over-inclusive, clustering and evidence ranking absorb the noise, and **a term the filter never proposes can never be recovered downstream** |
+| Label variant | **`broad`** | Two earlier variants both missed the same class — mining activities and materials (coal mining, land clearing, waste removal, joint survey). `broad` adds `mining activity`, `material or commodity`, `data source or system`, `work category`, `analysis method` to cover it |
+| Window size | **130 words**, overlap **30** | Not 220. GLiNER truncates past ~384 of its own tokens and **warns rather than failing** — silently dropping the tail of every long chunk. Indonesian technical prose subword-tokenises at roughly **2.5×**, so a 220-word window still tripped the cap. Chunks are fed as overlapping word windows with offsets remapped |
+| Span token cap | **12** (GLiNER's own) | Compound terms ("UA plan grouping (composite) actual") exceed it. Candidates that hit the cap are **flagged, not silently truncated**, so a recall shortfall can be attributed to the cap |
+
+**Label sets** (`labels.yaml`) — three variants were tried and A/B'd without touching code. Keep
+that property: label phrasing is the main recall lever and GLiNER is very sensitive to it. The
+Indonesian-phrasing variant (`istilah produksi tambang`, …) did **not** win; `broad` did.
+
+---
+
+## 2. Clustering
+
+| Constant | Value | Why this value |
+|---|---|---|
+| Match order | exact (normalised) → abbreviation↔expansion → fuzzy | Cheapest and most certain first |
+| Fuzzy threshold | **92** (`rapidfuzz.token_set_ratio`) | Conservative on purpose |
+| **Fuzzy disabled below 5 characters** | `min(len(a), len(b)) < 5` → skip | **The one to not lose.** `PA` vs `UA` scores high on `token_set_ratio`. Below 5 chars only exact matching is allowed |
+| Canonical surface | shortest variant ≥ 2 chars | That is how a reader looks a term up — `PA`, not `Physical Availability (PA) untuk …` |
+
+**Over-merging is worse than under-merging, and the asymmetry is not close.** An under-merge costs
+one extra LLM call and one extra review row. A wrong merge silently destroys a distinct term and
+**the expert never sees it** — there is no downstream mechanism that recovers it.
+
+**Legend extraction must run before clustering.** Without the abbreviation index, `PA` and
+`Physical Availability` cluster as two unrelated terms.
+
+**Stop surfaces** — dropped as whole surface forms only (never as substrings, so no term is lost):
+`unit · type · class · equipment · equipment unit · parameter · activity · data · nilai · proses ·
+hasil · total`.
+
+**Normalisation** (clustering only, never for span checking): NFKC → casefold → `-`/`_` → space →
+strip `.’'` → non-word to space → collapse whitespace → strip ` ()/`.
+
+---
+
+## 3. Evidence ranking
+
+Six signals plus one penalty. The full ranked list is retained, not just the top K — escalation
+needs the tail.
+
+| Signal | Weight |
+|---|---|
+| Definitional cue within 100 chars of the mention | **+5.0** |
+| Term appears in the chunk heading | **+4.0** |
+| Mention sits inside a legend block | **+3.5** |
+| Chunk contains a formula | **+2.0** |
+| Mention is bold/italic | **+1.5** |
+| First occurrence of the cluster | **+1.0** |
+| Chunk is tabular | **−3.0** |
+
+| Constant | Value | Why |
+|---|---|---|
+| `evidence_k` | **3** | Chunks fed per extraction round |
+| Cue proximity | **100 chars** | Distance from mention to a definitional cue |
+
+**Heading matching is word-boundary, never substring.** `PA` is a substring of `Parameter`, `pada`
+and `composite` — substring matching handed the +4.0 heading bonus to almost every Indonesian
+heading and pushed real definition sections *below* formula tables.
+
+**Why this filter is legitimate when the relevance gate was rejected:** no term is dropped (only its
+*evidence* is narrowed), every mention stays in provenance so a bad top-3 is visible, and the
+escalation loop self-corrects. The rejected gate had none of those three properties. Preserve all
+three if ranking is ever revisited.
+
+---
+
+## 4. Chunking
+
+| Constant | Value | Why |
+|---|---|---|
+| Max chunk tokens | **1500** | With paragraph-boundary splitting; sentence boundaries as fallback for a single over-budget paragraph |
+| Max heading length | **90 chars** | Longer lines are sentences or formula rows, not headings |
+| Boilerplate frequency | **≥ 0.6 of pages** | A line repeating on 60%+ of pages is a running header/footer. Detected by frequency with **digits normalised to `#`** so page counters collapse — never by a hardcoded document-specific string |
+| Tabular detection | short lines > 0.6 **and** numeric lines > 0.4 | Feeds the −3.0 ranking penalty |
+
+**Three things the real document forced, all of which corrupt every downstream number if missed:**
+
+1. **Running headers/footers** — every page repeats the title block and "Confidential".
+2. **Breadcrumb headings** — pages re-print `2. PENJELASAN PARAMETER / 2.1. Production Parameter / …`
+   at the top. A naive splitter re-opens the section and shatters it. Rule used: *a heading that is
+   the current section or an ancestor of it, already seen, is a breadcrumb* — the section continues
+   and `page_end` extends.
+3. **Colon-continuation** — legend rows arrive as `MOHH` then `: Machine on Hand Hours` on the next
+   line. Rejoined before the legend filter sees them.
+
+Points 1 and 2 apply to MinerU output too: its `title` blocks repeat the same breadcrumbs on pages
+2–8 of the standard.
+
+---
+
+## 5. Extraction (the paid stage)
+
+| Constant | Value | Why |
+|---|---|---|
+| Temperature | **0.0** | And still not deterministic — see below |
+| Seed | 7 | |
+| **Cache minimum** | **1024 prompt tokens** | OpenAI-family prompt caching **does not engage at all** below this. The glossary prefix is padded past it *on purpose*. Measured hit rate **54%** — 125,184 of 155,313 prompt tokens |
+| Structured output | `json_schema`, fallback `json_object` + validate-retry | Which mode was used is recorded in the run manifest |
+| API version | `2024-12-01-preview` | `json_schema` needs ≥ `2024-08-01-preview` |
+
+**Prompts live in files, not code, and the fixed prefix must stay byte-identical across calls** —
+any drift and caching stops engaging, silently, at ~10× the input cost.
+
+**`temperature=0` is not determinism.** Consecutive runs of the identical pipeline scored 0.75 and
+0.625 on schema fill. Never report a single run as a measurement on a small base.
+
+---
+
+## 6. Validation
+
+| Constant | Value | Why |
+|---|---|---|
+| Span normalisation | **whitespace only** (+ NFKC) | Not case, not punctuation, not diacritics. Every additional normalisation is a hole a fabrication fits through |
+| Escalation rounds | **max 2** | Then keep the entry flagged `no_definition_found` and pass it to review anyway — a term we found but could not define is still useful; dropping it hides a known unknown |
+| Conflict overlap threshold | **0.4** token overlap | Token overlap, not embeddings: cheaper, needs no model, and **explainable to the reviewer who has to act on it** |
+| Duplicate overlap threshold | **0.8** | Above it, a differing definition is a duplicate; below, a conflict |
+
+**Guarded fields** (span-checked; failure sets the field to `None` and logs it):
+
+| Branch | Fields |
+|---|---|
+| glossary | `definition`, `full_name`, `formula_latex`, `interpretation` |
+| rule | `statement`, `condition`, `consequence` |
+| formula | `formula_latex` |
+| summary | *(none — generation cannot be span-checked at all)* |
+
+**If the provenance span itself is not verbatim, every guarded field on the entry is rejected** —
+the entry's only evidence link is broken, so nothing on it can be trusted.
+
+**Never repair a failed span.** A repaired span is an unfalsifiable claim, which is exactly what the
+control exists to prevent.
+
+**Conflict detection never picks a winner.** The expert decides.
+
+---
+
+## 7. Negative findings — do not re-derive these
+
+| Finding | Detail |
+|---|---|
+| **nano is not sufficient for schema fill** | 0.75 precision against a 0.80 kill line. Decision 2026-08-19: **stay on nano anyway** for now; the `__54n` env quad is provisioned. Revisit if quality blocks the demo |
+| **Escalation is unreachable on small documents** | 54 of 66 clusters had exactly **1** evidence chunk against K=3, so `rounds_available()` correctly returned 0 and the loop never fired. `escalated = 0` is not a bug. It stays unverified until a larger corpus runs |
+| **Conflict detection had nothing to find** | 0 conflicts on a single internally-consistent standard. Untested against real disagreement |
+| **85% abstention** | 56 of 66 entries returned no definition. Correct behaviour, but it means the review queue is mostly *"term found, no definition in document"*. Whether that is useful or noise is a review-experience question for Mas Beta |
+| **Diff only ever ran the empty-baseline path** | The prototype diffed against the file it then overwrote, so every entry came back `new`. v2 needs a real active-version pointer |
+| **Literal source wording was normalised away** | The standard's heading reads *"Physical **of** Availability (PA)"*; the entry carried `full_name: "Physical Availability"`. The gold set records the literal form correctly, so **gold was right and the pipeline was wrong**. Mechanism: the heading is a separate field from the chunk text, and the chunk text does not contain the phrase, so the literal wording never reached the model. Fix: feed the heading verbatim + a span-checked `source_wording` field |
+| **A `glob('*.pdf')` picked the wrong document once** | Scored the textbook against the standard's gold set → recall 0.05. Document selection must be explicit and error on ambiguity |
+| **torch wheels install corrupt** | `torchgen` missing, version reports `None`. Fix is delete `site-packages/torch*` and reinstall. Needs **torch ≥ 2.6**. Documented so nobody debugs it as a GLiNER problem |
+
+---
+
+## 8. What was measured, so v2 can be compared
+
+Frozen in `eval/knowledge/results/baseline_prototype_2026-08-13_145132.json`.
+
+Funnel on the 9-page standard: **9 pages → 13 chunks → 169 mentions** (195 before noise filtering)
+**→ 66 clusters → 83 LLM calls → 66 entries → 66 review-queue items.**
+
+E1 recall **0.854** · E2 compression **2.56×** · E3 schema-fill precision **0.75** (FAIL) ·
+E4 fabrication rejection **1.00** with **0.00** false rejections. Cost **$0.0069**, 242 s wall
+clock, 3 fields rejected by span check.
+
+**When quoting E2 externally:** the context doc's **6.2×** is a 1,000-page projection; **2.56×** is
+the 9-page measurement. Compression grows as terms repeat across a corpus. Both are true — use the
+right one for the audience.
+
+---
+
+## 9. Deliberate deviations from the prototype (v2)
+
+Changes made while rebuilding, with the reason. Each one alters behaviour, so
+each is listed rather than folded silently into the tables above.
+
+| # | Change | Why | Measured? |
+|---|---|---|---|
+| **V1** | **A heading that names the term counts as a mention at offset 0** for cue-proximity scoring | Indonesian standards are written *"2.1.3 Physical of Availability (PA)"* / *"Adalah ketersediaan fisik…"* — the heading names the term and the body opens with the definitional cue **without repeating it**. The prototype therefore awarded no cue bonus on the definition section itself. Measured on the BUMA standard: PA's own definition section (§2.1.3) scored 7.5 and ranked **third**, below a computation section (§2.2.4, 8.5) that merely mentions PA often. After the fix, all five terms rank their own definition section first — PA→2.1.3, UA→2.1.4, Qty→2.1.2, Pty→2.1.5, MOHH→2.1.1 | Structurally verified on the 9-page standard. **Not yet scored** — needs the paid stage to show up in E3 |
+| **V2** | **`first_occurrence` follows document order**, and ranking ties break on document order | The prototype awarded the +1.0 to whichever chunk a `set` iteration yielded first, making the bonus arbitrary and the ranking unreproducible across runs. "First occurrence" should mean the term's first appearance in the document | Determinism locked by test; verified stable across `PYTHONHASHSEED` 0/1/7/42/99 |
+| **V3** | **`source_wording` field added** to the glossary entry | The control for the literal-source-wording defect (§7). Not yet populated — needs the seam to carry the heading verbatim and the extraction prompt to fill it | No |
+
+**Both V1 and V2 change which chunks reach the paid stage**, so they can move E3
+when the extraction branch lands. Re-score against
+`eval/knowledge/results/baseline_prototype_2026-08-13_145132.json` at that point
+and record the delta rather than assuming the improvement carries.
+
+**V1 and the R1 literal-wording defect share a root cause:** the heading is a
+separate field from the chunk text, so anything that reasons over "the term and
+its definition together" has to be told to look at both.
+
+| **V4** | **`source_wording` is taken from the section heading, not from the model** | Asked to quote the source wording, the model returned "Physical Availability (PA)" — a genuine verbatim quote from the page-1 intro, but not the §2.1.3 heading "Physical **of** Availability (PA)". Both occur in the document; only the heading form reveals the disagreement. Taking it deterministically means it cannot be normalised away | Live-verified on the pilot: the entry now carries both forms and the queue flags the mismatch |
+| **V5** | **`evidence_text` includes chunk headings** | The heading is part of the source document and is often the only place a term is formally named. Excluding it rejected a correct verbatim quotation of the document's own section title (2 rejections on a 3-entry pilot, both spurious) | Live-verified: 0 rejections after the change |
+| **V6** | **`full_name` and `source_wording` are span-checked against the source themselves** | Both claim to be literal transcriptions. The provenance span passing does not prove the transcription is faithful, and a normalised "full name" is exactly the silent correction this pipeline exists to surface | Unit-tested |
+
+**Cache-floor finding (measured on the real prompts):** only the **glossary**
+prefix clears the 1024-token floor at **1,401 tokens**. `rule` (731), `formula`
+(459) and `summary` (288) are **not cacheable** and cache nothing today. Glossary
+is also the branch with by far the most calls (66 of 83 on the reference
+document), so the padding is on the branch that matters — but padding the other
+three is free money if their call counts ever grow.
+
+---
+
+## 10. Parser comparison — MinerU vs PyMuPDF (2026-08-19)
+
+First run of the extraction half on **MinerU-parsed input**, same document, same
+gold set, same pipeline code. Result:
+`eval/knowledge/results/v2_parser_comparison_2026-08-19_152842.json`.
+
+| Input | E1 recall | Terms found |
+|---|---|---|
+| PyMuPDF (what the frozen baseline used) | **0.8537** | 35 / 41 |
+| MinerU, raw markup | **0.7561** | 31 / 41 |
+| MinerU, formulas + tables rendered to prose | **0.8293** | 34 / 41 |
+
+**The span model reads prose, not markup.** MinerU's structure is an asset for
+the formula branch and a liability for the filter unless it is rendered first:
+
+- Terms wrapped in LaTeX are invisible. `\text {Total Hours} - \text {Breakdown}`
+  yields no mentions for either term, though both are gold terms and both are
+  found in the plain-text parse. Unwrapping `\text{}`/`\mathrm{}`/`\frac{}{}`
+  and collapsing MinerU's character-spaced output recovers them.
+- Raw `<table>` HTML in the chunk text is noise the model has to read past. Eight
+  of fourteen chunks carried it before rendering.
+
+**The residual ~2.4 points are chunking, not content.** MinerU marks non-section
+labels — `Keterangan:`, `Keterangan grafik:` — as `text_level` titles. Opening a
+new section at each one splits a legend away from the figure it explains, and
+`Other Activity` / `Uncontrollable` are then missed **despite being present as
+prose in a chunk the filter processed**. Only **numbered** headings should open a
+section, which is consistent with §9's finding that MinerU flattens every heading
+to `level: 2` — numbering is the only reliable section signal it gives us.
+
+**Caveat on attribution:** the MinerU artifacts here were built by a scratch
+converter, because section-aware chunking belongs to the parsing half. These are
+findings about what that chunker must handle, not a measurement of MinerU
+itself.

KNOWLEDGE_PIPELINE_TODO.md

@@ -0,0 +1,210 @@
+# Knowledge Pipeline — Task Checklist
+
+**Date:** 2026-08-19 · **Owner of this doc:** Rifqi · **Shared with:** Sofhia
+**Intent + rationale:** [knowledge_pipeline_context.md](knowledge_pipeline_context.md)
+**Tracked in:** [DEV_PLAN.md](DEV_PLAN.md) §0.8
+**Target repo:** `Agentic-Service-Data-Eyond-Catalog` (Python agentic service)
+
+**Status legend:** ⬜ not started · 🔄 in progress · ✅ done · ⛔ blocked · 🔎 built but unverified · ⏸️ deferred
+
+---
+
+## How to read this
+
+The pipeline splits into two independently runnable halves that meet at **one persisted, versioned
+parsed-document artifact**: **parsing** (Sofhia, MinerU) and **knowledge extraction** (Rifqi).
+Extraction never takes a file path — that constraint is the point of the split, because it keeps the
+parser swappable and lets extraction be re-run cheaply on an already-parsed corpus.
+
+**The extraction half has a working prototype (`kex`) that is not the product.** It lives outside
+this repo, is not under version control, and will **not** be ported. It answered the design
+questions; v2 gets rebuilt here against this repo's patterns. That distinction drives the whole
+structure below — §2 disposes of the prototype, §5 builds v2, and §5 deliberately separates *design
+validated* from *implemented here*, because the two are very different amounts of work.
+
+---
+
+## 1. What the prototype already proved
+
+`kex` — ~2,763 LOC, 21 passing tests, 8-command CLI, four experiments against real Azure calls.
+Measured on the BUMA standard `STD_2026_006_MNO` (9 pages, majority Bahasa Indonesia), run
+`20260813-145132`.
+
+**Funnel:** 9 pages → 13 chunks → 169 mentions (195 raw) → 66 clusters → 66 entries → 66 queue items.
+
+| Experiment | Question | Result | Kill line | Verdict |
+|---|---|---|---|---|
+| **E1** | GLiNER recall on Bahasa Indonesia technical prose | **0.854** | 0.70 | **PASS** |
+| **E2** | Does clustering cut expert review burden? | **2.56×** (169 → 66) | 2.0× | **PASS** |
+| **E3** | Is nano sufficient for schema fill? | **0.75** | 0.80 | **FAIL** |
+| **E4** | Does verbatim-span validation catch fabrication? | **1.00** (false-rejection 0.00) | 0.90 | **PASS** |
+
+Also measured: **$0.0069** for the document · 155,313 prompt tokens of which **125,184 cached**
+(54% hit rate) · 242 s wall clock · 3 fields rejected by span check · 56 of 66 entries returned no
+definition · 0 escalations · 0 definition conflicts.
+
+**Two results that settle open questions:**
+
+- **E1 closes the Indonesian risk.** The context doc names GLiNER's unverified Indonesian
+  performance as the headline unknown. It is now measured, on the exact document named as the test
+  case, comfortably above the kill line.
+- **E3 half-settles the model tier.** nano **fails** at 0.75 against a 0.80 line. Caveats: the
+  scoreable base is 8 entries, and nano is not deterministic at `temperature=0` (consecutive
+  identical runs gave 0.75 and 0.625). No mini deployment exists, so the intended nano-vs-mini A/B
+  has never run and all four branches currently route to nano.
+
+**Caveat when quoting E2.** The context doc's **6.2×** is a 1,000-page projection; **2.56×** is the
+9-page measurement. Compression grows as terms repeat across a corpus. Both are true — quote the
+right one to the right audience.
+
+---
+
+## 2. Prototype disposition — do this before building v2
+
+`kex` is three different assets with three different fates. Treating it as one thing is how you
+either port scaffolding you didn't want or discard measurements you can't recreate.
+
+| # | Task | Owner | Status | Description |
+|---|---|---|---|---|
+| **P1** | Freeze the prototype | Rifqi | ✅ | **Resolved 2026-08-19: kept as-is, in place, unversioned — Rifqi's call.** The machine-failure risk is accepted knowingly. It is materially reduced anyway now that P2 and P3 have lifted the two things that could not be recreated (the calibration knowledge and the gold set + baseline) into this repo |
+| **P2** | Harvest the tuned constants | Rifqi | ✅ | **Done 2026-08-19 → [KNOWLEDGE_PIPELINE_CALIBRATION.md](KNOWLEDGE_PIPELINE_CALIBRATION.md).** 8 sections, every value with its reason: GLiNER threshold 0.25 (recall 0.854 vs 0.658 @ 0.35) and the 130-word window (Indonesian subword-tokenises ~2.5×, so 220 still tripped GLiNER's 384 cap); rapidfuzz 92 with fuzzy disabled below 5 chars (`PA`/`UA`); the six-signal weight table + word-boundary heading matching; chunking's 1500-token cap, 0.6 boilerplate frequency and the three real-document traps; the 1024-token cache floor; span/escalation/conflict thresholds and the guarded-field table |
+| **P3** | Migrate the evidence into `eval/knowledge/` | Rifqi | ✅ | **Done 2026-08-19.** `eval/knowledge/` now carries `knowledge_gold.yaml` (41 terms + 15 rules, verbatim), `score.py` (ported; only the prototype's `sys.path` bootstrap removed), `README.md`, and the frozen `results/baseline_prototype_2026-08-13_145132.json`. **Migration verified faithful:** the ported scorer re-derives the baseline exactly from the prototype's artifacts — E1 recall 0.8537, E3 precision 0.75, scoreable base 8, abstentions 56, all MATCH. `ruff` clean; `import main` OK. No `run_eval.py` yet — it lands with the first v2 stage that produces scoreable output |
+| **P4** | Record the negative findings | Rifqi | ✅ | **Done 2026-08-19 — calibration doc §7.** Eight entries: nano insufficient for schema fill; escalation unreachable on small documents; conflict detection had nothing to find; 85% abstention; diff only ever ran the empty-baseline path; literal source wording normalised away (gold was right, pipeline wrong); a `glob('*.pdf')` once scored the wrong document (recall 0.05); torch wheels install corrupt and need ≥ 2.6 |
+
+---
+
+## 3. The seam — shared with Sofhia
+
+**Status update 2026-08-19 (Sofhia).** The contract has landed in
+`src/knowledge_parsing/contracts.py` and every open item raised against the draft is closed except
+the two that need a joint decision (S6a formatting, S6b page indexing). Rows below updated
+accordingly; the extraction-side rows (S7, S8) are left as Rifqi has them.
+
+**Original note — under active discussion.** Sofhia circulated a draft `contracts.py` on 2026-08-19,
+derived from real MinerU output rather than designed on paper, and it is adopted as the base. Three
+of its choices improve on the prototype: a `kind` discriminator (text/table/chart/equation),
+`source_items` index back-references into MinerU's `content_list.json`, and an explicit
+verbatim-text warning that independently identifies the same failure mode as the span-check control
+— including that it fails *silently to null* and looks like a bad LLM.
+
+| # | Task | Owner | Status | Description |
+|---|---|---|---|---|
+| **S1** | Agree the parsed-artifact schema | Sofhia + Rifqi | 🔄 | **Adopted as the base and landed** in `src/knowledge_parsing/contracts.py` (pydantic, per S6e). Everything raised against the draft is closed except the two joint decisions in S6 (a: formatting signal, b: page indexing). |
+| **S2** | Add a document-level envelope | Sofhia + Rifqi | ✅ | **Done 2026-08-19.** `ParsedDocument{doc_id, source_path, content_hash, version, n_pages, parser_name, parser_version, chunks}` as proposed, plus three fields worth having: **`parser_backend`** read back from MinerU's `_middle.json` so the artifact records what *actually* ran rather than what was configured; `parser_config` (fingerprint of the settings that affect output); and `raw_output_dir`, pointing at the untouched MinerU output kept in the parse cache. |
+| **S3** | Correct the section-structure claim | Rifqi → Sofhia | ✅ | **Corrected 2026-08-19.** Checked against MinerU's source: `pipeline` and `vlm` run *identical* `text_level` logic and both emit `bbox`, so heading availability is document-dependent, not backend-dependent — the original claim came from sampling the handbook, which has no numbered headings. The normalizer now **reads** the hierarchy into a `heading_path` breadcrumb instead of deriving it, with numbering as fallback. Fields stay Optional. |
+| **S4** | Add `source_wording` while the contract is open | Rifqi → Sofhia | ✅ | **Done 2026-08-19.** `source_wording` added to `TermRecord`, and `heading` is carried verbatim — so "Physical **of** Availability" survives instead of being silently normalised (R1). |
+| **S5** | Keep the seam off the file path | Sofhia + Rifqi | 🔄 | **Satisfied on the parsing side.** `ParsedDocument` never carries a file handle; `source_path` is provenance only, and extraction receives the artifact. Left open for Rifqi to confirm from his side — the prototype's `Parser.parse(path)` boundary is the shape this must not reproduce. |
+| **S6** | Minor contract items (deferred to review) | Rifqi → Sofhia | 🔄 | **Four of six closed 2026-08-19; two need a joint call.** **(b) page indexing — OPEN:** the artifact uses `page_idx`/`page_idxs`, **0-based**, no conversion anywhere, so it always matches the raw MinerU output kept beside it; converting to 1-based is the UI's job, done once at display. Say if you want a 1-based `page_no` added instead, since this number reaches Mas Beta. **(a) formatting signal — OPEN, but the premise was wrong:** `text_format` is emitted by *both* backends and is not a formatting signal at all — it is the constant `'latex'` on equation blocks. Checked further: MinerU carries **no emphasis information anywhere** (zero `**` in its markdown, no style/font/span keys in `middle.json`). So the signal has no source in MinerU; it would need a separate pass reading the PDF directly (PyMuPDF exposes per-span bold flags). Recommend dropping it from the ranker for now — one of six signals, and adding a PDF pass buys nothing measured yet. Your ranker, your call. **(c) chunk size — done:** ~1500-token cap, matching calibration §4; splits at item boundaries so text stays verbatim. **(d) `chapter` lossy — done:** now `chapters: list[str]`, one per page the chunk spans. **(e) pydantic — done.** **(f) `TermRecord`** — left to you; the declaration in `contracts.py` is a placeholder, not a claim. |
+| **S7** | Commit fixture artifacts | Rifqi | ⬜ | Both source PDFs exist (BUMA standard + Open Pit textbook). Promote a parsed artifact of each into the target repo as committed fixtures, so the extraction half is testable **with MinerU not installed at all** — the seam doing its job |
+| **S8** | Write the seam rule into the repo docs | Rifqi | ⬜ | Extraction never takes a file path; neither half imports the other's internals; parsing stays swappable behind the artifact. The prototype drifted from this, so the rule needs to outlive our memory of it |
+
+---
+
+## 4. Parsing pipeline — Sofhia
+
+MinerU replaces the current text+table-only extraction: it adds formula, image, chart and diagram
+detection with type labels. Presented to the execs 13 Aug as the prerequisite for Scenario 2.
+
+| # | Task | Owner | Status | Description |
+|---|---|---|---|---|
+| **B1** | MinerU cost justification (written) | Sofhia | ⬜ | Owed for the budget proposal, from the 13 Aug exec review |
+| **B2** | MinerU install + run on the sample docs | Sofhia | ✅ | Complete 9-page parse of the BUMA standard delivered 2026-08-19: **118 items** (90 text, 14 table, 11 equation, 3 chart), pages 0–8. Includes the rule-of-thumb grounding content (Other Activity, Controllable/Uncontrollable, Standby), so the fixture exercises all four extraction branches. Supersedes an earlier 5-page run |
+| **B3** | Evaluate MinerU quality vs. the current path | Sofhia | ⬜ | Compare on tables, the p.2 legend block, formulas and figures — against the PyMuPDF baseline the prototype used, which has real numbers rather than a guess |
+| **B4** | Indonesian check on **parsing** | Sofhia | 🔄 | Distinct from E1, which measured GLiNER, not the parser. Partly answered 2026-08-19: MinerU has **no `id` language code** — Indonesian documents use `ch`, whose model covers Latin script (`config.periksa()` now rejects a wrong code before models load). `lang` only applies to the `pipeline` backend and only when OCR actually runs, so it is moot under `vlm`. Text/layout **quality** on Bahasa Indonesia still to be judged against B3 |
+| **B5** | Section-aware chunking | Sofhia | 🔄 | **Built 2026-08-19** in `normalize.py`. Splits on MinerU's `text_level` hierarchy into a `heading_path` breadcrumb, with numbering as fallback. Three real-document traps handled, two of them from §4 of the calibration doc: repeated breadcrumbs no longer reopen a section, **only numbered headings** may open one (so `Keterangan:` keeps a legend with its figure), and a ~1500-token cap guards documents with no detected headings. **Acceptance test still open** — needs the Open Pit textbook (S7) to confirm the NSR definition survives pp. 92–94 intact |
+| **B6** | Emit the agreed artifact | Sofhia | ✅ | **Done 2026-08-19** (`5840d201`). `src/knowledge_parsing/` writes `ParsedDocument`, pydantic, not MinerU's native shape. `parse.py` keeps a content-addressed cache of the untouched MinerU output beside it, and `parser_backend` is read back from `_middle.json` so the artifact records what actually ran |
+| **B7** | Fallback path behind the same artifact | Sofhia | ⬜ | Tesseract / Azure Document Intelligence emits the *same* artifact, so extraction never knows which parser ran |
+| **B8** | Keep the parsing subsystem additive + gated | Sofhia | ✅ | **Done 2026-08-19.** `src/knowledge/` is untouched; `src/knowledge_parsing/` sits beside it. MinerU is an **optional extra** (`knowledge-parsing`), not a main dependency, and is imported inside `parse.py` at call time — verified that importing the package leaves `mineru` and `torch` out of `sys.modules`, so the deployed Space neither builds nor ships them |
+
+**Findings from the real MinerU output that affect B5/B6:**
+
+- **Heading depth is flattened** — all 29 `title` blocks carry `"level": 2`, including `1. TUJUAN
+  PARAMETER` and `2.1.3. Physical of Availability (PA)`. Hierarchy must come from the numbering
+  text, never from `level`. The heading *text* is supplied (see S3).
+- **Equation LaTeX is character-spaced** — `P r o d u c t i o n = M O H H \times Q t y \times P A
+  \times U A \times P t y`. Naive matching will never find `MOHH`. Needs normalising before the
+  formula/legend branch, and before span validation can locate anything inside a formula.
+- **Breadcrumb headings repeat per page** — `2. PENJELASAN PARAMETER` / `2.1. Production Parameter`
+  re-appear as `title` blocks on pages 2–8. They must not be treated as new sections.
+- **Charts carry captions only** — all 3 chart items have `content: ""`, with the caption in
+  `chart_caption`. A chart chunk contributes a caption plus an image reference and nothing else.
+  Chart detection buys Scenario 2, not glossary coverage.
+
+---
+
+## 5. Extraction pipeline v2 — Rifqi, built in this repo
+
+**Two separate columns, deliberately.** *Design* = validated by the prototype, with a number behind
+it. *Build* = exists in this repo. The gap between them is the remaining engineering effort, and
+collapsing the two is how a rebuild gets under-estimated.
+
+**Module: `src/knowledge_extraction/`** (decided 2026-08-19 — `src/knowledge/` is the existing
+OCR→pgvector service and stays untouched). Offline runner: `scripts/run_knowledge_extraction.py`.
+Structure follows `src/query/`: typed models → deterministic stages → thin entry point.
+
+**The seam is absorbed by one file.** `adapter.py` is the only module that knows the artifact's
+shape; every stage depends on the internal `models.Chunk` subset instead. Verified against **both**
+live shapes — Sofhia's bare-list draft (`contoh-chunk.json`, `kind` discriminator) and the
+envelope shape proposed in §3 — so S1 can settle either way without touching a stage.
+
+| # | Stage | Design | Build | Note |
+|---|---|---|---|---|
+| **X1** | Candidate-entry schema | ✅ | ✅ | `models.py`. All three invariants encoded: content fields Optional, `SubdomainEnum`, `Provenance.span` mandatory. Adds `source_wording` for R1. Pydantic v2 |
+| **X2** | Offline runner over a fixture | ✅ | ✅ | `scripts/run_knowledge_extraction.py`. Takes an artifact, **never a PDF**; each stage writes its own JSON. Runs green on the 9-page BUMA chunks and on Sofhia's draft |
+| **X3** | GLiNER span filter | ✅ | ✅ | `filters/span_filter.py`. Runs on all chunks unconditionally; overlapping 130-word windows with character offsets remapped; span-cap hits flagged not truncated; never-throw. **Scored: E1 recall 0.8537 — exactly the frozen baseline**, 35/41 gold terms, 66 clusters (baseline 66), compression 2.47× (baseline 2.56×). Result: `eval/knowledge/results/v2_term_filter_2026-08-19_144318.json` |
+| **X4** | Indonesian performance | ✅ **0.854** | ✅ | Re-measured on v2: **0.8537, zero regression** against the prototype |
+| **X5** | Discourse-cue filter → rule-of-thumb | ✅ | ✅ | `filters/cue_filter.py`, cues in `config/cues.yaml`. 9 rule candidates on the BUMA standard. Also supplies `definitional_hits` for the ranker's +5.0 signal |
+| **X6** | Legend-block filter → formula vars | ✅ | ✅ | `filters/legend_filter.py`. Extracts the p.2 legend block **exactly** — MOHH · Qty · PA · UA · Pty, 5/5 — and rejects equation rows. LaTeX normalisation (§4) still pending for MinerU input |
+| **X7** | Section pass → summary units | ✅ | ⬜ | The quiet cost risk: few calls but ~¼ of all input tokens, because summarisation cannot be filtered — it needs whole documents |
+| **X8** | Normalise + cluster mentions | ✅ **2.56×** | ✅ | `cluster/`. Constants carried from P2 with their reasons. Locked by tests: PA/UA never merge; abbreviation↔expansion merges only with legend pairs; noise surfaces dropped as whole forms only |
+| **X9** | Evidence ranking → top-K | ✅ | ✅ | `rank/evidence.py`, six signals + tabular penalty. **Verified on the real document: every term's top-ranked chunk is its own definition section** — PA→2.1.3, UA→2.1.4, Qty→2.1.2, Pty→2.1.5. Full ranked list retained; word-boundary heading matching locked by test |
+| **X10** | LLM extraction call | ✅ | ✅ | `extract/` — client (nano `__54n`, probed structured output, API-sourced cached tokens), 4 branches, prompts as files. **Live pilot: 3 calls, 10.1s, 7,168 of 8,029 prompt tokens cached (89%)**, correct ID definitions + tags, 0 span rejections. `--dry-run` and `--limit` for cost control; `--mock` runs the whole pipeline with no spend |
+| **X11** | Verbatim-span validation | ✅ **1.00** | ✅ | `validate/span_check.py`. Whitespace-only normalisation; unlocatable provenance rejects every guarded field; never repairs. **Extended:** `full_name`/`source_wording` are checked against the source themselves, so a silent normalisation is caught rather than stored |
+| **X12** | Null-definition escalation | 🔎 | ✅ | `validate/escalate.py` + the glossary loop. Still **unexercised on this document** — most clusters have too little evidence to escalate to, which is a property of a 9-page corpus, not a defect |
+| **X13** | Conflict detection | 🔎 | ✅ | `validate/conflict.py`, token overlap (explainable to the reviewer, unlike embeddings); never picks a winner. Unit-tested both ways; still **0 conflicts on real data** — one consistent standard gives it nothing to find. Works only because clustering puts all evidence in one call |
+| **X14** | Diff vs. active glossary version | 🔎 | ✅ | `diff/glossary_diff.py`. The baseline is passed in explicitly (`--active-glossary`) rather than read from wherever the last run wrote, so the duplicate and conflicting paths are reachable — all three verified by test |
+| **X15** | Frequency-sorted review queue | ✅ | ✅ | `queue/review_queue.py`. Each row carries page, section and the verbatim span so review is quote-vs-page. Gained a review reason for **wording discrepancies**, which is how the literal-wording decision reaches the expert |
+| **X16** | Bulk vs. incremental ingest | ⬜ | ⬜ | Bulk setup ingest (corpus-frequency statistics as a candidate booster) and incremental single-file add (no corpus context; diff against the active version). Neither exists |
+| **X17** | Persistence | ⬜ | 🔄 | Stage artifacts are written as JSON by the CLI, which is enough for the offline path. Tables still needed → D2 |
+
+---
+
+## 6. Decisions needed
+
+| # | Decision | Owner | Status | Description |
+|---|---|---|---|---|
+| **D1** | New dependencies | Rifqi | ✅ | **All approved and added 2026-08-19:** `rapidfuzz>=3.14.5`, `gliner>=0.2.13` (resolved 0.2.28), `torch>=2.6` (resolved 2.11.0+cpu). Install verified clean — `import torchgen` OK, so not the corrupt-wheel failure the calibration doc warns about. `PyYAML` was already present transitively |
+| **D2** | New tables (DDL handoff) | Rifqi → Harry | ⬜ | Parsed artifacts, candidate entries, glossary versions + approval audit trail. Go owns the dedorch schema — Python never executes DDL. One consolidated Harry-ready handoff beats three |
+| **D3** | Model tier per branch | Rifqi | ✅ | **Decided 2026-08-19: stay on nano for all four branches.** The `.env` carries the quad `azureai__{api_key,endpoint__url,deployment__name,api__version}__54n`, mirroring the existing `__54m` scheme. Accepted knowingly: nano measured 0.75 against a 0.80 line, and `rule`/`summary` run on the tier whose failure mode is least detectable. Revisit if extraction quality blocks the demo. **`src/config/settings.py` does not expose the `__54n` quad yet** — four `Field(alias=…)` entries, first build step |
+| **D4** | Admin ingest surface | Rifqi | ⬜ | Whether it is HTTP at all, and if so its auth posture. A write surface triggered by an admin is a different risk class from the current unauthenticated read-only chat surface — raise it, don't inherit the posture by default. Only after the offline path works |
+| **D5** | v2 module name | Rifqi | ✅ | **`src/knowledge_extraction/`** (2026-08-19). Offline runner in `scripts/`, per the repo's existing precedent |
+| **D6** | Expert review of the gold set | Rifqi → Mas Beta | ⏸️ | **Deferred 2026-08-19: provisional gold is good enough for now** so the rebuild is not blocked on expert time. Standing caveat: E1 and E3 both move when it is reviewed, and E3's scoreable base is 8 entries, so each adjudication shifts it ~12 points. Every E3 failure is listed verbatim in the frozen baseline for when he does review it |
+| **D7** | GLiNER + LangExtract ensembling | — | ⏸️ | Proposed, not decided; flagged as scope creep. Cheap in API terms but adds a reconciliation step between two candidate sets that may disagree. Not in scope for the test integration |
+
+---
+
+## 7. Defects and risks to carry into v2
+
+| # | Finding | Severity | Detail |
+|---|---|---|---|
+| **R1** | ~~Literal source wording is normalised away~~ | **FIXED** | **Resolved 2026-08-19 in v2, live-verified.** Three parts: `source_wording` on the entry, taken **deterministically from the document's own section heading** rather than left to the model (which quoted the normalised page-1 form instead); `evidence_text` now includes headings, so quoting a section title is not wrongly rejected; and the review queue gained a reason for the mismatch. On the live pilot PA carries `full_name="Physical Availability"` **and** `source_wording="Physical of Availability (PA)"`, and ranks as *"source wording differs from the expanded name — confirm which is correct"* — the discrepancy reaches the expert instead of being silently corrected |
+| **R2** | **Prototype is not under version control** | **High** | Addressed by P1 |
+| **R3** | **85% abstention** | Medium | 56 of 66 entries carry no definition. Correct behaviour — for a term the document never defines, null *is* the right answer — but it means the review queue is mostly "term found, no definition in document". Whether that is useful to Mas Beta or noise is a **review-experience question to put to him**, and review experience is explicitly where engineering time is worth spending |
+| **R4** | **Everything generalises from one 9-page document** | Medium | Single document, single language, single company. E2's compression, E3's scoreable base and the escalation path are all constrained by it |
+| **R5** | **Rule and summary run on the wrong tier** | Medium | Both route to nano because it is the only provisioned deployment. Same blocker as D3 |
+| **R6** | **Rewrites drop accidental knowledge** | Medium | Addressed by P2. The `PA`/`UA` fuzzy-collision guard and the 1024-token caching threshold are the known examples; there will be others |
+
+---
+
+## 8. Settled — do not reopen
+
+1. The LLM call unit is the **term cluster**. Justification is review-burden reduction, **not** tokens (~$0.45 saved is trivial).
+2. **Conflict detection depends on the clustering** — two contradictory definitions can only be spotted because all evidence for a term arrives in one call.
+3. **No standalone relevance gate.** A pre-filter's false negatives drop content before extraction sees it, and dropped content never reaches review to be caught.
+4. **Evidence ranking is allowed** even though it is also a filter — no term is dropped, all mentions stay in provenance, and the escalation loop self-corrects. Preserve those three properties if it is ever revisited.
+5. **`provenance.span` is mandatory and verbatim**, never repaired. The primary anti-hallucination control, and what makes expert review finishable: the reviewer checks a quote against a page, not a claim against their memory.
+6. **All content fields Optional.** Abstention is correct behaviour, not an error.
+7. **Frequency-sorted review queue** — validate the ~40 terms carrying the document first, let the long tail wait.
+8. **API cost is negligible, latency irrelevant** (measured $0.0069 for 9 pages). This runs a few times a year, admin-triggered, not continuous and not per-user. Optimise the review experience, never tokens. Prefer boring, inspectable stages.
+9. **The pipeline records the literal source wording** and surfaces discrepancies rather than normalising them (currently violated — R1).
+10. **This is a test integration, not a productionisation.** Both halves are new, gated subsystems alongside the existing document/catalog pipelines — not a rewrite of them.

eval/knowledge/README.md

@@ -0,0 +1,84 @@
+# eval/knowledge — knowledge-extraction eval harness
+
+Scores the knowledge-extraction pipeline against a gold set built from
+**BUMA `STD/2026/006/MNO Rev.0.0` — Production Parameter & ECA** (9 pages, majority Bahasa
+Indonesia).
+
+Migrated from the `kex` prototype on 2026-08-19. **The prototype itself is not being ported** — it
+answered the design questions and stays where it is. This directory carries the parts that survive
+it: the gold set, the scorer, and a frozen baseline. Plan: [KNOWLEDGE_PIPELINE_TODO.md](../../KNOWLEDGE_PIPELINE_TODO.md),
+tuned constants: [KNOWLEDGE_PIPELINE_CALIBRATION.md](../../KNOWLEDGE_PIPELINE_CALIBRATION.md).
+
+## Why this landed before any v2 pipeline code
+
+Without a standing baseline, "new and improved" is unfalsifiable — and one of the four experiments
+already **failed** (E3, schema fill). Quality is what this work gets judged on, so the yardstick
+ships first and every v2 stage is measured from its first commit rather than retroactively.
+
+## Contents
+
+| Path | What it is |
+|---|---|
+| `knowledge_gold.yaml` | 41 terms + 15 rules. **Provisional** — bootstrapped by reading the source PDF, *not* expert-reviewed |
+| `score.py` | Precision / recall / F1 per branch. Pipeline-independent: takes plain lists of surfaces and entry dicts, so it scores the prototype's artifacts and v2 alike |
+| `results/baseline_prototype_2026-08-13_145132.json` | **Frozen.** The prototype's measured numbers. Never regenerate — add new timestamped files beside it |
+
+No `run_eval.py` yet: there is no v2 pipeline to run. It lands with the first stage that produces
+scoreable output, following the house convention (module mode, timestamped results, never
+overwritten) used by `eval/{intent,help,readiness,planner}/`.
+
+## The baseline to beat
+
+Measured by the prototype on the document above, run `20260813-145132`, deployment `gpt-5.4-nano`:
+
+| Experiment | Question | Result | Kill line | Verdict |
+|---|---|---|---|---|
+| **E1** | GLiNER recall on Bahasa Indonesia technical prose | **0.854** | 0.70 | **PASS** |
+| **E2** | Does clustering cut expert review burden? | **2.56×** (169 → 66) | 2.0× | **PASS** |
+| **E3** | Is nano sufficient for schema fill? | **0.75** | 0.80 | **FAIL** |
+| **E4** | Does verbatim-span validation catch fabrication? | **1.00** (false-rejection 0.00) | 0.90 | **PASS** |
+
+Funnel: 9 pages → 13 chunks → 169 mentions → 66 clusters → 83 LLM calls → 66 entries → 66 queue
+items. Cost $0.0069; 125,184 of 155,313 prompt tokens served from cache.
+
+## Two things the scorer refuses to do
+
+**It never conflates term-filter recall with extraction precision.** They are different failure
+modes with different fixes — recall is fixed at the filter stage (GLiNER labels), precision at the
+extraction stage (model tier, prompt). E1 is the recall number specifically.
+
+**It never counts abstention as an error.** For a term the document does not define, `null` is the
+correct answer. Scoring is restricted to the *scoreable* subset: entries that produced a definition,
+whose term is in the gold set, and whose gold record carries `definition_contains` to check against.
+Counting the rest as errors would measure gold coverage while claiming to measure model accuracy.
+Coverage is reported separately in `Score.as_dict()`.
+
+## Reading E3 carefully
+
+The scoreable base is **8 entries**, so each adjudication moves the number ~12 points. nano is also
+not deterministic at `temperature=0` — consecutive runs of the identical pipeline scored 0.75 and
+0.625. Treat E3 as a signal, not a measurement, until the gold set is reviewed.
+
+Every failure is listed verbatim under `experiments.E3.scoreable_basis.failures` in the baseline
+file, for Mas Beta to adjudicate. Some are genuine extraction errors; others are cases where the
+document carries two valid definitions and the provisional gold names only one. **Gold was not
+edited after seeing output**, so the figure is a lower bound.
+
+## Gold-set conventions (keep these when extending it)
+
+- `term` is the string a reader would look up.
+- `full_name` is the **literal source wording**, never normalised. The standard writes *"Physical of
+  Availability (PA)"* in the heading and *"Physical Availability"* in the legend — both are recorded
+  as variants. This is deliberate: the pipeline is required to surface that discrepancy to the
+  expert rather than silently correct it.
+- `definition_contains` are substrings that **must** appear in a correct definition. Substring
+  matching, not exact, so a correct-but-differently-worded extraction is not scored as a miss.
+- The gold set is **partial by design** — the expert is the labelling bottleneck, so scoring reports
+  coverage rather than blocking on a complete file.
+
+## Status of the gold set
+
+`status: provisional`, `labelled_by: claude-bootstrap`, `labelled_on: 2026-08-13`. It has **not**
+been reviewed by Mas Beta. Every score derived from it carries that caveat, and both E1 and E3 will
+move when it is reviewed. Accepted as good enough for now (2026-08-19, Rifqi) so the rebuild is not
+blocked on expert time.

eval/knowledge/knowledge_gold.yaml

@@ -0,0 +1,312 @@
+# PROVISIONAL gold set — bootstrapped by reading the source PDF, NOT yet expert-reviewed.
+# Reviewer: Mas Beta. Until reviewed, every score derived from this file carries that caveat.
+#
+# Labelling rules used:
+#  - `term` is the string a reader would look up.
+#  - `full_name` is the LITERAL source wording, not a normalised or corrected form.
+#    (e.g. the STD writes "Physical of Availability (PA)" in the heading and
+#     "Physical Availability" in the legend — both are recorded as variants.)
+#  - `definition_contains` are substrings that MUST appear in a correct definition.
+#    Substring matching, not exact match, so a correct-but-differently-worded
+#    extraction is not scored as a miss (spec §5).
+
+doc_id: STD_2026_006_MNO
+status: provisional
+labelled_by: claude-bootstrap
+labelled_on: 2026-08-13
+source_pages: 9
+
+terms:
+  - term: Production
+    full_name: Production
+    variants: ["Production", "produksi"]
+    definition_contains: ["nilai tambah", "sumber daya"]
+    formula_present: true
+    page: 2
+  - term: MOHH
+    full_name: Machine on Hand Hours
+    variants: ["MOHH"]
+    definition_contains: ["Machine on Hand Hours"]
+    formula_present: false
+    page: 2
+  - term: Qty
+    full_name: Quantity (Qty)
+    variants: ["Qty", "Quantity", "quantity"]
+    definition_contains: ["jumlah equipment", "periode"]
+    formula_present: true
+    page: 2
+  - term: PA
+    full_name: Physical of Availability (PA)
+    variants: ["PA", "Physical Availability", "Physical of Availability"]
+    definition_contains: ["ketersediaan fisik", "available"]
+    formula_present: true
+    page: 4
+  - term: UA
+    full_name: Utilization of Availability (UA)
+    variants: ["UA", "Utilization of Availability"]
+    definition_contains: ["efektifitas penggunaan", "working hours"]
+    formula_present: true
+    page: 4
+  - term: Pty
+    full_name: Productivity (Pty)
+    variants: ["Pty", "PTY", "Productivity", "Produktivitas"]
+    definition_contains: ["efektifitas penggunaan sumber daya", "output"]
+    formula_present: true
+    page: 5
+  - term: ECA
+    full_name: Equipment Capacity Analysis (ECA)
+    variants: ["ECA", "Equipment Capacity Analysis"]
+    definition_contains: ["ketidaktercapaian", "kapasitas produksi"]
+    formula_present: false
+    page: 6
+  - term: INPR
+    full_name: in operation
+    variants: ["INPR", "in operation", "INPR Hours"]
+    definition_contains: ["in operation"]
+    formula_present: false
+    page: 2
+  - term: Total Hours
+    full_name: Total Hours
+    variants: ["Total Hours", "total jam"]
+    definition_contains: ["total jam"]
+    formula_present: true
+    page: 2
+  - term: Breakdown
+    full_name: Breakdown
+    variants: ["Breakdown", "breakdown"]
+    definition_contains: []
+    formula_present: true
+    page: 4
+  - term: Working Hours
+    full_name: Working Hours
+    variants: ["Working Hours", "working hours"]
+    definition_contains: ["waktu kerja"]
+    formula_present: true
+    page: 4
+  - term: EWH
+    full_name: Effective Working Hours (EWH)
+    variants: ["EWH", "Effective Working Hours", "productive working hours"]
+    definition_contains: ["waktu kerja efektif"]
+    formula_present: false
+    page: 5
+  - term: Land clearing
+    full_name: land clearing
+    variants: ["land clearing"]
+    definition_contains: []
+    formula_present: false
+    page: 2
+  - term: Overburden removal
+    full_name: pengupasan lapisan penutup (overburden removal)
+    variants: ["overburden removal", "pengupasan lapisan penutup"]
+    definition_contains: []
+    formula_present: false
+    page: 2
+  - term: Coal mining
+    full_name: penambangan batubara (coal mining)
+    variants: ["coal mining", "Coal Mining", "CM", "penambangan batubara"]
+    definition_contains: []
+    formula_present: false
+    page: 2
+  - term: Coal hauling
+    full_name: pengangkutan batubara (coal hauling/coal transporting)
+    variants: ["coal hauling", "coal transporting", "Coal Transport", "CT"]
+    definition_contains: []
+    formula_present: false
+    page: 2
+  - term: Coal barging
+    full_name: coal barging atau ship loading
+    variants: ["coal barging", "ship loading", "pengapalan"]
+    definition_contains: []
+    formula_present: false
+    page: 2
+  - term: Waste Removal
+    full_name: Waste Removal (WR)
+    variants: ["Waste Removal", "WR", "waste removal"]
+    definition_contains: []
+    formula_present: false
+    page: 3
+  - term: General Work
+    full_name: General Work
+    variants: ["General Work"]
+    definition_contains: []
+    formula_present: false
+    page: 3
+  - term: Rental
+    full_name: Rental
+    variants: ["Rental"]
+    definition_contains: []
+    formula_present: false
+    page: 3
+  - term: Grouping (Composite)
+    full_name: Grouping (Composite)
+    variants: ["Grouping (Composite)", "grouping (composite)", "Composite"]
+    definition_contains: ["type", "class"]
+    formula_present: true
+    page: 3
+  - term: EX2500
+    full_name: EX2500
+    variants: ["EX2500"]
+    definition_contains: []
+    formula_present: false
+    page: 3
+  - term: Time Performance MCD Application
+    full_name: raw data Time Performance MCD Application
+    variants: ["Time Performance MCD Application", "MCD Application", "MCD"]
+    definition_contains: []
+    formula_present: false
+    page: 3
+  - term: Joint survey
+    full_name: joint survey
+    variants: ["joint survey"]
+    definition_contains: []
+    formula_present: false
+    page: 5
+  - term: Truck count
+    full_name: truck count
+    variants: ["truck count"]
+    definition_contains: []
+    formula_present: false
+    page: 5
+  - term: Weight average
+    full_name: rata-rata tertimbang (weight average)
+    variants: ["weight average", "rata-rata tertimbang", "pembobotan"]
+    definition_contains: []
+    formula_present: false
+    page: 4
+  - term: Gain/Loss
+    full_name: Gain/Loss
+    variants: ["Gain/Loss", "Gain / Loss"]
+    definition_contains: ["plan", "actual"]
+    formula_present: true
+    page: 6
+  - term: Plan
+    full_name: rencana (plan)
+    variants: ["plan", "Plan", "rencana", "P(M)", "P(G)"]
+    definition_contains: []
+    formula_present: false
+    page: 6
+  - term: Actual
+    full_name: realisasi (actual)
+    variants: ["actual", "Actual", "realisasi", "A(M)", "A(G)"]
+    definition_contains: []
+    formula_present: false
+    page: 6
+  - term: Model Unit
+    full_name: Model Unit
+    variants: ["Model Unit", "model unit", "(M)", "Model"]
+    definition_contains: []
+    formula_present: true
+    page: 6
+  - term: Standby
+    full_name: Delay/Standby
+    variants: ["Standby", "standby", "Delay/Standby", "Gap Standby"]
+    definition_contains: []
+    formula_present: true
+    page: 6
+  - term: BCM
+    full_name: BCM
+    variants: ["BCM"]
+    definition_contains: []
+    formula_present: false
+    page: 6
+  - term: Waterfall Analysis
+    full_name: Waterfall Analysis
+    variants: ["Waterfall Analysis", "grafik Waterfall"]
+    definition_contains: ["visualisasi", "Gain/Loss"]
+    formula_present: false
+    page: 8
+  - term: Other Activity
+    full_name: Other Activity
+    variants: ["Other Activity"]
+    definition_contains: ["tidak sesuai", "direncanakan"]
+    formula_present: false
+    page: 8
+  - term: Loader
+    full_name: loader
+    variants: ["loader", "PA Loader"]
+    definition_contains: []
+    formula_present: false
+    page: 8
+  - term: Hauler
+    full_name: hauler
+    variants: ["hauler"]
+    definition_contains: []
+    formula_present: false
+    page: 8
+  - term: Hierarki Level Analysis
+    full_name: Hierarki Level Analyisis
+    variants: ["Hierarki Level Analyisis", "Hierarki Level Analysis"]
+    definition_contains: ["level"]
+    formula_present: false
+    page: 9
+  - term: Uncontrollable
+    full_name: Uncontrollable
+    variants: ["Uncontrollable", "uncontrollable"]
+    definition_contains: ["level 1"]
+    formula_present: false
+    page: 9
+  - term: Controllable
+    full_name: controllable
+    variants: ["controllable", "UA Controllable"]
+    definition_contains: ["level 1"]
+    formula_present: false
+    page: 9
+  - term: Fleet management
+    full_name: fleet management
+    variants: ["fleet management"]
+    definition_contains: []
+    formula_present: false
+    page: 9
+  - term: Mineplan
+    full_name: mineplan
+    variants: ["mineplan"]
+    definition_contains: []
+    formula_present: false
+    page: 9
+
+rules:
+  - rule_id: QTY_PER_ACTIVITY
+    statement_contains: ["Qty", "kategori aktivitas", "terpisah"]
+    page: 3
+  - rule_id: QTY_GROUPING_SUM
+    statement_contains: ["Qty Grouping", "menjumlahkan"]
+    page: 3
+  - rule_id: ACTIVITY_CLASSIFICATION_SOURCE
+    statement_contains: ["status equipment", "Time Performance MCD"]
+    page: 3
+  - rule_id: PA_COMPOSITE_WEIGHTED
+    statement_contains: ["weight average", "Quantity", "pembobot"]
+    page: 4
+  - rule_id: UA_BY_ACTIVITY
+    statement_contains: ["UA", "kategori aktivitas"]
+    page: 4
+  - rule_id: PTY_PRODUCTION_SOURCE
+    statement_contains: ["joint survey", "truck count"]
+    page: 5
+  - rule_id: PTY_WORKING_HOURS_DEFINITION
+    statement_contains: ["Effective Working Hours", "waktu kerja efektif"]
+    page: 5
+  - rule_id: PTY_COMPOSITE_WEIGHTED
+    statement_contains: ["Pty", "pembobotan", "Quantity"]
+    page: 5
+  - rule_id: GAINLOSS_QTY_ZERO_PLAN
+    statement_contains: ["QtyP(M) = 0", "QtyA(M) > 0"]
+    page: 6
+  - rule_id: GAINLOSS_COMPOSITE_SUM
+    statement_contains: ["grouping", "penjumlahan", "model unit"]
+    page: 6
+  - rule_id: GAINLOSS_UNITS
+    statement_contains: ["BCM", "ton", "jam"]
+    page: 6
+  - rule_id: QTY_ACTUAL_ZERO_PLAN_ALT
+    statement_contains: ["QtyA", "QtyP", "0"]
+    page: 7
+  - rule_id: QTY_PHYSICAL_REPORTING
+    statement_contains: ["fisik unit", "INPR"]
+    page: 7
+  - rule_id: ECA_OTHER_ACTIVITY
+    statement_contains: ["alokasi", "tidak sesuai", "direncanakan"]
+    page: 8
+  - rule_id: WATERFALL_TWO_METHODS
+    statement_contains: ["Waterfall", "dua metode"]
+    page: 8

eval/knowledge/results/baseline_prototype_2026-08-13_145132.json

@@ -0,0 +1,157 @@
+{
+  "_note": "FROZEN BASELINE — measured by the `kex` prototype, not by this repo. Extraction v2 must match or beat these numbers on the same document and the same gold set. Copied from the prototype's eval/out/metrics.json on 2026-08-19; never regenerate this file, add a new timestamped one.",
+  "_source": {
+    "repo": "kex prototype (outside version control)",
+    "file": "eval/out/metrics.json",
+    "copied_on": "2026-08-19"
+  },
+  "provenance": {
+    "run_id": "20260813-145132",
+    "config_hash": "89545a78c702",
+    "created_at": "2026-08-13T14:51:32",
+    "doc_id": "STD_2026_006_MNO",
+    "client_mode": "azure",
+    "deployments": {
+      "glossary": "gpt-5.4-nano",
+      "rule": "gpt-5.4-nano",
+      "formula": "gpt-5.4-nano",
+      "summary": "gpt-5.4-nano"
+    },
+    "api_version": "2024-12-01-preview",
+    "gold_status": "provisional — not yet expert-reviewed"
+  },
+  "funnel": {
+    "status": "measured",
+    "stages": [
+      {
+        "stage": "pages",
+        "count": 9
+      },
+      {
+        "stage": "chunks",
+        "count": 13
+      },
+      {
+        "stage": "mentions",
+        "count": 169
+      },
+      {
+        "stage": "clusters",
+        "count": 66
+      },
+      {
+        "stage": "llm_calls",
+        "count": 83
+      },
+      {
+        "stage": "glossary_entries",
+        "count": 66
+      },
+      {
+        "stage": "review_queue",
+        "count": 66
+      }
+    ],
+    "raw_mentions_before_noise_filter": 195
+  },
+  "experiments": {
+    "E1": {
+      "question": "GLiNER recall on Bahasa Indonesia technical prose",
+      "status": "measured",
+      "value": 0.8537,
+      "kill_line": 0.7,
+      "verdict": "PASS",
+      "detail": "broad @ threshold 0.25",
+      "n_gold": 41
+    },
+    "E2": {
+      "question": "Does clustering cut expert review burden?",
+      "status": "measured",
+      "value": 2.561,
+      "kill_line": 2.0,
+      "verdict": "PASS",
+      "detail": "169 mentions -> 66 clusters",
+      "llm_calls_avoided": 103
+    },
+    "E3": {
+      "question": "Is nano sufficient for schema fill?",
+      "status": "measured_nano_only",
+      "value": 0.75,
+      "kill_line": 0.8,
+      "verdict": "FAIL",
+      "detail": "nano only — no mini deployment provisioned, so the spec's nano-vs-mini A/B could not be run",
+      "glossary_precision": 0.75,
+      "glossary_recall": 0.3333,
+      "glossary_f1": 0.4615,
+      "rule_precision": 0.5714,
+      "rule_recall": 0.2667,
+      "n_gold_terms": 18,
+      "scoreable_basis": {
+        "entries_total": 66,
+        "entries_with_definition": 10,
+        "abstained_null_definition": 56,
+        "scoreable": 8,
+        "unscoreable_term_not_in_gold": 2,
+        "unscoreable_gold_has_no_criteria": 0,
+        "gold_terms_with_criteria": 18,
+        "failures": [
+          {
+            "term": "produksi",
+            "gold_requires": [
+              "nilai tambah",
+              "sumber daya"
+            ],
+            "extracted": "Dalam konteks industri pertambangan, produksi merupakan seluruh rangkaian kegiatan operasional penambangan yang dimulai dari land clearing, pengupasan lapisan penutup (overburden removal), penambangan batubara (coal mining), pengangkutan ba"
+          },
+          {
+            "term": "Production",
+            "gold_requires": [
+              "nilai tambah",
+              "sumber daya"
+            ],
+            "extracted": "Dalam konteks industri pertambangan, produksi merupakan seluruh rangkaian kegiatan operasional penambangan yang dimulai dari land clearing, pengupasan lapisan penutup (overburden removal), penambangan batubara (coal mining), pengangkutan ba"
+          }
+        ]
+      },
+      "abstention_note": "entries where nano returned null are NOT counted as errors — for a term the document never defines, null is the correct answer",
+      "adjudication_pending": "The scoreable base is small, so each failure moves the number by ~12 points. Every failure is listed verbatim under scoreable_basis.failures for Mas Beta to adjudicate: some are genuine extraction errors, others are cases where the document carries two valid definitions and the provisional gold names only one. Gold was NOT edited after seeing output, so this figure is a lower bound.",
+      "stability_warning": "nano is not fully deterministic at temperature=0: consecutive runs of the identical pipeline produced precision 0.75 and 0.625. On a base this small, treat E3 as a signal, not a measurement."
+    },
+    "E4": {
+      "question": "Does verbatim-span validation catch fabrication?",
+      "status": "measured",
+      "value": 1.0,
+      "kill_line": 0.9,
+      "verdict": "PASS",
+      "false_rejection_rate": 0.0,
+      "detail": "span_check is deterministic; measures the control itself"
+    }
+  },
+  "quality": {
+    "status": "measured",
+    "glossary_entries": 66,
+    "rule_entries": 7,
+    "formula_entries": 7,
+    "fields_rejected_by_span_check": 3,
+    "no_definition_found": 56,
+    "escalated": 0,
+    "definition_conflicts": 0
+  },
+  "efficiency": {
+    "status": "measured + stated assumption",
+    "llm_calls_avoided_by_clustering": 103,
+    "review_queue_size": 66,
+    "assumed_minutes_per_term_review": 3.0,
+    "assumed_minutes_per_term_manual": 12.0,
+    "est_review_hours": 3.3,
+    "est_manual_hours": 13.2,
+    "est_hours_saved": 9.9
+  },
+  "cost_measured": {
+    "prompt": 155313,
+    "cached": 125184,
+    "completion": 11862,
+    "usd": 0.006877,
+    "latency_s": 242.4
+  }
+}

eval/knowledge/results/v2_parser_comparison_2026-08-19_152842.json

@@ -0,0 +1,59 @@
+{
+  "_note": "Parser comparison on the SAME document, gold set and pipeline code. Answers whether MinerU input is interchangeable with the PyMuPDF parse the frozen baseline was measured on.",
+  "_caveat": "The MinerU artifacts were built by a SCRATCH converter in the scratchpad, not by production code — section-aware chunking is Sofhia's B5. Treat the chunking-related findings as indicative of what her chunker must handle, not as a measurement of MinerU.",
+  "run": {
+    "created_at": "2026-08-19_152842",
+    "doc_id": "STD_2026_006_MNO",
+    "implementation": "src/knowledge_extraction (v2)"
+  },
+  "variants": {
+    "pymupdf_prototype_parse": {
+      "E1_recall": 0.8537,
+      "true_positives": 35,
+      "chunks": 13,
+      "note": "what the frozen baseline used"
+    },
+    "mineru_raw_markup": {
+      "E1_recall": 0.7561,
+      "true_positives": 31,
+      "chunks": 14,
+      "misses": [
+        "Total Hours",
+        "Breakdown",
+        "Overburden removal",
+        "Grouping (Composite)",
+        "Weight average",
+        "Plan",
+        "Other Activity",
+        "Uncontrollable",
+        "Fleet management",
+        "Mineplan"
+      ],
+      "note": "formulas left as LaTeX, tables as HTML — the span model reads prose, not markup"
+    },
+    "mineru_rendered_to_prose": {
+      "E1_recall": 0.8293,
+      "true_positives": 34,
+      "chunks": 14,
+      "misses": [
+        "Overburden removal",
+        "Grouping (Composite)",
+        "Weight average",
+        "Other Activity",
+        "Uncontrollable",
+        "Fleet management",
+        "Mineplan"
+      ],
+      "note": "LaTeX unwrapped, tables flattened to rows; recovers most of the loss"
+    }
+  },
+  "n_gold": 41,
+  "kill_line": 0.7,
+  "findings": [
+    "Raw MinerU markup costs ~9.8 recall points: LaTeX-wrapped terms (Total Hours, Breakdown) are invisible to the span filter.",
+    "Rendering formulas and tables to prose recovers +7.3 points, to within ~2.4 points of the PyMuPDF parse.",
+    "The residual gap is CHUNKING, not content: MinerU marks non-section labels (Keterangan:, Keterangan grafik:) as text_level titles. Opening a new section at each one splits a legend from the figure it explains, and Other Activity / Uncontrollable are then missed despite being present as prose.",
+    "Actionable for the parsing half: only NUMBERED headings should open a section. This matches the earlier finding that MinerU flattens every heading to level 2, so numbering is the only reliable section signal.",
+    "The artifact should carry formulas and tables as readable text, not raw LaTeX/HTML, or the filter stage must render them before NER."
+  ]
+}

eval/knowledge/results/v2_term_filter_2026-08-19_144318.json

@@ -0,0 +1,53 @@
+{
+  "_note": "First v2 run. Term-filter (E1) only — the paid extraction stage is not built, so E3 is not scoreable yet.",
+  "run": {
+    "created_at": "2026-08-19_144318",
+    "implementation": "src/knowledge_extraction (v2)",
+    "doc_id": "STD_2026_006_MNO"
+  },
+  "config": {
+    "model": "urchade/gliner_multi-v2.1",
+    "labels_variant": "broad",
+    "threshold": 0.25,
+    "window_words": 130,
+    "window_overlap": 30
+  },
+  "gold": {
+    "path": "eval/knowledge/knowledge_gold.yaml",
+    "status": "provisional — not expert-reviewed",
+    "n_terms": 41
+  },
+  "funnel": {
+    "chunks": 13,
+    "mentions_raw": 188,
+    "mentions_after_noise": 163,
+    "clusters": 66,
+    "compression_ratio": 2.47
+  },
+  "E1": {
+    "metric": "term_filter_recall",
+    "value": 0.8537,
+    "kill_line": 0.7,
+    "verdict": "PASS",
+    "true_positives": 35,
+    "precision": 0.4118,
+    "precision_note": "reported, NOT the E1 criterion — the filter is deliberately over-inclusive",
+    "misses": [
+      "Overburden removal",
+      "Grouping (Composite)",
+      "Weight average",
+      "Plan",
+      "Fleet management",
+      "Mineplan"
+    ]
+  },
+  "baseline_comparison": {
+    "baseline_file": "results/baseline_prototype_2026-08-13_145132.json",
+    "E1_baseline": 0.8537,
+    "E1_v2": 0.8537,
+    "delta": 0.0,
+    "clusters_baseline": 66,
+    "clusters_v2": 66,
+    "verdict": "MATCH — v2 reproduces the prototype recall exactly"
+  }
+}

eval/knowledge/score.py

@@ -0,0 +1,227 @@
+"""Precision / recall / F1 per branch of the knowledge-extraction pipeline.
+
+Migrated verbatim (bar the import bootstrap) from the `kex` prototype, 2026-08-19.
+The prototype is not being ported; this scorer is, because it is the only thing
+that can show extraction v2 matches or beats the measured baseline in
+`results/baseline_prototype_2026-08-13_145132.json`.
+
+Pipeline-independent by design: it scores plain lists of surfaces / entry dicts,
+so it works against the prototype's artifacts and against v2 alike.
+
+The one thing this module refuses to do is conflate **term-filter recall** with
+**extraction precision** (spec §5). They are different failure modes with
+different fixes: recall is fixed at stage 2 (GLiNER labels), precision is fixed
+at stage 3 (model tier / prompt). E1 is the recall number specifically.
+
+Gold sets are treated as partial by design — Mas Beta is the labelling
+bottleneck, so scoring reports coverage rather than blocking on a complete file.
+"""
+
+from __future__ import annotations
+
+import re
+import unicodedata
+from dataclasses import asdict, dataclass
+from pathlib import Path
+
+import yaml
+
+GOLD_PATH = Path(__file__).resolve().parent / "knowledge_gold.yaml"
+
+
+def norm(s: str) -> str:
+    s = unicodedata.normalize("NFKC", s).casefold()
+    s = re.sub(r"[^\w\s]", " ", s)
+    return re.sub(r"\s+", " ", s).strip()
+
+
+@dataclass
+class Score:
+    label: str
+    n_gold: int
+    n_pred: int
+    true_positives: int
+    precision: float
+    recall: float
+    f1: float
+    misses: list[str]
+
+    def as_dict(self) -> dict:
+        d = asdict(self)
+        if hasattr(self, "coverage"):
+            d["coverage"] = self.coverage
+        return d
+
+
+def load_gold(path: Path) -> dict:
+    with open(path, encoding="utf-8") as fh:
+        return yaml.safe_load(fh)
+
+
+def _prf(tp: int, n_pred: int, n_gold: int) -> tuple[float, float, float]:
+    precision = tp / n_pred if n_pred else 0.0
+    recall = tp / n_gold if n_gold else 0.0
+    f1 = (
+        2 * precision * recall / (precision + recall) if (precision + recall) else 0.0
+    )
+    return round(precision, 4), round(recall, 4), round(f1, 4)
+
+
+def score_term_filter(gold: dict, surfaces: list[str]) -> Score:
+    """E1. A gold term counts as recalled if ANY of its variants appears among
+    the filter's mention surfaces (substring match both ways, so 'PA' inside
+    'Physical Availability (PA)' counts).
+
+    Precision is reported but is NOT E1's criterion: the filter is deliberately
+    over-inclusive, and clustering plus evidence ranking absorb the noise.
+    """
+    normed = {norm(s) for s in surfaces if norm(s)}
+    blob = " | ".join(sorted(normed))
+    tp, misses = 0, []
+    gold_terms = gold.get("terms", [])
+    for entry in gold_terms:
+        variants = [entry["term"]] + list(entry.get("variants", []))
+        if entry.get("full_name"):
+            variants.append(entry["full_name"])
+        hit = False
+        for v in variants:
+            nv = norm(v)
+            if not nv:
+                continue
+            if nv in normed or re.search(rf"(?<![\w]){re.escape(nv)}(?![\w])", blob):
+                hit = True
+                break
+        if hit:
+            tp += 1
+        else:
+            misses.append(entry["term"])
+    precision, recall, f1 = _prf(tp, len(normed), len(gold_terms))
+    return Score(
+        label="term_filter_recall(E1)",
+        n_gold=len(gold_terms),
+        n_pred=len(normed),
+        true_positives=tp,
+        precision=precision,
+        recall=recall,
+        f1=f1,
+        misses=misses,
+    )
+
+
+def score_glossary(gold: dict, entries: list[dict]) -> Score:
+    """E3: when nano fills the schema, is it right?
+
+    Scoring is restricted to the SCOREABLE subset: entries that produced a
+    definition AND whose term is in the gold set AND whose gold record carries
+    `definition_contains` to check against.
+
+    Why not simply tp/len(entries): the term filter is deliberately
+    over-inclusive and the gold set is deliberately partial, so most entries are
+    for terms gold says nothing about. Counting those as errors would measure
+    gold coverage while claiming to measure nano's accuracy — precisely the
+    conflation spec §5 forbids. Coverage is reported separately in as_dict().
+
+    Substring matching, not exact — exact match would under-report
+    correct-but-differently-worded extractions (spec §5).
+    """
+    gold_by_term: dict[str, dict] = {}
+    for entry in gold.get("terms", []):
+        for v in [entry["term"], *entry.get("variants", [])]:
+            gold_by_term.setdefault(norm(v), entry)
+
+    checkable_gold = [
+        g for g in gold.get("terms", []) if g.get("definition_contains")
+    ]
+    n_checkable_gold = len(checkable_gold)
+
+    correct, incorrect = 0, 0
+    unscoreable_no_gold, unscoreable_no_criteria = 0, 0
+    matched_gold, wrong, failures = set(), [], []
+
+    for pred in entries:
+        if not (pred.get("definition") or "").strip():
+            continue  # abstention is scored separately, not as an error
+        g = gold_by_term.get(norm(pred.get("term", "")))
+        if not g:
+            unscoreable_no_gold += 1
+            continue
+        required = [norm(x) for x in g.get("definition_contains", [])]
+        if not required:
+            unscoreable_no_criteria += 1
+            continue
+        definition = norm(pred.get("definition") or "")
+        if all(r in definition for r in required):
+            correct += 1
+            matched_gold.add(g["term"])
+        else:
+            incorrect += 1
+            wrong.append(f"{pred.get('term')} (definition did not match gold)")
+            failures.append(
+                {
+                    "term": pred.get("term"),
+                    "gold_requires": g.get("definition_contains"),
+                    "extracted": (pred.get("definition") or "")[:240],
+                }
+            )
+
+    misses = [g["term"] for g in checkable_gold if g["term"] not in matched_gold]
+    n_scoreable = correct + incorrect
+    precision, recall, f1 = _prf(correct, n_scoreable, n_checkable_gold)
+
+    score = Score(
+        label="glossary_schema_fill(E3)",
+        n_gold=n_checkable_gold,
+        n_pred=n_scoreable,
+        true_positives=correct,
+        precision=precision,
+        recall=recall,
+        f1=f1,
+        misses=misses + wrong,
+    )
+    score.coverage = {  # type: ignore[attr-defined]
+        "entries_total": len(entries),
+        "entries_with_definition": sum(
+            1 for e in entries if (e.get("definition") or "").strip()
+        ),
+        "abstained_null_definition": sum(
+            1 for e in entries if not (e.get("definition") or "").strip()
+        ),
+        "scoreable": n_scoreable,
+        "unscoreable_term_not_in_gold": unscoreable_no_gold,
+        "unscoreable_gold_has_no_criteria": unscoreable_no_criteria,
+        "gold_terms_with_criteria": n_checkable_gold,
+        "failures": failures,
+    }
+    return score
+
+
+def score_rules(gold: dict, entries: list[dict]) -> Score:
+    gold_rules = gold.get("rules", [])
+    pred_blobs = [
+        norm(
+            " ".join(
+                str(v)
+                for v in (e.get("statement"), e.get("condition"), e.get("consequence"))
+                if v
+            )
+        )
+        for e in entries
+    ]
+    tp, misses = 0, []
+    for rule in gold_rules:
+        required = [norm(x) for x in rule.get("statement_contains", [])]
+        if any(all(r in blob for r in required) for blob in pred_blobs):
+            tp += 1
+        else:
+            misses.append(rule["rule_id"])
+    precision, recall, f1 = _prf(tp, len(entries), len(gold_rules))
+    return Score(
+        label="rule",
+        n_gold=len(gold_rules),
+        n_pred=len(entries),
+        true_positives=tp,
+        precision=precision,
+        recall=recall,
+        f1=f1,
+        misses=misses,
+    )

eval/planner/results/planner_result_2026-07-23_134947.json

@@ -0,0 +1,662 @@
+{
+  "timestamp": "2026-07-23_134947",
+  "total": 27,
+  "passed": 20,
+  "cases": [
+    {
+      "id": "count_zero_pa",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records have PA_Percent = 0?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "group_by_present=False"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3591,
+      "tokens": 14570
+    },
+    {
+      "id": "count_mttr_gt20_id",
+      "category": "count",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record dengan MTTR di atas 20?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2154,
+      "tokens": 14571
+    },
+    {
+      "id": "count_section_hauler",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many rows are in section OB HAULER?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2063,
+      "tokens": 14573
+    },
+    {
+      "id": "rank_units_worst_pa_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "5 unit dengan PA terburuk?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "group_by_present=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "want_id=c_equipment_number"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=5",
+          "ok": true,
+          "detail": "limits=[5]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2664,
+      "tokens": 14784
+    },
+    {
+      "id": "rank_models_top_mttr_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": true,
+      "question": "top 3 model dengan MTTR tertinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "group_by_present=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "want_id=c_model_unit"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        },
+        {
+          "check": "limit=3",
+          "ok": true,
+          "detail": "limits=[3]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2757,
+      "tokens": 14719
+    },
+    {
+      "id": "rank_sections_lowest_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": true,
+      "question": "which section has the lowest average PA?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": false,
+          "detail": "group_by_present=False"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": false,
+          "detail": "want_id=c_section"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": false,
+          "detail": "aggs=[]"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": false,
+          "detail": "dirs=[]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3366,
+      "tokens": 14834
+    },
+    {
+      "id": "rank_units_most_breakdown_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "unit mana yang paling sering breakdown?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "group_by_present=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "want_id=c_equipment_number"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2925,
+      "tokens": 14736
+    },
+    {
+      "id": "rank_units_worst_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": false,
+      "question": "list the 10 worst units by availability",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "group_by_present=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "want_id=c_equipment_number"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3001,
+      "tokens": 14758
+    },
+    {
+      "id": "fuzzy_model_777_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": false,
+      "question": "berapa banyak model 777?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "filter_op=like",
+          "ok": false,
+          "detail": "ops=['=']"
+        },
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['=']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 1831,
+      "tokens": 14578
+    },
+    {
+      "id": "fuzzy_model_hd785_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak unit HD785?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "filter_op=like",
+          "ok": false,
+          "detail": "ops=['=']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2219,
+      "tokens": 14578
+    },
+    {
+      "id": "fuzzy_model_ex_en",
+      "category": "fuzzy_filter",
+      "lang": "en",
+      "carried_over": false,
+      "question": "how many EX excavator units are there?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "filter_op=like",
+          "ok": true,
+          "detail": "ops=['like']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 5243,
+      "tokens": 29547
+    },
+    {
+      "id": "agg_pa_per_section_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa rata-rata PA per section?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": false,
+          "detail": "group_by_present=False"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": false,
+          "detail": "want_id=c_section"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": false,
+          "detail": "aggs=[]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2817,
+      "tokens": 14757
+    },
+    {
+      "id": "agg_mttr_per_model_en",
+      "category": "aggregate",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the average MTTR per model unit?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": false,
+          "detail": "group_by_present=False"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": false,
+          "detail": "want_id=c_model_unit"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": false,
+          "detail": "aggs=[]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2859,
+      "tokens": 14773
+    },
+    {
+      "id": "agg_downhour_per_group_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "total down hour per equipment group?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": false,
+          "detail": "group_by_present=False"
+        },
+        {
+          "check": "group_by_col=Equipment_Group_ID",
+          "ok": false,
+          "detail": "want_id=c_equipment_group_id"
+        },
+        {
+          "check": "select_agg=sum",
+          "ok": false,
+          "detail": "aggs=[]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2855,
+      "tokens": 14769
+    },
+    {
+      "id": "desc_mttr_stats_id",
+      "category": "descriptive",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berikan ringkasan statistik MTTR",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2928,
+      "tokens": 14714
+    },
+    {
+      "id": "desc_pa_stats_en",
+      "category": "descriptive",
+      "lang": "en",
+      "carried_over": true,
+      "question": "give me the summary statistics for PA_Percent",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2862,
+      "tokens": 14704
+    },
+    {
+      "id": "corr_mttr_pa_id",
+      "category": "correlation",
+      "lang": "id",
+      "carried_over": true,
+      "question": "apakah ada korelasi antara MTTR dan PA?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3171,
+      "tokens": 14866
+    },
+    {
+      "id": "corr_freq_pa_en",
+      "category": "correlation",
+      "lang": "en",
+      "carried_over": true,
+      "question": "is breakdown frequency correlated with availability?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3259,
+      "tokens": 14921
+    },
+    {
+      "id": "trend_pa_daily_id",
+      "category": "trend",
+      "lang": "id",
+      "carried_over": true,
+      "question": "bagaimana trend PA harian?",
+      "passed": false,
+      "checks": [],
+      "error": "ValidationError: 6 validation errors for TaskList\ntasks.2.id\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing\ntasks.2.stage\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing\ntasks.2.objective\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing\ntasks.2.tool_calls\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing\ntasks.2.expected_output\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing\ntasks.2.success_criteria\n  Field required [type=missing, input_value={'tool': 'analyze_trend',...: 'day', 'agg': 'mean'}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing",
+      "latency_ms": 3008,
+      "tokens": 14775
+    },
+    {
+      "id": "trend_downhour_en",
+      "category": "trend",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show the trend of total down hours over time",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_trend",
+          "ok": true,
+          "detail": "tools=['analyze_trend', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "error": null,
+      "latency_ms": 3193,
+      "tokens": 14842
+    },
+    {
+      "id": "merge_worst_pa_and_mttr_id",
+      "category": "merge",
+      "lang": "id",
+      "carried_over": true,
+      "question": "model mana yang PA-nya paling buruk sekaligus MTTR-nya paling tinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "group_by_present=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "want_id=c_model_unit"
+        }
+      ],
+      "error": null,
+      "latency_ms": 4061,
+      "tokens": 15116
+    },
+    {
+      "id": "raw_rows_low_pa_id",
+      "category": "counter_raw_rows",
+      "lang": "id",
+      "carried_over": true,
+      "question": "tampilkan 10 record dengan PA di bawah 50",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "group_by_present=False"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2944,
+      "tokens": 14868
+    },
+    {
+      "id": "raw_rows_head_en",
+      "category": "counter_raw_rows",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show me the first 5 rows of the data",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "group_by_present=False"
+        }
+      ],
+      "error": null,
+      "latency_ms": 4427,
+      "tokens": 15315
+    },
+    {
+      "id": "exact_model_777d_id",
+      "category": "counter_exact_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record untuk model 777D?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2314,
+      "tokens": 14568
+    },
+    {
+      "id": "exact_section_loader_en",
+      "category": "counter_exact_filter",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records are in the OB LOADER section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2216,
+      "tokens": 14595
+    },
+    {
+      "id": "infeasible_churn_id",
+      "category": "infeasible",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa churn rate pelanggan?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 1775,
+      "tokens": 14477
+    },
+    {
+      "id": "infeasible_profit_en",
+      "category": "infeasible",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the monthly profit margin?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "error": null,
+      "latency_ms": 2282,
+      "tokens": 14551
+    }
+  ]
+}

eval/planner/results/planner_result_2026-07-23_141931.json

@@ -0,0 +1,1818 @@
+{
+  "timestamp": "2026-07-23_141931",
+  "total": 27,
+  "passed": 25,
+  "cases": [
+    {
+      "id": "count_zero_pa",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records have PA_Percent = 0?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_pa_percent",
+                "op": "=",
+                "value": 0,
+                "value_type": "decimal"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3652,
+      "tokens": 14570
+    },
+    {
+      "id": "count_mttr_gt20_id",
+      "category": "count",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record dengan MTTR di atas 20?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_mttr",
+                "op": ">",
+                "value": 20,
+                "value_type": "decimal"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2176,
+      "tokens": 14571
+    },
+    {
+      "id": "count_section_hauler",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many rows are in section OB HAULER?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "row_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_section",
+                "op": "=",
+                "value": "OB HAULER",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2342,
+      "tokens": 14571
+    },
+    {
+      "id": "rank_units_worst_pa_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "5 unit dengan PA terburuk?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[]"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=5",
+          "ok": true,
+          "detail": "limits=[5]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa_percent"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 5
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3519,
+      "tokens": 14733
+    },
+    {
+      "id": "rank_models_top_mttr_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": true,
+      "question": "top 3 model dengan MTTR tertinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=['c_model_unit'] aliases=[]"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        },
+        {
+          "check": "limit=3",
+          "ok": true,
+          "detail": "limits=[3]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_mttr",
+                "alias": "avg_mttr"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_mttr",
+                "dir": "desc"
+              }
+            ],
+            "limit": 3
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2928,
+      "tokens": 14723
+    },
+    {
+      "id": "rank_sections_lowest_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": true,
+      "question": "which section has the lowest average PA?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": true,
+          "detail": "ids=[] aliases=['section']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": false,
+          "detail": "dirs=[]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "pa_percent": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "section"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3445,
+      "tokens": 14833
+    },
+    {
+      "id": "rank_units_most_breakdown_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "unit mana yang paling sering breakdown?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[]"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "sum",
+                "column_id": "c_total_frequency_down",
+                "alias": "total_breakdown_frequency"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "total_breakdown_frequency",
+                "dir": "desc"
+              }
+            ],
+            "limit": 1
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2590,
+      "tokens": 14760
+    },
+    {
+      "id": "rank_units_worst_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": false,
+      "question": "list the 10 worst units by availability",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[]"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 10
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3221,
+      "tokens": 14791
+    },
+    {
+      "id": "fuzzy_model_777_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": false,
+      "question": "berapa banyak model 777?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['=']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "777",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2332,
+      "tokens": 14576
+    },
+    {
+      "id": "fuzzy_model_hd785_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak unit HD785?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['=']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "unit_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "HD785",
+                "value_type": "string"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 4613,
+      "tokens": 29329
+    },
+    {
+      "id": "fuzzy_model_ex_en",
+      "category": "fuzzy_filter",
+      "lang": "en",
+      "carried_over": false,
+      "question": "how many EX excavator units are there?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['like']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "unit_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_equipment_number",
+                "op": "like",
+                "value": "EX%",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 5577,
+      "tokens": 29385
+    },
+    {
+      "id": "agg_pa_per_section_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa rata-rata PA per section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": true,
+          "detail": "ids=[] aliases=['section']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "pa_percent": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "section"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3785,
+      "tokens": 14754
+    },
+    {
+      "id": "agg_mttr_per_model_en",
+      "category": "aggregate",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the average MTTR per model unit?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=[] aliases=['model_unit']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model_unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "mttr": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "model_unit"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2925,
+      "tokens": 14790
+    },
+    {
+      "id": "agg_downhour_per_group_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "total down hour per equipment group?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Group_ID",
+          "ok": false,
+          "detail": "ids=[] aliases=['equipment_group']"
+        },
+        {
+          "check": "select_agg=sum",
+          "ok": true,
+          "detail": "aggs=['sum']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "equipment_group"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "total_down_hour"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "total_down_hour": [
+                "sum"
+              ]
+            },
+            "group_by": [
+              "equipment_group"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3449,
+      "tokens": 14785
+    },
+    {
+      "id": "desc_mttr_stats_id",
+      "category": "descriptive",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berikan ringkasan statistik MTTR",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_descriptive",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2895,
+      "tokens": 14767
+    },
+    {
+      "id": "desc_pa_stats_en",
+      "category": "descriptive",
+      "lang": "en",
+      "carried_over": true,
+      "question": "give me the summary statistics for PA_Percent",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_descriptive",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2850,
+      "tokens": 14703
+    },
+    {
+      "id": "corr_mttr_pa_id",
+      "category": "correlation",
+      "lang": "id",
+      "carried_over": true,
+      "question": "apakah ada korelasi antara MTTR dan PA?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_correlation",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3665,
+      "tokens": 14850
+    },
+    {
+      "id": "corr_freq_pa_en",
+      "category": "correlation",
+      "lang": "en",
+      "carried_over": true,
+      "question": "is breakdown frequency correlated with availability?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_correlation",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_schedule",
+                "alias": "breakdown_freq_schedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_unschedule",
+                "alias": "breakdown_freq_unschedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3299,
+      "tokens": 14859
+    },
+    {
+      "id": "trend_pa_daily_id",
+      "category": "trend",
+      "lang": "id",
+      "carried_over": true,
+      "question": "bagaimana trend PA harian?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_trend",
+          "ok": true,
+          "detail": "tools=['analyze_trend', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_trend",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3472,
+      "tokens": 14861
+    },
+    {
+      "id": "trend_downhour_en",
+      "category": "trend",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show the trend of total down hours over time",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_trend",
+          "ok": true,
+          "detail": "tools=['analyze_trend', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_trend",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "total_down_hours"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3703,
+      "tokens": 14825
+    },
+    {
+      "id": "merge_worst_pa_and_mttr_id",
+      "category": "merge",
+      "lang": "id",
+      "carried_over": true,
+      "question": "model mana yang PA-nya paling buruk sekaligus MTTR-nya paling tinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=['c_model_unit', 'c_model_unit'] aliases=[]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_merge",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa",
+                "dir": "asc"
+              }
+            ],
+            "limit": 1
+          },
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_mttr",
+                "alias": "avg_mttr"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_mttr",
+                "dir": "desc"
+              }
+            ],
+            "limit": 1
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 4058,
+      "tokens": 15042
+    },
+    {
+      "id": "raw_rows_low_pa_id",
+      "category": "counter_raw_rows",
+      "lang": "id",
+      "carried_over": true,
+      "question": "tampilkan 10 record dengan PA di bawah 50",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_keyid",
+                "alias": "key_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_month_id",
+                "alias": "month_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_site_id",
+                "alias": "site_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "from_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_to_date",
+                "alias": "to_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_time_description",
+                "alias": "time_description"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model_unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "equipment_group_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unit_status",
+                "alias": "unit_status"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_pa_percent",
+                "op": "<",
+                "value": 50,
+                "value_type": "decimal"
+              }
+            ],
+            "order_by": [
+              {
+                "column_id": "pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 10
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3493,
+      "tokens": 14790
+    },
+    {
+      "id": "raw_rows_head_en",
+      "category": "counter_raw_rows",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show me the first 5 rows of the data",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_keyid",
+                "alias": "KeyId"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_month_id",
+                "alias": "Month_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_site_id",
+                "alias": "Site_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "From_Date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_to_date",
+                "alias": "To_Date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_time_description",
+                "alias": "Time_Description"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "Model_Unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "Equipment_Number"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "Equipment_Group_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unit_status",
+                "alias": "Unit_Status"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_breakdown_schedule_hour",
+                "alias": "Total_Breakdown_Schedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_breakdown_unschedule_hour",
+                "alias": "Total_Breakdown_Unschedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_adj_breakdown_schedule_hour",
+                "alias": "Total_Adj_Breakdown_Schedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_adj_breakdown_unschedule_hour",
+                "alias": "Total_Adj_Breakdown_Unschedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_mtc_hour",
+                "alias": "Total_MTC_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "Total_Down_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_schedule",
+                "alias": "Total_Frequency_Breakdown_Schedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_unschedule",
+                "alias": "Total_Frequency_Breakdown_Unschedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_maintenance",
+                "alias": "Total_Frequency_Maintenance"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_tire",
+                "alias": "Total_Frequency_Tire"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_down",
+                "alias": "Total_Frequency_Down"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_hours",
+                "alias": "Total_Hours"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_inpr_hour",
+                "alias": "Total_INPR_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_record_hm_hour",
+                "alias": "Total_Record_HM_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_hm_mtc_down_hour",
+                "alias": "Total_HM_Mtc_Down_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_plan_pa_percent",
+                "alias": "Plan_PA_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "PA_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mtbs",
+                "alias": "MTBS"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "MTTR"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_sm_percent",
+                "alias": "SM_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unschedule_sm_percent",
+                "alias": "Unschedule_SM_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_isdeleted",
+                "alias": "IsDeleted"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "Section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_week_id",
+                "alias": "Week_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_plan_pa_percent_2",
+                "alias": "Plan_PA_Percent_2"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_updated_date",
+                "alias": "Updated_Date"
+              }
+            ],
+            "limit": 5
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 4440,
+      "tokens": 15314
+    },
+    {
+      "id": "exact_model_777d_id",
+      "category": "counter_exact_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record untuk model 777D?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "777D",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2035,
+      "tokens": 14580
+    },
+    {
+      "id": "exact_section_loader_en",
+      "category": "counter_exact_filter",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records are in the OB LOADER section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_section",
+                "op": "=",
+                "value": "OB LOADER",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2133,
+      "tokens": 14570
+    },
+    {
+      "id": "infeasible_churn_id",
+      "category": "infeasible",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa churn rate pelanggan?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "facts": {
+        "tools": [],
+        "irs": [],
+        "agg_args": [],
+        "infeasible": true
+      },
+      "error": null,
+      "latency_ms": 2586,
+      "tokens": 14512
+    },
+    {
+      "id": "infeasible_profit_en",
+      "category": "infeasible",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the monthly profit margin?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "facts": {
+        "tools": [],
+        "irs": [],
+        "agg_args": [],
+        "infeasible": true
+      },
+      "error": null,
+      "latency_ms": 2206,
+      "tokens": 14551
+    }
+  ]
+}

eval/planner/results/planner_result_2026-07-23_152650.json

@@ -0,0 +1,2123 @@
+{
+  "timestamp": "2026-07-23_152650",
+  "total": 31,
+  "passed": 30,
+  "cases": [
+    {
+      "id": "count_zero_pa",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records have PA_Percent = 0?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_pa_percent",
+                "op": "=",
+                "value": 0,
+                "value_type": "decimal"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 4042,
+      "tokens": 14569
+    },
+    {
+      "id": "count_mttr_gt20_id",
+      "category": "count",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record dengan MTTR di atas 20?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_mttr",
+                "op": ">",
+                "value": 20,
+                "value_type": "decimal"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2495,
+      "tokens": 14571
+    },
+    {
+      "id": "count_section_hauler",
+      "category": "count",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many rows are in section OB HAULER?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "row_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_section",
+                "op": "=",
+                "value": "OB HAULER",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2162,
+      "tokens": 14574
+    },
+    {
+      "id": "rank_units_worst_pa_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "5 unit dengan PA terburuk?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[] resolved=['c_equipment_number']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=5",
+          "ok": true,
+          "detail": "limits=[5]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa_percent"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 5
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3918,
+      "tokens": 14745
+    },
+    {
+      "id": "rank_models_top_mttr_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": true,
+      "question": "top 3 model dengan MTTR tertinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=['c_model_unit'] aliases=[] resolved=['c_model_unit']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        },
+        {
+          "check": "limit=3",
+          "ok": true,
+          "detail": "limits=[3]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_mttr",
+                "alias": "avg_mttr"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_mttr",
+                "dir": "desc"
+              }
+            ],
+            "limit": 3
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2820,
+      "tokens": 14719
+    },
+    {
+      "id": "rank_sections_lowest_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": true,
+      "question": "which section has the lowest average PA?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": true,
+          "detail": "ids=[] aliases=['section'] resolved=['c_section']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "pa_percent": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "section"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3103,
+      "tokens": 14818
+    },
+    {
+      "id": "rank_units_most_breakdown_id",
+      "category": "ranking",
+      "lang": "id",
+      "carried_over": false,
+      "question": "unit mana yang paling sering breakdown?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[] resolved=['c_equipment_number']"
+        },
+        {
+          "check": "order_dir=desc",
+          "ok": true,
+          "detail": "dirs=['desc']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "sum",
+                "column_id": "c_total_frequency_down",
+                "alias": "total_breakdown_frequency"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "total_breakdown_frequency",
+                "dir": "desc"
+              }
+            ],
+            "limit": 1
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2902,
+      "tokens": 14761
+    },
+    {
+      "id": "rank_units_worst_pa_en",
+      "category": "ranking",
+      "lang": "en",
+      "carried_over": false,
+      "question": "list the 10 worst units by availability",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Number",
+          "ok": true,
+          "detail": "ids=['c_equipment_number'] aliases=[] resolved=['c_equipment_number']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['avg']"
+        },
+        {
+          "check": "order_dir=asc",
+          "ok": true,
+          "detail": "dirs=['asc']"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              }
+            ],
+            "group_by": [
+              "c_equipment_number"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 10
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3112,
+      "tokens": 14762
+    },
+    {
+      "id": "fuzzy_model_777_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": false,
+      "question": "berapa banyak model 777?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['=']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "777",
+                "value_type": "string"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 13617,
+      "tokens": 14582
+    },
+    {
+      "id": "fuzzy_model_hd785_id",
+      "category": "fuzzy_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak unit HD785?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['=']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "unit_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "HD785",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3472,
+      "tokens": 14660
+    },
+    {
+      "id": "fuzzy_model_ex_en",
+      "category": "fuzzy_filter",
+      "lang": "en",
+      "carried_over": false,
+      "question": "how many EX excavator units are there?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_filter_op=in",
+          "ok": true,
+          "detail": "ops=['like']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "unit_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "like",
+                "value": "EX%",
+                "value_type": "string"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 5809,
+      "tokens": 29353
+    },
+    {
+      "id": "agg_pa_per_section_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa rata-rata PA per section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Section",
+          "ok": true,
+          "detail": "ids=[] aliases=['section'] resolved=['c_section']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "pa_percent": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "section"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3942,
+      "tokens": 14785
+    },
+    {
+      "id": "agg_mttr_per_model_en",
+      "category": "aggregate",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the average MTTR per model unit?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=[] aliases=['model_unit'] resolved=['c_model_unit']"
+        },
+        {
+          "check": "select_agg=avg",
+          "ok": true,
+          "detail": "aggs=['mean']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model_unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "mttr": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "model_unit"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3648,
+      "tokens": 14781
+    },
+    {
+      "id": "agg_downhour_per_group_id",
+      "category": "aggregate",
+      "lang": "id",
+      "carried_over": true,
+      "question": "total down hour per equipment group?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Equipment_Group_ID",
+          "ok": true,
+          "detail": "ids=[] aliases=['equipment_group'] resolved=['c_equipment_group_id']"
+        },
+        {
+          "check": "select_agg=sum",
+          "ok": true,
+          "detail": "aggs=['sum']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "equipment_group"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "down_hour"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "down_hour": [
+                "sum"
+              ]
+            },
+            "group_by": [
+              "equipment_group"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 5057,
+      "tokens": 14830
+    },
+    {
+      "id": "desc_mttr_stats_id",
+      "category": "descriptive",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berikan ringkasan statistik MTTR",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_descriptive",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3173,
+      "tokens": 14707
+    },
+    {
+      "id": "desc_pa_stats_en",
+      "category": "descriptive",
+      "lang": "en",
+      "carried_over": true,
+      "question": "give me the summary statistics for PA_Percent",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_descriptive",
+          "ok": true,
+          "detail": "tools=['analyze_descriptive', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_descriptive",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "PA_Percent"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3135,
+      "tokens": 14717
+    },
+    {
+      "id": "corr_mttr_pa_id",
+      "category": "correlation",
+      "lang": "id",
+      "carried_over": true,
+      "question": "apakah ada korelasi antara MTTR dan PA?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_correlation",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "mttr"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3920,
+      "tokens": 14798
+    },
+    {
+      "id": "corr_freq_pa_en",
+      "category": "correlation",
+      "lang": "en",
+      "carried_over": true,
+      "question": "is breakdown frequency correlated with availability?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_correlation",
+          "ok": true,
+          "detail": "tools=['analyze_correlation', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_correlation",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_down",
+                "alias": "breakdown_frequency"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "availability"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3774,
+      "tokens": 14845
+    },
+    {
+      "id": "trend_pa_daily_id",
+      "category": "trend",
+      "lang": "id",
+      "carried_over": true,
+      "question": "bagaimana trend PA harian?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_trend",
+          "ok": true,
+          "detail": "tools=['analyze_trend', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_trend",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 4485,
+      "tokens": 14852
+    },
+    {
+      "id": "trend_downhour_en",
+      "category": "trend",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show the trend of total down hours over time",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=analyze_trend",
+          "ok": true,
+          "detail": "tools=['analyze_trend', 'check_data', 'retrieve_data']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_trend",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "from_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "total_down_hour"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3686,
+      "tokens": 14832
+    },
+    {
+      "id": "merge_worst_pa_and_mttr_id",
+      "category": "merge",
+      "lang": "id",
+      "carried_over": true,
+      "question": "model mana yang PA-nya paling buruk sekaligus MTTR-nya paling tinggi?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "group_by_col=Model_Unit",
+          "ok": true,
+          "detail": "ids=['c_model_unit', 'c_model_unit'] aliases=[] resolved=['c_model_unit', 'c_model_unit']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_merge",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_from_date",
+                "op": "between",
+                "value": [
+                  "2026-04-01",
+                  "2026-04-30"
+                ],
+                "value_type": "date"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_pa",
+                "dir": "asc"
+              }
+            ],
+            "limit": 1
+          },
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_mttr",
+                "alias": "avg_mttr"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_from_date",
+                "op": "between",
+                "value": [
+                  "2026-04-01",
+                  "2026-04-30"
+                ],
+                "value_type": "date"
+              }
+            ],
+            "group_by": [
+              "c_model_unit"
+            ],
+            "order_by": [
+              {
+                "column_id": "avg_mttr",
+                "dir": "desc"
+              }
+            ],
+            "limit": 1
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 5546,
+      "tokens": 15165
+    },
+    {
+      "id": "raw_rows_low_pa_id",
+      "category": "counter_raw_rows",
+      "lang": "id",
+      "carried_over": true,
+      "question": "tampilkan 10 record dengan PA di bawah 50",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        },
+        {
+          "check": "limit=10",
+          "ok": true,
+          "detail": "limits=[10]"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_keyid",
+                "alias": "key_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_month_id",
+                "alias": "month_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_site_id",
+                "alias": "site_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "from_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_to_date",
+                "alias": "to_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_time_description",
+                "alias": "time_description"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "model_unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "equipment_number"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "equipment_group_id"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unit_status",
+                "alias": "unit_status"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_pa_percent",
+                "op": "<",
+                "value": 50,
+                "value_type": "decimal"
+              }
+            ],
+            "order_by": [
+              {
+                "column_id": "pa_percent",
+                "dir": "asc"
+              }
+            ],
+            "limit": 10
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3525,
+      "tokens": 14840
+    },
+    {
+      "id": "raw_rows_head_en",
+      "category": "counter_raw_rows",
+      "lang": "en",
+      "carried_over": true,
+      "question": "show me the first 5 rows of the data",
+      "passed": true,
+      "checks": [
+        {
+          "check": "no_group_by",
+          "ok": true,
+          "detail": "grouped=False"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_keyid",
+                "alias": "KeyId"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_month_id",
+                "alias": "Month_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_site_id",
+                "alias": "Site_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "From_Date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_to_date",
+                "alias": "To_Date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_time_description",
+                "alias": "Time_Description"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_model_unit",
+                "alias": "Model_Unit"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_number",
+                "alias": "Equipment_Number"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_equipment_group_id",
+                "alias": "Equipment_Group_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unit_status",
+                "alias": "Unit_Status"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_breakdown_schedule_hour",
+                "alias": "Total_Breakdown_Schedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_breakdown_unschedule_hour",
+                "alias": "Total_Breakdown_Unschedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_adj_breakdown_schedule_hour",
+                "alias": "Total_Adj_Breakdown_Schedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_adj_breakdown_unschedule_hour",
+                "alias": "Total_Adj_Breakdown_Unschedule_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_mtc_hour",
+                "alias": "Total_MTC_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_down_hour",
+                "alias": "Total_Down_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_schedule",
+                "alias": "Total_Frequency_Breakdown_Schedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_breakdown_unschedule",
+                "alias": "Total_Frequency_Breakdown_Unschedule"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_maintenance",
+                "alias": "Total_Frequency_Maintenance"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_tire",
+                "alias": "Total_Frequency_Tire"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_frequency_down",
+                "alias": "Total_Frequency_Down"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_hours",
+                "alias": "Total_Hours"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_inpr_hour",
+                "alias": "Total_INPR_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_record_hm_hour",
+                "alias": "Total_Record_HM_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_total_hm_mtc_down_hour",
+                "alias": "Total_HM_Mtc_Down_Hour"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_plan_pa_percent",
+                "alias": "Plan_PA_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "PA_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mtbs",
+                "alias": "MTBS"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_mttr",
+                "alias": "MTTR"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_sm_percent",
+                "alias": "SM_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_unschedule_sm_percent",
+                "alias": "Unschedule_SM_Percent"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_isdeleted",
+                "alias": "IsDeleted"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "Section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_week_id",
+                "alias": "Week_ID"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_plan_pa_percent_2",
+                "alias": "Plan_PA_Percent_2"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_updated_date",
+                "alias": "Updated_Date"
+              }
+            ],
+            "limit": 5
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 6662,
+      "tokens": 15316
+    },
+    {
+      "id": "exact_model_777d_id",
+      "category": "counter_exact_filter",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa banyak record untuk model 777D?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_model_unit",
+                "op": "=",
+                "value": "777D",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2422,
+      "tokens": 14573
+    },
+    {
+      "id": "exact_section_loader_en",
+      "category": "counter_exact_filter",
+      "lang": "en",
+      "carried_over": true,
+      "question": "how many records are in the OB LOADER section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "select_agg=count",
+          "ok": true,
+          "detail": "aggs=['count']"
+        },
+        {
+          "check": "has_filter",
+          "ok": true,
+          "detail": "filter_present=True"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "agg",
+                "fn": "count",
+                "alias": "record_count"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_section",
+                "op": "=",
+                "value": "OB LOADER",
+                "value_type": "string"
+              }
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 2378,
+      "tokens": 14570
+    },
+    {
+      "id": "disambig_trend_pa_id",
+      "category": "column_disambiguation",
+      "lang": "id",
+      "carried_over": false,
+      "question": "bagaimana trend PA?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "selects_col=PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_from_date', 'c_pa_percent']"
+        },
+        {
+          "check": "not_selects_col=Plan_PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_from_date', 'c_pa_percent']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_trend",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3527,
+      "tokens": 14860
+    },
+    {
+      "id": "disambig_avg_pa_en",
+      "category": "column_disambiguation",
+      "lang": "en",
+      "carried_over": false,
+      "question": "what is the average PA per section?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "selects_col=PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_section', 'c_pa_percent']"
+        },
+        {
+          "check": "not_selects_col=Plan_PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_section', 'c_pa_percent']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "analyze_aggregate",
+          "check_data",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [
+          {
+            "data": "${t2}",
+            "aggregations": {
+              "pa_percent": [
+                "mean"
+              ]
+            },
+            "group_by": [
+              "section"
+            ]
+          }
+        ],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3972,
+      "tokens": 14810
+    },
+    {
+      "id": "chart_trend_pa_id",
+      "category": "chart",
+      "lang": "id",
+      "carried_over": false,
+      "question": "bagaimana visualisasi trend PA?",
+      "passed": false,
+      "checks": [
+        {
+          "check": "has_tool=render_chart",
+          "ok": true,
+          "detail": "tools=['check_data', 'render_chart', 'retrieve_data']"
+        },
+        {
+          "check": "any_tool=['analyze_aggregate', 'analyze_trend']",
+          "ok": false,
+          "detail": "tools=['check_data', 'render_chart', 'retrieve_data']"
+        },
+        {
+          "check": "selects_col=PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_from_date', 'c_pa_percent']"
+        },
+        {
+          "check": "not_selects_col=Plan_PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_from_date', 'c_pa_percent']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "render_chart",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_from_date",
+                "alias": "from_date"
+              },
+              {
+                "kind": "column",
+                "column_id": "c_pa_percent",
+                "alias": "pa_percent"
+              }
+            ],
+            "filters": [
+              {
+                "column_id": "c_month_id",
+                "op": "=",
+                "value": 202604,
+                "value_type": "int"
+              },
+              {
+                "column_id": "c_isdeleted",
+                "op": "=",
+                "value": 0,
+                "value_type": "int"
+              }
+            ],
+            "limit": 10000
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3641,
+      "tokens": 14859
+    },
+    {
+      "id": "chart_avg_pa_by_section_en",
+      "category": "chart",
+      "lang": "en",
+      "carried_over": false,
+      "question": "show me a bar chart of average PA per section",
+      "passed": true,
+      "checks": [
+        {
+          "check": "has_tool=render_chart",
+          "ok": true,
+          "detail": "tools=['check_data', 'render_chart', 'retrieve_data']"
+        },
+        {
+          "check": "group_by",
+          "ok": true,
+          "detail": "grouped=True"
+        },
+        {
+          "check": "selects_col=PA_Percent",
+          "ok": true,
+          "detail": "selected=['c_section', 'c_pa_percent']"
+        }
+      ],
+      "facts": {
+        "tools": [
+          "check_data",
+          "render_chart",
+          "retrieve_data"
+        ],
+        "irs": [
+          {
+            "source_id": "src_pa",
+            "table_id": "t_pa",
+            "select": [
+              {
+                "kind": "column",
+                "column_id": "c_section",
+                "alias": "section"
+              },
+              {
+                "kind": "agg",
+                "fn": "avg",
+                "column_id": "c_pa_percent",
+                "alias": "avg_pa"
+              }
+            ],
+            "group_by": [
+              "c_section"
+            ]
+          }
+        ],
+        "agg_args": [],
+        "infeasible": false
+      },
+      "error": null,
+      "latency_ms": 3166,
+      "tokens": 14769
+    },
+    {
+      "id": "infeasible_churn_id",
+      "category": "infeasible",
+      "lang": "id",
+      "carried_over": true,
+      "question": "berapa churn rate pelanggan?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "facts": {
+        "tools": [],
+        "irs": [],
+        "agg_args": [],
+        "infeasible": true
+      },
+      "error": null,
+      "latency_ms": 2055,
+      "tokens": 14486
+    },
+    {
+      "id": "infeasible_profit_en",
+      "category": "infeasible",
+      "lang": "en",
+      "carried_over": true,
+      "question": "what is the monthly profit margin?",
+      "passed": true,
+      "checks": [
+        {
+          "check": "infeasible",
+          "ok": true,
+          "detail": "infeasible=True"
+        }
+      ],
+      "facts": {
+        "tools": [],
+        "irs": [],
+        "agg_args": [],
+        "infeasible": true
+      },
+      "error": null,
+      "latency_ms": 2072,
+      "tokens": 14501
+    }
+  ]
+}

eval/readiness/results/readiness_result_2026-06-23_094622.json

@@ -0,0 +1,284 @@
+{
+  "run": {
+    "timestamp": "2026-06-23T09:46:22",
+    "dataset": "readiness_dataset.json",
+    "target": "src/agents/report/readiness.is_report_ready",
+    "total": 17,
+    "passed": 17,
+    "accuracy": 1.0,
+    "runtime_avg_ms": 0.0
+  },
+  "alignment_gap": {
+    "count": 2,
+    "ids": [
+      "align_01",
+      "align_02"
+    ]
+  },
+  "by_group": {
+    "floor": {
+      "n": 8,
+      "passed": 8,
+      "accuracy": 1.0
+    },
+    "delta": {
+      "n": 5,
+      "passed": 5,
+      "accuracy": 1.0
+    },
+    "edge": {
+      "n": 1,
+      "passed": 1,
+      "accuracy": 1.0
+    },
+    "alignment": {
+      "n": 3,
+      "passed": 3,
+      "accuracy": 1.0
+    }
+  },
+  "cases": [
+    {
+      "id": "floor_01",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a validated problem statement",
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "a validated problem statement",
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_02",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a validated problem statement"
+      ],
+      "got_missing": [
+        "a validated problem statement"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_03",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_04",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_05",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_06",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_07",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_08",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_01",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_02",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_03",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_04",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_05",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "edge_01",
+      "group": "edge",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_01",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_02",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_03",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    }
+  ]
+}

eval/readiness/results/readiness_result_2026-07-02_083335.json

@@ -0,0 +1,250 @@
+{
+  "run": {
+    "timestamp": "2026-07-02T08:33:35",
+    "dataset": "readiness_dataset.json",
+    "target": "src/agents/report/readiness.is_report_ready",
+    "total": 15,
+    "passed": 15,
+    "accuracy": 1.0,
+    "runtime_avg_ms": 0.0
+  },
+  "alignment_gap": {
+    "count": 2,
+    "ids": [
+      "align_01",
+      "align_02"
+    ]
+  },
+  "by_group": {
+    "floor": {
+      "n": 6,
+      "passed": 6,
+      "accuracy": 1.0
+    },
+    "delta": {
+      "n": 5,
+      "passed": 5,
+      "accuracy": 1.0
+    },
+    "edge": {
+      "n": 1,
+      "passed": 1,
+      "accuracy": 1.0
+    },
+    "alignment": {
+      "n": 3,
+      "passed": 3,
+      "accuracy": 1.0
+    }
+  },
+  "cases": [
+    {
+      "id": "floor_01",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_02",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_03",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_04",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_05",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_06",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_01",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_02",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_03",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_04",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_05",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "edge_01",
+      "group": "edge",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_01",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_02",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_03",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    }
+  ]
+}

eval/readiness/results/readiness_result_2026-07-02_093621.json

@@ -0,0 +1,250 @@
+{
+  "run": {
+    "timestamp": "2026-07-02T09:36:21",
+    "dataset": "readiness_dataset.json",
+    "target": "src/agents/report/readiness.is_report_ready",
+    "total": 15,
+    "passed": 15,
+    "accuracy": 1.0,
+    "runtime_avg_ms": 0.0
+  },
+  "alignment_gap": {
+    "count": 2,
+    "ids": [
+      "align_01",
+      "align_02"
+    ]
+  },
+  "by_group": {
+    "floor": {
+      "n": 6,
+      "passed": 6,
+      "accuracy": 1.0
+    },
+    "delta": {
+      "n": 5,
+      "passed": 5,
+      "accuracy": 1.0
+    },
+    "edge": {
+      "n": 1,
+      "passed": 1,
+      "accuracy": 1.0
+    },
+    "alignment": {
+      "n": 3,
+      "passed": 3,
+      "accuracy": 1.0
+    }
+  },
+  "cases": [
+    {
+      "id": "floor_01",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_02",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_03",
+      "group": "floor",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_04",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_05",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "floor_06",
+      "group": "floor",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_01",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_02",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_03",
+      "group": "delta",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_04",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "delta_05",
+      "group": "delta",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "a new analysis since the last report"
+      ],
+      "got_missing": [
+        "a new analysis since the last report"
+      ],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "edge_01",
+      "group": "edge",
+      "expected_ready": false,
+      "got_ready": false,
+      "expected_missing": [
+        "at least one completed analysis"
+      ],
+      "got_missing": [
+        "at least one completed analysis"
+      ],
+      "correct": true,
+      "aligned": false,
+      "gap": false,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_01",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_02",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": false,
+      "gap": true,
+      "latency_ms": 0.0
+    },
+    {
+      "id": "align_03",
+      "group": "alignment",
+      "expected_ready": true,
+      "got_ready": true,
+      "expected_missing": [],
+      "got_missing": [],
+      "correct": true,
+      "aligned": true,
+      "gap": false,
+      "latency_ms": 0.0
+    }
+  ]
+}

knowledge_pipeline_context.md

@@ -0,0 +1,299 @@
+# Knowledge Pipeline — Context for the Data Eyond Python Repo (19 Aug 2026)
+
+> **Who this is for:** a Claude Code session working in `Agentic-Service-Data-Eyond-Catalog`
+> (the Python agentic service). This doc carries *context and intent* only — the why, the
+> ownership split, the shape of the pipeline, and the decisions already settled. It does not
+> prescribe folder layout, module names, or endpoint signatures; work those out against
+> `CLAUDE.md`, `REPO_STATUS.md` and the existing subsystem patterns in the repo.
+>
+---
+
+## 1. What we are doing and why now
+
+Data Eyond is an AI data-analyst platform. The strategic thesis from the 30 July exec review
+is that **domain knowledge is the differentiator, not architecture** — orchestrators and
+connectors are replicable in a quarter, model capability is rented, only encoded domain
+knowledge compounds. The end goal is an **MCP product** exposing modular domain knowledge
+that EMA and later clients connect to.
+
+That knowledge currently comes from experts typing it in. **Mas Beta** (plant & maintenance)
+is the single validation bottleneck — nothing becomes an artifact without him. The knowledge
+pipeline exists to change the expert's job from *authoring* to *reviewing*: the pipeline reads
+the client's own documents (standards, SOPs, handbooks) and proposes candidate knowledge
+entries; the expert approves, edits or rejects them.
+
+**Why it lands in this repo now.** We want to integrate the pipeline into Data Eyond to test
+it end-to-end on real documents, in the place where the resulting knowledge will actually be
+consumed. This is a **test integration**, not a productionisation — treat it as a new, gated
+subsystem alongside the existing document/catalog pipelines, not as a rewrite of them.
+
+---
+
+## 2. Ownership — two items, two people, one seam
+
+The pipeline splits cleanly in two, and the split is deliberate: **they are separate,
+independently runnable, independently swappable pipelines.**
+
+| Half | Owner | Scope |
+|---|---|---|
+| **① Parsing (MinerU)** | **Sofhia Az-Zahra** | PDF/DOCX in → structured, typed extracted content out. MinerU replaces the current text+table-only extraction: it adds formula, image, chart and diagram detection, and emits Markdown or JSON with type labels. Presented to the execs 13 Aug as the **prerequisite for Scenario 2**; a written cost justification for MinerU is owed for the budget proposal. |
+| **② Knowledge extraction** | **Rifqi (me)** | Parsed content in → validated candidate knowledge entries out. Term filtering, clustering, evidence ranking, schema-guided LLM attribute fill, span validation, diff against the active glossary version, and the frequency-sorted review queue. |
+
+**The seam.** The two halves must meet only at a **persisted, versioned parsed-document
+artifact** — extraction consumes that artifact, never a raw file, never the parser's API.
+This is the single most important structural constraint in the whole plan, because:
+
+- MinerU can be swapped (or fall back to the existing Tesseract / Azure Document Intelligence
+  paths) without extraction knowing;
+- extraction can be re-run cheaply on an already-parsed corpus when a prompt or schema changes
+  — and it will be, often, because prompt iteration is the main development loop here;
+- Sofhia and I can work and test independently, against a fixture artifact, without blocking
+  each other;
+- parsing is the slow/expensive-to-rerun half; extraction is the fast-to-iterate half. Keeping
+  the boundary hard means we never re-parse to fix an extraction bug.
+
+Anything that makes the two halves import each other's internals, or that makes extraction
+take a file path, breaks the point of the split.
+
+---
+
+## 3. The pipeline
+
+Six stages. The cost story matters as much as the flow: almost everything is free CPU/regex
+work, and the LLM appears in exactly one stage.
+
+```mermaid
+flowchart TD
+    A["Admin / expert triggers ingest<br/><i>not continuous, not per-user</i>"] --> B
+
+    subgraph P["① PARSING — Sofhia"]
+        B["MinerU<br/>text · tables · formulas · figures · charts"] --> C["Section-aware chunking<br/>split on numbered headings 2.1.1, 2.1.2 …"]
+    end
+
+    C --> SEAM[("Parsed-document artifact<br/>versioned · persisted<br/><b>THE SEAM</b>")]
+    SEAM --> D
+
+    subgraph F["② TERM FILTER — CPU / regex, free"]
+        D["GLiNER span filter on ALL chunks → term mentions"] --> H
+        E["Discourse-cue regex → rule-of-thumb candidates"] --> H
+        G["Legend-block regex → formula vars + abbreviation dictionary"] --> H
+        G2["Section pass → summary units"] --> H
+        H["Normalise + cluster mentions<br/>PA · P.A. · Physical Availability → 1 cluster"] --> I["Evidence ranking<br/>cue · heading · legend · formula · formatting · first-occurrence"]
+        I --> J["Top-3 evidence chunks per cluster"]
+    end
+
+    SEAM --> E
+    SEAM --> G
+    SEAM --> G2
+
+    J --> K
+
+    subgraph X["③ KNOWLEDGE EXTRACTION — the only paid stage"]
+        K["1 LLM call per TERM CLUSTER<br/>(not per mention, not per chunk)"]
+        L["1 call per cue chunk · 1 per unique formula · 1 per document (summary)"]
+    end
+
+    K --> M
+    L --> M
+
+    subgraph V["④ VALIDATION — deterministic"]
+        M["Verbatim-span check → reject field if span not locatable"] --> N["Null definition → escalate to next 3 evidence chunks<br/>max 2 rounds → else flag 'term found, no definition located'"]
+        N --> O["Conflicting definitions → definition_conflict + variants[]"]
+    end
+
+    O --> Q["⑤ DIFF vs active glossary version<br/>duplicate (skip) · new · conflicting"]
+    Q --> R
+
+    subgraph RV["⑥ REVIEW — human"]
+        R["Queue sorted by mention frequency<br/>PA 250 → UA 180 → … → long tail"] --> S["Expert (Mas Beta) approves / edits / rejects"]
+        S --> T["Glossary · Interpretation Pack · Brief Context · Formula"]
+        T --> U["Versioning + approval audit trail"]
+    end
+
+    U --> W["MCP server — the consumption surface"]
+
+    style P fill:#e8f4ff,stroke:#4a90d9
+    style F fill:#e6f7e6,stroke:#4caf50
+    style X fill:#dbe9ff,stroke:#2f6fd0
+    style V fill:#f0e6ff,stroke:#8e5fd0
+    style RV fill:#fff0e0,stroke:#e08b3c
+    style SEAM fill:#fffbe0,stroke:#c9a227,stroke-width:3px
+```
+
+Colour convention, carried from the diagrams shown to the team:
+**green = free** (CPU/regex — most of the pipeline) · **blue = LLM** (the only paid part) ·
+**purple = deterministic code** · **orange = human**.
+
+### The five-box version (how this is explained to execs)
+
+```mermaid
+flowchart LR
+    A["PARSE<br/>MinerU"] --> B["FILTER<br/>free, CPU"] --> C["EXTRACT<br/>LLM, per term cluster"] --> D["VALIDATE + DIFF<br/>deterministic"] --> E["EXPERT REVIEW<br/>frequency-sorted queue"]
+    style A fill:#e8f4ff,stroke:#4a90d9
+    style B fill:#e6f7e6,stroke:#4caf50
+    style C fill:#dbe9ff,stroke:#2f6fd0
+    style D fill:#f0e6ff,stroke:#8e5fd0
+    style E fill:#fff0e0,stroke:#e08b3c
+```
+
+### Ownership view
+
+```mermaid
+flowchart LR
+    subgraph S["Sofhia"]
+        P["Parsing pipeline<br/>MinerU + backends"]
+    end
+    subgraph R["Rifqi"]
+        K["Extraction pipeline<br/>filter → LLM → validate → diff → queue"]
+    end
+    subgraph H["Mas Har / Mas Beta"]
+        C["Curation UI + expert review<br/>versioning, approval"]
+    end
+    P -->|"versioned parsed artifact<br/>(the only contract)"| K
+    K -->|"candidate entries + provenance"| C
+    C -->|"active versions"| M["MCP server"]
+```
+
+---
+
+## 4. Decisions already settled — do not reopen these
+
+1. **The LLM call unit is the term cluster** — not the chunk, not the mention. Per-chunk works
+   for a 9-page document and breaks at 1,000+ pages: "PA" mentioned 250× would produce 250
+   near-identical candidates. Clustering first cuts expert review burden ~6.2× (3,125 raw
+   extractions → ~500 term entries on a 1,000-page document). The token saving is trivial
+   (~$0.45); **the review-burden reduction is the justification.**
+2. **Conflict detection depends on the clustering.** Two contradictory definitions of the same
+   term can only be spotted because all evidence for that term arrives in one call. Two
+   separate calls would never meet, and the conflict would surface as two silently-accepted
+   entries.
+3. **No standalone relevance gate.** A binary relevance classifier before the term filter is a
+   second imperfect classifier whose false negatives drop content *before* extraction sees it —
+   and dropped content never reaches expert review to be caught. GLiNER is cheap enough (CPU,
+   ~400MB) to run unconditionally on every chunk; "zero candidate spans" becomes the relevance
+   signal as a byproduct of looking, rather than a judgment made before looking.
+4. **Evidence ranking is allowed even though it is also a filter** — because no term is
+   dropped (only which passages feed the call is narrowed), all mentions stay in provenance so
+   a bad top-3 is visible, and the escalation loop self-corrects. The rejected relevance gate
+   had none of those three properties. This distinction is the reasoning to preserve if the
+   ranking is ever revisited.
+5. **`provenance.span` is mandatory and verbatim.** A field whose span cannot be located in the
+   source is rejected. This is the primary anti-hallucination control and the thing that makes
+   expert review *finishable* — the reviewer checks a quote against a page, not a claim against
+   their memory.
+6. **All content fields are Optional in the schema.** The model must be able to answer null; a
+   required field is a fabrication invitation for smaller models. `subdomain_tags` is an enum —
+   classification, not generation.
+7. **Frequency-sorted review queue.** Directly targets the Mas Beta bottleneck: validate the
+   ~40 terms carrying the document's meaning first, let the long tail wait.
+8. **Model tier is per branch.** Glossary and Formula are extraction tasks well-guarded by span
+   verification → a small/cheap tier is fine. Rule-of-thumb and Summary are reasoning/generation
+   tasks where a small model's failure mode is least detectable (a plausible summary cannot be
+   span-checked) → larger tier. Test before committing the small tier on the rule-of-thumb
+   branch.
+9. **GLiNER-hybrid stays the glossary-branch default.** Pure NER can only fill 2 of 6 schema
+   fields (definition, formula and interpretation are not spans — they are assembled from text
+   that may be pages apart). Pure small-LLM shows recall 0.93–1.00 at precision 0.15–0.43;
+   perfect recall is the failure signature of a model fabricating a value for every field.
+10. **The pipeline records the literal source wording.** It recorded "Physical of Availability"
+    from the BUMA standard rather than the textbook's "Physical Availability" — surfacing the
+    discrepancy to the expert instead of silently normalising it. Keep that behaviour.
+
+---
+
+## 5. Deployment reality — this shapes every engineering trade-off
+
+The pipeline runs **once at setup (bulk corpus ingest)** and **occasionally when an admin or
+expert adds a file**. It is **not continuous** and **not per-user** — normal engineers cannot
+add data. Consequences, in the order they matter:
+
+- **Extraction quality and expert-review efficiency dominate.** A missed term has no
+  self-correcting mechanism, because engineers can't add data to fill a gap later.
+- **Maintainability matters more than it looks.** A glue chain invoked every few months is
+  exactly the thing nobody remembers how to debug. Prefer boring, inspectable stages over
+  clever ones.
+- **API cost is negligible and latency is irrelevant.** ~$8–12 for a 6,000-page corpus;
+  an incremental single 9-page file is ~1.5 US cents. Do not spend engineering time optimising
+  tokens — engineering time on the review experience is worth more than any token saving
+  available here.
+- **No GPU required.** GLiNER is a CPU encoder model (~1.6 seq/s Python, ~6.7 seq/s via the
+  Rust port on an i9 8-core). ~15,000 chunks ≈ 2.6h Python / ~40min Rust for a job that runs a
+  few times a year. The LLM half is an API call. (Runtime as reported to the team on 13 Aug:
+  ~11 s/page GLiNER, ~26 s/page LLM — with the caveat that **cost scales on term count, not
+  page count.**)
+- **Two ingest modes to build:** *bulk setup ingest* (corpus-scale — the only place
+  corpus-frequency statistics are usable as a candidate-term booster) and *incremental
+  single-file add* (no corpus context; diff against the active glossary version).
+- **The summary branch is the quiet cost risk** — only ~300 calls but a quarter of all input
+  tokens, because summarisation can't be filtered; it needs whole documents.
+
+---
+
+## 6. Where the output goes
+
+Approved candidates become the project's **four artifacts** (see the `context` doc for the
+full architecture):
+
+1. **Domain knowledge** — system-prompt material, capped ~1500 tokens: operating and equipment
+   hierarchy, time convention (WITA), seasonality, most-important glossary terms, high-level
+   business process, and agent limitations.
+2. **Data dictionary** — from DB column profiling; used for query generation. Pak Ricky
+   connected the extraction work directly to **automating this**.
+3. **Interpretation pack** — expert-curated interpretation logic, action rules, benchmark
+   history, tied to a use case ("PA missed" reads differently from "PA missed but MTBS
+   achieved"). Revived on 13 Aug after earlier removal for being too normative — reintroduce
+   carefully.
+4. **Skill registry** — query rules and expert-approved formulas. Approval is mandatory because
+   the same term computes differently per company (MTTR at BUMA = breakdown duration ÷
+   breakdown frequency; MTTR in IT = mean time to resolve).
+
+All four carry **versioning plus an approval audit trail**, and the expert chooses which
+version is active. Delivery format is **MCP** — skill registry → MCP skills; data dictionary →
+MCP on demand; interpretation pack → attached to the skill; domain knowledge → system prompt
+but preferably served via MCP so owners can change it without redeploying the engine.
+
+---
+
+## 7. Grounding material
+
+Two real sample documents anchor every design claim above, and should anchor test fixtures too:
+
+- **BUMA STD/2026/006/MNO Rev.0.0** — Production Parameter & ECA. 9 pages, majority Bahasa
+  Indonesia. Carries: PA (glossary, p.4), Other Activity + the controllability hierarchy
+  (rule-of-thumb, pp.8–9), the MOHH/Qty/PA/UA/Pty legend block (formula branch, p.2).
+- **Open Pit Mine Planning & Design** (textbook excerpt) — carries NSR as a **cross-page
+  definition**: intro p.92, formula p.93, interpretive remark p.94 (§2.3.5). This is the case
+  that justifies section-aware chunking over fixed token windows — the whole definition must
+  stay inside one semantic unit.
+
+**Known open risk:** GLiNER multi-v2.1's Indonesian performance is unverified, and the BUMA
+standard is majority Bahasa Indonesia. This is the standing "Indonesian degradation" question,
+now with a concrete test case. Verify before trusting recall numbers.
+
+**Flagged as scope creep, proposed not decided:** ensembling GLiNER + LangExtract on every
+ingest. Cheap in API terms, but it adds a reconciliation/dedup step between two candidate sets
+that may disagree — new engineering surface for uncertain gain.
+
+---
+
+## 8. Integration guidance for this repo
+
+Deliberately shape-agnostic — decide the specifics against `CLAUDE.md` and the existing
+subsystem patterns, not against this doc.
+
+- **Both halves are new subsystems, not modifications** to the existing document/catalog
+  pipelines. The current unstructured path (Tesseract OCR → chunk → pgvector) stays as it is;
+  the MinerU path is additive and, at least initially, gated.
+- **Follow the repo's existing multi-stage precedent.** The query subsystem (IR → validator →
+  compiler → executor, orchestrated from a service facade, triggered by thin entry points) is
+  the closest structural analogue to what both halves need.
+- **Respect the hard boundaries.** Go owns the dedorch schema — any new table needs a
+  Harry-ready DDL handoff, never DDL executed from Python. Any new endpoint on the live surface
+  needs a contract-doc entry, and an admin ingest surface is a different risk class than the
+  read-only chat surface — raise the authentication question rather than assuming.
+- **Build the offline path first.** Because this runs a few times a year and is triggered by an
+  admin, a script-driven run over a fixture document is the honest first milestone; HTTP
+  endpoints are a convenience layer over it, not the other way round.
+- **Fixtures over live parsing in tests.** A committed parsed-artifact fixture from one of the
+  two sample documents lets the extraction half be tested without MinerU installed at all —
+  which is the seam doing its job.

pyproject.toml

@@ -90,6 +90,9 @@ dependencies = [
     "pytesseract>=0.3.13",
     "pypdf2>=3.0.1",
     "pyarrow>=24.0.0",
+    "rapidfuzz>=3.14.5",
+    "gliner>=0.2.13",
+    "torch>=2.6",
 ]
 
 [project.optional-dependencies]
@@ -103,6 +106,18 @@ dev = [
     "pre-commit==4.0.1",
 ]
 
+# Document parsing for the knowledge pipeline (src/knowledge_parsing/).
+#
+# Deliberately an EXTRA, not a main dependency: MinerU pulls torch and its model
+# tail (GBs), and the agent service never parses documents at request time —
+# parsing is an offline, admin-triggered batch job. Keeping it here means the
+# deployed Space does not build or ship any of it.
+#
+#     pip install -e ".[parsing]"
+knowledge-parsing = [
+    "mineru==3.4.4",
+]
+
 [tool.hatch.build.targets.wheel]
 packages = ["src/agent_service"]
 
@@ -125,6 +140,13 @@ ignore = [
 "tests/**" = ["S101", "S105", "S106", "S608"]
 # T201: eval/ scripts are CLIs — print() is their intended output channel.
 "eval/**" = ["T201"]
+# Same rule, same reason: this is an operator-run CLI, and its printed output IS
+# the deliverable.
+"src/knowledge_extraction/cli.py" = ["T201"]
+# Same again for the knowledge-parsing CLI entry points. Scoped to the two files
+# rather than the package, so the library modules stay print-free.
+"src/knowledge_parsing/run.py" = ["T201"]
+"src/knowledge_parsing/report.py" = ["T201"]
 
 [tool.mypy]
 python_version = "3.12"

src/config/settings.py

@@ -47,6 +47,13 @@ class Settings(BaseSettings):
     azureai_deployment_name_54m: str = Field(alias="azureai__deployment__name__54m", default="")
     azureai_api_version_54m: str = Field(alias="azureai__api__version__54m", default="")
 
+    # Azure OpenAI - GPT-5.4-nano (knowledge-extraction pipeline; all four
+    # branches route here — see KNOWLEDGE_PIPELINE_TODO.md D3)
+    azureai_api_key_54n: str = Field(alias="azureai__api_key__54n", default="")
+    azureai_endpoint_url_54n: str = Field(alias="azureai__endpoint__url__54n", default="")
+    azureai_deployment_name_54n: str = Field(alias="azureai__deployment__name__54n", default="")
+    azureai_api_version_54n: str = Field(alias="azureai__api__version__54n", default="")
+
     # Azure OpenAI - Embeddings
     azureai_api_key_embedding: str = Field(alias="azureai__api_key__embedding", default="")
     azureai_endpoint_url_embedding: str = Field(alias="azureai__endpoint__url__embedding", default="")

src/knowledge_extraction/__init__.py

@@ -0,0 +1,62 @@
+"""Knowledge-extraction pipeline: parsed document -> candidate knowledge entries.
+
+Distinct from `src/knowledge/`, which is the existing OCR -> chunk -> pgvector
+ingestion path for unstructured RAG. This package does not touch it.
+
+Stage order, and which stages cost money:
+    adapter   seam artifact -> internal Chunk          free
+    filters   cue / legend / span NER -> mentions      free (CPU)
+    cluster   normalise + cluster mentions             free
+    rank      evidence scoring, top-K selection        free
+    extract   one LLM call per TERM CLUSTER            PAID
+    validate  verbatim span check, escalation          free
+    diff      new / duplicate / conflicting            free
+    queue     frequency-sorted review queue            free
+
+`service.py` is the facade; `cli.py` is the operator entry point.
+
+Design rationale: knowledge_pipeline_context.md
+Calibrated constants and why: KNOWLEDGE_PIPELINE_CALIBRATION.md
+"""
+
+from .adapter import parsed_doc_from_artifact
+from .models import (
+    AbbrevPair,
+    BriefContext,
+    CallUsage,
+    Chunk,
+    ClusterResult,
+    FilterResult,
+    FormulaEntry,
+    GlossaryEntry,
+    Mention,
+    ParsedDoc,
+    Provenance,
+    RuleCandidate,
+    RuleEntry,
+    TermCluster,
+)
+from .service import ExtractionResult, build_clusters, estimate_cost, extract_all, run_filters
+
+__all__ = [
+    "AbbrevPair",
+    "BriefContext",
+    "CallUsage",
+    "Chunk",
+    "ClusterResult",
+    "ExtractionResult",
+    "FilterResult",
+    "FormulaEntry",
+    "GlossaryEntry",
+    "Mention",
+    "ParsedDoc",
+    "Provenance",
+    "RuleCandidate",
+    "RuleEntry",
+    "TermCluster",
+    "build_clusters",
+    "estimate_cost",
+    "extract_all",
+    "parsed_doc_from_artifact",
+    "run_filters",
+]

src/knowledge_extraction/adapter.py

@@ -0,0 +1,136 @@
+"""Seam adapter: parsed-document artifact → the pipeline's internal `Chunk`.
+
+**This is the only file that knows the seam's shape.** Every stage downstream
+depends on `models.Chunk` alone, so when the artifact contract settles with
+Sofhia the change lands here and nowhere else.
+
+The seam is still under discussion (KNOWLEDGE_PIPELINE_TODO.md §3), so this
+reads defensively: it accepts either the draft's field names or the prototype's,
+takes plain dicts, and tolerates missing optional structure. It deliberately
+does **not** accept a file path — extraction never opens a document. That
+constraint is the point of the split, not an implementation detail.
+
+Two things it must never do:
+  - reflow, strip or whitespace-normalise `text`. Span validation locates quoted
+    spans literally inside it; cleaning the text makes the lookup fail and the
+    field go silently null, which presents as a bad model.
+  - infer a page number it was not given. A wrong page sends the reviewer to the
+    wrong part of the document, which is worse than no page at all.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+from typing import Any
+
+from .models import Chunk, ParsedDoc
+
+# Field names accepted for the same concept. `page_idx`/`page_idxs` are the
+# parsing half's contract; the rest are earlier drafts and the prototype's shape,
+# kept so old fixtures still load.
+#
+# Page numbers are 0-BASED throughout, exactly as the parser reports them. No
+# conversion happens anywhere in this pipeline: converting to 1-based is the
+# review UI's job, done once at display time. An off-by-one here would be
+# invisible until an expert opened the wrong page.
+_PAGE_KEYS = ("page_idx", "page_start", "page")
+_PAGES_KEYS = ("page_idxs", "pages", "page_list")
+_PAGE_END_KEYS = ("page_idx_end", "page_end")
+
+
+def chunk_from_dict(raw: dict[str, Any], doc_id: str, ordinal: int = 0) -> Chunk:
+    """Map one artifact item onto the internal chunk.
+
+    `kind`/`is_tabular` are reconciled: the draft carries a `kind` discriminator
+    while the prototype carried booleans. Either is accepted.
+    """
+    kind = raw.get("kind")
+    pages = _first(raw, _PAGES_KEYS) or []
+    page_start = _first(raw, _PAGE_KEYS)
+    if page_start is None:
+        page_start = min(pages) if pages else 0
+    page_end = _first(raw, _PAGE_END_KEYS)
+    if page_end is None:
+        page_end = max(pages) if pages else page_start
+
+    return Chunk(
+        chunk_id=raw.get("chunk_id") or f"{doc_id}#{ordinal:04d}",
+        doc_id=raw.get("doc_id") or doc_id,
+        text=raw["text"],  # verbatim, never cleaned
+        page_start=int(page_start),
+        page_end=int(page_end),
+        ordinal=int(raw.get("ordinal", ordinal)),
+        section_no=raw.get("section_no"),
+        heading=raw.get("heading"),
+        has_formula=bool(raw.get("has_formula", kind == "equation")),
+        is_tabular=bool(raw.get("is_tabular", kind == "table")),
+        bold_spans=list(raw.get("bold_spans") or []),
+    )
+
+
+def parsed_doc_from_artifact(
+    artifact: Any,
+    doc_id: str | None = None,
+    source_ref: str = "",
+    parser_name: str = "unknown",
+    parser_version: str = "",
+) -> ParsedDoc:
+    """Build a `ParsedDoc` from either shape of the artifact.
+
+    Accepts a bare `list[chunk]` (the draft's current shape) or a mapping with a
+    `chunks` key (the shape proposed for the document-level envelope). When the
+    envelope lands, its `content_hash`/`n_pages`/`version` are preferred over
+    the values derived here.
+    """
+    if hasattr(artifact, "model_dump"):  # a ParsedDocument from the parsing half
+        artifact = artifact.model_dump(mode="json")
+
+    if isinstance(artifact, dict):
+        items = artifact.get("chunks") or []
+        doc_id = doc_id or artifact.get("doc_id")
+        source_ref = source_ref or artifact.get("source_path") or artifact.get("source_ref") or ""
+        parser_name = artifact.get("parser_name") or parser_name
+        parser_version = artifact.get("parser_version") or parser_version
+        # The backend matters as much as the version: the same MinerU build can
+        # emit different text from `pipeline` and `vlm`, so a shift in extraction
+        # output has to be attributable to one or the other.
+        backend = artifact.get("parser_backend")
+        if backend:
+            parser_version = f"{parser_version}/{backend}" if parser_version else backend
+        declared_hash = artifact.get("content_hash")
+        declared_pages = artifact.get("n_pages")
+    else:
+        items = list(artifact)
+        declared_hash, declared_pages = None, None
+
+    if not doc_id:
+        doc_id = (items[0].get("doc_id") if items else None) or "unknown"
+
+    chunks = [chunk_from_dict(raw, doc_id, i) for i, raw in enumerate(items)]
+    pages = {p for c in chunks for p in (c.page_start, c.page_end)}
+
+    return ParsedDoc(
+        doc_id=doc_id,
+        source_ref=source_ref,
+        content_hash=declared_hash or content_hash(chunks),
+        n_pages=int(declared_pages) if declared_pages else (max(pages) + 1 if pages else 0),
+        chunks=chunks,
+        parser_name=parser_name,
+        parser_version=parser_version,
+        used_heading_split=any(c.section_no for c in chunks),
+    )
+
+
+def content_hash(chunks: list[Chunk]) -> str:
+    """Stable hash of the chunk text, so a re-parse that changed nothing can be
+    detected and the expensive stages skipped."""
+    blob = json.dumps([c.text for c in chunks], ensure_ascii=False).encode()
+    return hashlib.sha256(blob).hexdigest()[:16]
+
+
+def _first(raw: dict[str, Any], keys: tuple[str, ...]) -> Any:
+    for key in keys:
+        if raw.get(key) is not None:
+            return raw[key]
+    return None

src/knowledge_extraction/cli.py

@@ -0,0 +1,251 @@
+"""Offline runner for the knowledge-extraction pipeline.
+
+The pipeline runs a few times a year, triggered by an admin — so a script over a
+parsed artifact is the honest entry point, and any HTTP surface is a convenience
+layer over this, never the other way round.
+
+Takes a **parsed-document artifact**, never a PDF: extraction does not parse.
+Every stage writes its own JSON so a later stage can be re-run without repeating
+an earlier one, which matters because prompt iteration is the main development
+loop and the span filter is the slow part.
+
+    # free stages only (default) — no API calls, no spend
+    uv run --no-sync python -m src.knowledge_extraction.cli <artifact.json>
+
+    # cost estimate before spending anything
+    uv run --no-sync python -m src.knowledge_extraction.cli <artifact.json> --dry-run
+
+    # small pilot, then the full run
+    uv run --no-sync python -m src.knowledge_extraction.cli <artifact.json> --extract --limit 5
+    uv run --no-sync python -m src.knowledge_extraction.cli <artifact.json> --extract
+
+    # exercise the wiring with no credentials and no spend
+    uv run --no-sync python -m src.knowledge_extraction.cli <artifact.json> --extract --mock
+
+Lives inside the package rather than in `scripts/`, which is gitignored: this
+runner is the pipeline's operator entry point and has to ship with the module.
+
+**Always --dry-run before a corpus-scale run.** It builds the exact prompts,
+prints the token estimate, and makes zero API calls.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from pathlib import Path
+
+from .adapter import parsed_doc_from_artifact
+from .cluster import cluster_mentions
+from .extract import MockExtractor, cacheable, prefix_tokens
+from .models import Mention
+from .rank import rank_evidence
+from .service import build_clusters, estimate_cost, extract_all, run_filters
+from .settings import EVIDENCE_K
+
+BRANCHES = ("glossary", "rule", "formula", "summary")
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    parser.add_argument("artifact", type=Path, help="parsed-document artifact JSON")
+    parser.add_argument("--out-dir", type=Path, default=Path("out/knowledge"))
+    parser.add_argument("--doc-id", help="override the artifact's doc_id")
+    parser.add_argument("--mentions", type=Path, help="span-NER mentions JSON")
+    parser.add_argument(
+        "--no-span-filter",
+        action="store_true",
+        help="skip the span model; legend terms only (wiring check, NOT a recall run)",
+    )
+    parser.add_argument(
+        "--extract", action="store_true", help="run the PAID extraction stage"
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="build the prompts and print a token estimate; makes no API calls",
+    )
+    parser.add_argument("--mock", action="store_true", help="mock extractor: no network, no spend")
+    parser.add_argument("--limit", type=int, help="cap the number of items per branch (pilot)")
+    parser.add_argument(
+        "--branches", nargs="+", choices=BRANCHES, default=list(BRANCHES)
+    )
+    parser.add_argument(
+        "--active-glossary", type=Path, help="approved glossary to diff against"
+    )
+    args = parser.parse_args(argv)
+
+    if not args.artifact.exists():
+        print(f"artifact not found: {args.artifact}", file=sys.stderr)
+        return 2
+
+    raw = json.loads(args.artifact.read_text(encoding="utf-8"))
+    doc = parsed_doc_from_artifact(raw, doc_id=args.doc_id, source_ref=str(args.artifact))
+    print(
+        f"[parse ] {doc.doc_id}: {len(doc.chunks)} chunks, {doc.n_pages} pages, "
+        f"hash {doc.content_hash}, heading-split={doc.used_heading_split}"
+    )
+
+    filtered = run_filters(doc, use_span_filter=not (args.no_span_filter or args.mentions))
+    if args.mentions:
+        filtered.mentions = _load_mentions(args.mentions)
+        source = "file"
+    elif args.no_span_filter:
+        filtered.mentions = _from_pairs(doc, filtered.abbrev_pairs)
+        source = "legend stand-in (NOT a recall run)"
+    else:
+        source = "span filter"
+
+    capped = sum(m.hit_span_cap for m in filtered.mentions)
+    cap_note = f", {capped} hit the span cap" if capped else ""
+    print(
+        f"[filter] {len(filtered.abbrev_pairs)} abbreviation pairs, "
+        f"{len(filtered.rule_candidates)} rule candidates"
+    )
+    print(f"[filter] {len(filtered.mentions)} mentions from {source}{cap_note}")
+
+    if args.mentions or args.no_span_filter:
+        clustered = cluster_mentions(filtered.mentions, filtered.abbrev_pairs, doc.doc_id)
+        rank_evidence(clustered.clusters, doc.chunks)
+    else:
+        clustered = build_clusters(doc, filtered)
+    print(
+        f"[cluster] {clustered.n_mentions} mentions -> {clustered.n_clusters} clusters "
+        f"(compression {clustered.compression_ratio}x)"
+    )
+    for cluster in clustered.clusters[:8]:
+        print(
+            f"         {cluster.canonical:<26} mentions={cluster.mention_count:<4} "
+            f"evidence={len(cluster.evidence_chunk_ids)} "
+            f"top={cluster.evidence_chunk_ids[:EVIDENCE_K]}"
+        )
+
+    args.out_dir.mkdir(parents=True, exist_ok=True)
+    _dump(args.out_dir / f"{doc.doc_id}.chunks.json", doc.model_dump(mode="json"))
+    _dump(args.out_dir / f"{doc.doc_id}.filters.json", filtered.model_dump(mode="json"))
+    _dump(args.out_dir / f"{doc.doc_id}.clusters.json", clustered.model_dump(mode="json"))
+
+    if args.dry_run:
+        est = estimate_cost(doc, clustered, filtered, args.limit)
+        print("[dry-run] NO API CALLS MADE")
+        for key, value in est.items():
+            print(f"          {key}: {value}")
+        for branch in args.branches:
+            print(
+                f"          prefix[{branch}]: {prefix_tokens(branch)} tokens, "
+                f"cacheable={cacheable(branch)}"
+            )
+        return 0
+
+    if not args.extract:
+        print(f"[write ] {args.out_dir}   (free stages only; --extract to run the paid stage)")
+        return 0
+
+    extractor = MockExtractor() if args.mock else _azure_extractor()
+    if extractor is None:
+        return 3
+    active = (
+        json.loads(args.active_glossary.read_text(encoding="utf-8"))
+        if args.active_glossary
+        else []
+    )
+
+    result = extract_all(
+        doc,
+        clustered,
+        filtered,
+        extractor,
+        limit=args.limit,
+        active_glossary=active,
+        branches=tuple(args.branches),
+    )
+
+    prompt, cached, completion = result.total_tokens
+    simulated = " [SIMULATED — not a quality measurement]" if args.mock else ""
+    print(f"[extract] {len(result.usages)} calls{simulated}")
+    print(
+        f"          glossary={len(result.glossary)} rules={len(result.rules)} "
+        f"formulas={len(result.formulas)}"
+    )
+    print(f"          tokens prompt={prompt} cached={cached} completion={completion}")
+    print(f"          fields rejected by span check: {len(result.rejected)}")
+    no_def = sum(1 for e in result.glossary if e.get("extraction_status") == "no_definition_found")
+    print(f"          abstained (no definition in document): {no_def}/{len(result.glossary)}")
+
+    print("[queue ] top of the review queue:")
+    for row in result.review_queue[:10]:
+        term = str(row.get("term"))[:26]
+        print(
+            f"         {row['rank']:>3}. {term:<26} "
+            f"n={row['mention_count']:<4} {row['review_reason']}"
+        )
+
+    _dump(args.out_dir / "glossary.json", result.glossary)
+    _dump(args.out_dir / "interpretation_pack.json", result.rules)
+    _dump(args.out_dir / "formulas.json", result.formulas)
+    _dump(args.out_dir / "review_queue.json", result.review_queue)
+    _dump(args.out_dir / "rejected.json", [r.model_dump(mode="json") for r in result.rejected])
+    if result.brief:
+        _dump(args.out_dir / "brief_context.json", result.brief)
+    _dump(args.out_dir / "usage.json", [u.model_dump(mode="json") for u in result.usages])
+    print(f"[write ] {args.out_dir}")
+    return 0
+
+
+def _azure_extractor():
+    from .extract import AzureExtractor
+
+    try:
+        return AzureExtractor()
+    except Exception as exc:
+        print(f"cannot build the Azure client: {exc}", file=sys.stderr)
+        print("use --mock to exercise the pipeline without credentials", file=sys.stderr)
+        return None
+
+
+def _load_mentions(path: Path) -> list[Mention]:
+    raw = json.loads(path.read_text(encoding="utf-8"))
+    items = raw.get("mentions", raw) if isinstance(raw, dict) else raw
+    return [Mention.model_validate(m) for m in items]
+
+
+def _from_pairs(doc, pairs) -> list[Mention]:
+    """Stand-in mentions from legend abbreviations, so the wiring is runnable
+    without the span model.
+
+    NOT a recall measurement — it only sees terms a legend block already named.
+
+    Word-boundary matching, never substring: "PA" occurs inside "parameter",
+    "pada", "capacity" and "composite", and substring matching produced 126
+    spurious PA mentions on a 9-page document (77x compression instead of the
+    measured 2.56x). Same trap the evidence ranker documents for headings.
+    """
+    surfaces = {p.abbrev for p in pairs} | {p.expansion for p in pairs}
+    patterns = [
+        (s, re.compile(rf"(?<!\w){re.escape(s)}(?!\w)", re.IGNORECASE)) for s in surfaces
+    ]
+    out: list[Mention] = []
+    for chunk in doc.chunks:
+        for surface, pattern in patterns:
+            for match in pattern.finditer(chunk.text):
+                out.append(
+                    Mention(
+                        surface=surface,
+                        chunk_id=chunk.chunk_id,
+                        char_start=match.start(),
+                        char_end=match.end(),
+                        label="legend",
+                        score=1.0,
+                    )
+                )
+    return out
+
+
+def _dump(path: Path, payload) -> None:
+    path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/knowledge_extraction/cluster/__init__.py

@@ -0,0 +1,4 @@
+from .cluster import cluster_mentions
+from .normalize import AbbrevIndex, is_noise, normalize
+
+__all__ = ["AbbrevIndex", "cluster_mentions", "is_noise", "normalize"]

src/knowledge_extraction/cluster/cluster.py

@@ -0,0 +1,111 @@
+"""Mention[] → TermCluster[].
+
+Matching order, cheapest and most certain first:
+    1. exact match after normalisation
+    2. abbreviation ↔ expansion (from legend blocks)
+    3. conservative fuzzy (token_set_ratio >= 92, and only above 5 characters)
+
+**Over-merging is much worse than under-merging.** An under-merge costs one
+extra LLM call and one extra review-queue row. A wrong merge silently destroys a
+distinct term, and no downstream stage recovers it — the expert never learns the
+term existed. Every threshold here is set on that asymmetry.
+
+Clustering is also what makes conflict detection possible at all: contradictory
+definitions can only be compared if all evidence for a term reaches the same
+call.
+"""
+
+from __future__ import annotations
+
+from rapidfuzz import fuzz
+
+from ..models import AbbrevPair, ClusterResult, Mention, TermCluster
+from ..settings import FUZZY_MIN_LEN, FUZZY_THRESHOLD
+from .normalize import AbbrevIndex, is_noise
+
+
+def cluster_mentions(
+    mentions: list[Mention],
+    abbrev_pairs: list[AbbrevPair],
+    doc_id: str,
+    fuzzy_threshold: int = FUZZY_THRESHOLD,
+) -> ClusterResult:
+    index = AbbrevIndex(abbrev_pairs)
+    kept = [m for m in mentions if not is_noise(m.surface)]
+
+    buckets: dict[str, dict] = {}
+
+    for mention in kept:
+        key = index.canonical_key(mention.surface)
+
+        if key in buckets:
+            _add(buckets[key], mention, "exact")
+            continue
+
+        linked = next((k for k in buckets if index.linked(mention.surface, k)), None)
+        if linked:
+            _add(buckets[linked], mention, "abbrev")
+            continue
+
+        match = _fuzzy_match(key, buckets.keys(), fuzzy_threshold)
+        if match:
+            _add(buckets[match], mention, "fuzzy")
+            continue
+
+        buckets[key] = {
+            "surfaces": {mention.surface},
+            "mentions": [mention],
+            "reasons": set(),
+        }
+
+    clusters: list[TermCluster] = []
+    ordered = sorted(buckets.items(), key=lambda kv: -len(kv[1]["mentions"]))
+    for i, (key, data) in enumerate(ordered):
+        variants = sorted(data["surfaces"], key=lambda s: (len(s), s))
+        clusters.append(
+            TermCluster(
+                cluster_id=f"{doc_id}#c{i:03d}",
+                canonical=_canonical(variants, key),
+                variants=variants,
+                mentions=data["mentions"],
+                mention_count=len(data["mentions"]),
+                merge_reasons=sorted(data["reasons"]),
+            )
+        )
+
+    n_mentions, n_clusters = len(kept), len(clusters)
+    return ClusterResult(
+        doc_id=doc_id,
+        clusters=clusters,
+        n_mentions=n_mentions,
+        n_clusters=n_clusters,
+        compression_ratio=round(n_mentions / n_clusters, 3) if n_clusters else 0.0,
+    )
+
+
+def _add(bucket: dict, mention: Mention, reason: str) -> None:
+    bucket["surfaces"].add(mention.surface)
+    bucket["mentions"].append(mention)
+    bucket["reasons"].add(reason)
+
+
+def _fuzzy_match(key: str, existing, threshold: int) -> str | None:
+    best, best_score = None, 0.0
+    for other in existing:
+        # Short strings fuzzy-match far too easily: "PA" vs "UA" scores high on
+        # token_set_ratio. Below FUZZY_MIN_LEN only exact matching is allowed.
+        if min(len(key), len(other)) < FUZZY_MIN_LEN:
+            continue
+        score = fuzz.token_set_ratio(key, other)
+        if score >= threshold and score > best_score:
+            best, best_score = other, score
+    return best
+
+
+def _canonical(variants: list[str], key: str) -> str:
+    """Prefer the shortest non-trivial surface — usually the term as a reader
+    would look it up ("PA", not "Physical Availability (PA) untuk ...")."""
+    for v in variants:
+        if len(v) >= 2:
+            return v
+    return key

src/knowledge_extraction/cluster/normalize.py

@@ -0,0 +1,77 @@
+"""Surface normalisation and the abbreviation index used by clustering.
+
+**This normalisation is for clustering only.** Span validation normalises
+whitespace and nothing else — every additional normalisation there is a hole a
+fabrication can fit through. Do not reuse `normalize()` in that path.
+"""
+
+from __future__ import annotations
+
+import re
+import unicodedata
+
+from ..models import AbbrevPair
+
+# Surfaces that carry no discriminating power on their own. A mention of just
+# "unit" or "parameter" is not a term. These are dropped as WHOLE surface forms
+# only, never as substrings — so no term containing them is ever lost.
+STOP_SURFACES = {
+    "unit",
+    "type",
+    "class",
+    "equipment",
+    "equipment unit",
+    "parameter",
+    "activity",
+    "data",
+    "nilai",
+    "proses",
+    "hasil",
+    "total",
+}
+
+
+def normalize(surface: str) -> str:
+    s = unicodedata.normalize("NFKC", surface).casefold()
+    s = s.replace("-", " ").replace("_", " ")
+    s = re.sub(r"[.’']", "", s)
+    s = re.sub(r"[^\w\s/()]", " ", s)
+    s = re.sub(r"\s+", " ", s)
+    return s.strip(" ()/")
+
+
+def is_noise(surface: str) -> bool:
+    n = normalize(surface)
+    if len(n) < 2:
+        return True
+    if n in STOP_SURFACES:
+        return True
+    return not re.search(r"[a-z]", n)  # pure numbers / symbols
+
+
+class AbbrevIndex:
+    """Bidirectional abbreviation ↔ expansion lookup built from legend blocks.
+
+    This is why the legend filter runs before clustering: without it, `PA` and
+    `Physical Availability` never meet.
+    """
+
+    def __init__(self, pairs: list[AbbrevPair]):
+        self.to_expansion: dict[str, str] = {}
+        self.to_abbrev: dict[str, str] = {}
+        for pair in pairs:
+            abbrev, expansion = normalize(pair.abbrev), normalize(pair.expansion)
+            if not abbrev or not expansion:
+                continue
+            self.to_expansion[abbrev] = expansion
+            self.to_abbrev[expansion] = abbrev
+
+    def canonical_key(self, surface: str) -> str:
+        """Map a surface to a shared key so an abbreviation and its expansion
+        collide into the same bucket."""
+        n = normalize(surface)
+        return self.to_abbrev.get(n, n)
+
+    def linked(self, a: str, b: str) -> bool:
+        na, nb = normalize(a), normalize(b)
+        return self.to_expansion.get(na) == nb or self.to_expansion.get(nb) == na

src/knowledge_extraction/config/cues.yaml

@@ -0,0 +1,51 @@
+# Discourse cues (definitional) and legend-block patterns.
+# Used by cue_filter (rule candidates) and evidence ranking (+5.0 signal).
+
+definitional_id:
+  - '\badalah kondisi ketika\b'
+  - '\badalah\b'
+  - '\bmerupakan\b'
+  - '\byaitu\b'
+  - '\bdidefinisikan sebagai\b'
+  - '\bdimaksud dengan\b'
+
+definitional_en:
+  - '\bis defined as\b'
+  - '\brefers to\b'
+  - '\bis the\b'
+  - '\bmeans\b'
+
+conditional_id:
+  - '\bapabila\b'
+  - '\bjika\b.{0,80}\bmaka\b'
+  - '\bbila\b'
+  - '\bharus\b'
+  - '\btidak boleh\b'
+  - '\bwajib\b'
+
+conditional_en:
+  - '\bif\b.{0,80}\bthen\b'
+  - '\bmust\b'
+  - '\bshall\b'
+  - '\bNote that\b'
+
+note_id:
+  - '^\s*Catatan\b'
+  - '^\s*Sebagai contoh\b'
+  - '^\s*Contoh\b'
+
+# Legend blocks: 'Keterangan:' / 'Dimana:' / 'where' introduce symbol tables.
+legend_headers:
+  - '^\s*Keterangan\s*[::]'
+  - '^\s*Dimana\s*[::]'
+  - '^\s*Di\s*mana\s*[::]'
+  - '^\s*[Ww]here\s*[::]'
+
+legend_entry: '^\s*([A-Za-z][A-Za-z0-9_\.]{0,12})\s*[:=]\s*(.+)$'
+
+# A chunk is 'formula present' if any of these appear.
+formula_markers:
+  - '='
+  - '×'
+  - '\bx\s*100\s*%'
+  - '/'

src/knowledge_extraction/config/labels.yaml

@@ -0,0 +1,44 @@
+# GLiNER entity labels. E1 tunes these WITHOUT touching code.
+# Two variants so E1 can A/B label phrasing (GLiNER is sensitive to label wording).
+
+default:
+  - mining production metric
+  - equipment availability term
+  - operational abbreviation
+  - machine or equipment type
+  - time category
+  - production activity
+  - measurement unit
+
+# Variant B: Indonesian-language label phrasing. GLiNER multi is multilingual;
+# whether ID label strings improve ID recall is exactly what E1 tests.
+id_phrasing:
+  - istilah produksi tambang
+  - parameter ketersediaan alat
+  - singkatan operasional
+  - jenis alat berat
+  - kategori waktu kerja
+  - aktivitas produksi
+  - satuan pengukuran
+
+# Variant C: the first two variants both missed the same class of terms —
+# mining activities and materials (coal mining, land clearing, waste removal,
+# joint survey). This variant adds labels for that class specifically.
+broad:
+  - mining production metric
+  - equipment availability term
+  - operational abbreviation
+  - machine or equipment type
+  - time category
+  - mining activity
+  - material or commodity
+  - measurement unit
+  - data source or system
+  - work category
+  - analysis method
+
+# 0.25, not 0.35: the E1 sweep measured recall 0.854 @ 0.25 vs 0.658 @ 0.35 on the
+# `broad` variant. Precision falls (0.41 vs 0.50) but the term filter is meant to be
+# over-inclusive — clustering and evidence ranking absorb the noise, and a term the
+# filter never proposes can never be recovered downstream.
+threshold: 0.25

src/knowledge_extraction/diff/__init__.py

@@ -0,0 +1,3 @@
+from .glossary_diff import classify, diff_glossary
+
+__all__ = ["classify", "diff_glossary"]

src/knowledge_extraction/diff/glossary_diff.py

@@ -0,0 +1,33 @@
+"""Classify each candidate against the ACTIVE glossary version: new, duplicate
+or conflicting.
+
+The prototype diffed against the file it then overwrote, so every entry came
+back `new` and the interesting paths never ran. The baseline must therefore be
+supplied explicitly — an approved, versioned set — rather than read from
+wherever the last run happened to write.
+"""
+
+from __future__ import annotations
+
+from ..models import DiffStatus
+from ..settings import DUPLICATE_OVERLAP_THRESHOLD
+from ..validate.conflict import overlap
+
+
+def classify(entry: dict, existing_by_term: dict[str, dict]) -> DiffStatus:
+    prior = existing_by_term.get((entry.get("term") or "").casefold())
+    if prior is None:
+        return "new"
+    a = (entry.get("definition") or "").strip()
+    b = (prior.get("definition") or "").strip()
+    if a and a == b:
+        return "duplicate"
+    if not a or not b:
+        # One side abstained: not a contradiction, just less information.
+        return "new"
+    return "duplicate" if overlap(a, b) >= DUPLICATE_OVERLAP_THRESHOLD else "conflicting"
+
+
+def diff_glossary(entries: list[dict], active: list[dict]) -> list[dict]:
+    existing_by_term = {(e.get("term") or "").casefold(): e for e in active}
+    return [{**entry, "diff_status": classify(entry, existing_by_term)} for entry in entries]

src/knowledge_extraction/extract/__init__.py

@@ -0,0 +1,27 @@
+from .base import cacheable, est_tokens, evidence_block, load_prompt, prefix_tokens
+from .branches import (
+    build_glossary_prompt,
+    extract_formula,
+    extract_glossary,
+    extract_rule,
+    extract_summary,
+)
+from .client import AzureExtractor, LLMResult, MockExtractor
+from .schemas import schema_for
+
+__all__ = [
+    "AzureExtractor",
+    "LLMResult",
+    "MockExtractor",
+    "build_glossary_prompt",
+    "cacheable",
+    "est_tokens",
+    "evidence_block",
+    "extract_formula",
+    "extract_glossary",
+    "extract_rule",
+    "extract_summary",
+    "load_prompt",
+    "prefix_tokens",
+    "schema_for",
+]

src/knowledge_extraction/extract/base.py

@@ -0,0 +1,67 @@
+"""Shared plumbing for the four extraction branches.
+
+Prompts live in `prompts/*.txt`, never in code, for two reasons: a prompt change
+is not a code change, and **the fixed prefix must stay byte-identical across
+calls** or prompt caching silently stops engaging at roughly 10x the input cost.
+"""
+
+from __future__ import annotations
+
+from functools import lru_cache
+from pathlib import Path
+
+from ..models import Chunk
+from ..settings import CACHE_MIN_TOKENS
+
+PROMPT_DIR = Path(__file__).parent / "prompts"
+
+
+def est_tokens(text: str) -> int:
+    """Cheap estimate, for dry-run budgeting only. Real counts come from the
+    API's usage object — never report a cached price from an estimate."""
+    return max(1, int(len(text) / 3.6))
+
+
+@lru_cache(maxsize=8)
+def load_prompt(branch: str) -> str:
+    return (PROMPT_DIR / f"{branch}.txt").read_text(encoding="utf-8")
+
+
+def prefix_tokens(branch: str) -> int:
+    return est_tokens(load_prompt(branch))
+
+
+def cacheable(branch: str) -> bool:
+    """Whether the fixed prefix is long enough to cache at all.
+
+    Reported, never assumed: caching does not engage below the floor, so a
+    shorter prefix caches nothing. Only the API's `cached_tokens` proves a hit.
+    """
+    return prefix_tokens(branch) >= CACHE_MIN_TOKENS
+
+
+def evidence_block(chunks: list[Chunk], scores: list[float] | None = None) -> str:
+    """Evidence labelled with chunk_id, section and page so the model can cite
+    provenance and we can trace which evidence produced which field.
+
+    **The heading is included, and must stay included.** Two reasons:
+
+    1. Indonesian standards name the term in the heading and open the body with
+       the definition — "2.1.3 Physical of Availability (PA)" / "Adalah
+       ketersediaan fisik…" — so the body often never repeats the term. Without
+       the heading the model is asked to define a term the evidence never names.
+    2. It keeps one invariant true: **what the model reads is exactly what the
+       span check searches.** `validate.evidence_text` composes heading + text;
+       if this block showed only the text, the model could never quote a
+       section title, and any field that did quote one would be rejected as
+       unlocatable.
+    """
+    parts = []
+    for i, chunk in enumerate(chunks):
+        score = f" score={scores[i]:.1f}" if scores and i < len(scores) else ""
+        head = f"{chunk.heading}\n" if chunk.heading else ""
+        parts.append(
+            f"[chunk_id={chunk.chunk_id} section={chunk.section_no or '-'} "
+            f"page={chunk.page_start}{score}]\n{head}{chunk.text}"
+        )
+    return "EVIDENCE\n" + "\n\n---\n\n".join(parts)

src/knowledge_extraction/extract/branches.py

@@ -0,0 +1,235 @@
+"""The four extraction branches. This is the only stage that costs money.
+
+**One call per TERM CLUSTER** — not per mention, not per chunk. That is the
+whole economic argument for clustering: 200 mentions of "PA" cost one call, not
+200. It is also what makes conflict detection possible, since contradictory
+definitions can only be compared when they arrive together.
+
+Each branch returns `(entry, usage)`, with `None` for the entry when the
+response fails schema validation. A failed parse is not an exception: one bad
+response must not abort a corpus-scale run that has already paid for parsing.
+"""
+
+from __future__ import annotations
+
+from ...middlewares.logging import get_logger
+from ..models import (
+    BriefContext,
+    CallUsage,
+    Chunk,
+    FormulaEntry,
+    FormulaVariable,
+    GlossaryEntry,
+    Provenance,
+    RuleCandidate,
+    RuleEntry,
+    TermCluster,
+)
+from ..rank import top_k
+from ..settings import EVIDENCE_K
+from .base import evidence_block, load_prompt
+from .schemas import FormulaDraft, GlossaryDraft, RuleDraft, SummaryDraft, schema_for
+
+logger = get_logger("knowledge_extract")
+
+
+def _prov(draft_prov, doc_id: str, chunk_id: str | None = None) -> Provenance:
+    return Provenance(
+        doc_id=doc_id,
+        span=draft_prov.span,
+        page=draft_prov.page,
+        section_no=draft_prov.section_no,
+        chunk_id=chunk_id,
+    )
+
+
+def _evidence_for(
+    cluster: TermCluster, chunks: list[Chunk], k: int, round_index: int
+) -> tuple[list[Chunk], list[float]]:
+    by_id = {c.chunk_id: c for c in chunks}
+    ids = top_k(cluster, k=k, round_index=round_index)
+    scores = cluster.evidence_scores[round_index * k : round_index * k + k]
+    return [by_id[i] for i in ids if i in by_id], scores
+
+
+# ── glossary ────────────────────────────────────────────────────────────
+
+
+def build_glossary_prompt(
+    cluster: TermCluster, chunks: list[Chunk], k: int = EVIDENCE_K, round_index: int = 0
+) -> tuple[str, str]:
+    evidence, scores = _evidence_for(cluster, chunks, k, round_index)
+    user = (
+        f"CANDIDATE TERM: {cluster.canonical}\n"
+        f"KNOWN VARIANTS: {', '.join(cluster.variants)}\n"
+        f"MENTION COUNT: {cluster.mention_count}\n\n"
+        + evidence_block(evidence, scores)
+    )
+    return load_prompt("glossary"), user
+
+
+def extract_glossary(
+    cluster: TermCluster,
+    chunks: list[Chunk],
+    extractor,
+    doc_id: str,
+    k: int = EVIDENCE_K,
+    round_index: int = 0,
+) -> tuple[GlossaryEntry | None, CallUsage]:
+    system, user = build_glossary_prompt(cluster, chunks, k, round_index)
+    result = extractor.complete(
+        "glossary", system, user, schema_for("glossary"), "GlossaryEntry"
+    )
+    try:
+        draft = GlossaryDraft.model_validate(result.data)
+    except Exception as exc:
+        logger.warning(
+            "glossary draft invalid", cluster=cluster.canonical, error=repr(exc)
+        )
+        return None, result.usage
+
+    evidence, _ = _evidence_for(cluster, chunks, k, round_index)
+    entry = GlossaryEntry(
+        term=draft.term,
+        full_name=draft.full_name,
+        source_wording=_heading_wording(cluster, evidence) or draft.source_wording,
+        definition=draft.definition,
+        formula_latex=draft.formula_latex,
+        interpretation=draft.interpretation,
+        subdomain_tags=draft.subdomain_tags,
+        domain=draft.domain,
+        company=draft.company,
+        language=draft.language,
+        mention_count=cluster.mention_count,
+        provenance=_prov(
+            draft.provenance, doc_id, evidence[0].chunk_id if evidence else None
+        ),
+    )
+    return entry, result.usage
+
+
+def _heading_wording(cluster: TermCluster, evidence: list[Chunk]) -> str | None:
+    """The verbatim heading of the evidence chunk whose title names this term.
+
+    Preferred over whatever the model chose to quote, because the section
+    heading is where the document formally names the term. Measured on the
+    reference standard: the model quoted "Physical Availability (PA)" from the
+    page-1 intro — a real verbatim quote — while the section itself is headed
+    "Physical **of** Availability (PA)". Both occur in the document; only the
+    heading form reveals that the two disagree.
+
+    Recording the literal form is a locked decision: the discrepancy belongs to
+    the expert, not to us. Taking it deterministically rather than asking the
+    model to volunteer it means it cannot be normalised away.
+    """
+    from ..cluster.normalize import normalize
+    from ..rank.evidence import _word_match
+
+    variants = [normalize(v) for v in cluster.variants]
+    for chunk in evidence:
+        heading = chunk.heading
+        if heading and any(_word_match(v, normalize(heading)) for v in variants):
+            return heading
+    return None
+
+
+# ── rule of thumb ───────────────────────────────────────────────────────
+
+
+def extract_rule(
+    candidate: RuleCandidate, chunk: Chunk, extractor, doc_id: str
+) -> tuple[RuleEntry | None, CallUsage]:
+    user = (
+        f"CUE: {candidate.cue}\n\n"
+        + evidence_block([chunk])
+        + f"\n\nFOCUS PASSAGE:\n{candidate.snippet}"
+    )
+    result = extractor.complete(
+        "rule", load_prompt("rule"), user, schema_for("rule"), "RuleEntry"
+    )
+    try:
+        draft = RuleDraft.model_validate(result.data)
+    except Exception as exc:
+        logger.warning("rule draft invalid", chunk_id=candidate.chunk_id, error=repr(exc))
+        return None, result.usage
+
+    return (
+        RuleEntry(
+            rule_id=draft.rule_id,
+            statement=draft.statement,
+            condition=draft.condition,
+            consequence=draft.consequence,
+            applies_to=draft.applies_to,
+            subdomain_tags=draft.subdomain_tags,
+            language=draft.language,
+            provenance=_prov(draft.provenance, doc_id, chunk.chunk_id),
+        ),
+        result.usage,
+    )
+
+
+# ── formula ─────────────────────────────────────────────────────────────
+
+
+def extract_formula(
+    chunk: Chunk, extractor, doc_id: str
+) -> tuple[FormulaEntry | None, CallUsage]:
+    user = evidence_block([chunk])
+    result = extractor.complete(
+        "formula", load_prompt("formula"), user, schema_for("formula"), "FormulaEntry"
+    )
+    try:
+        draft = FormulaDraft.model_validate(result.data)
+    except Exception as exc:
+        logger.warning("formula draft invalid", chunk_id=chunk.chunk_id, error=repr(exc))
+        return None, result.usage
+
+    return (
+        FormulaEntry(
+            name=draft.name,
+            formula_latex=draft.formula_latex,
+            variables=[
+                FormulaVariable(symbol=v.symbol, meaning=v.meaning) for v in draft.variables
+            ],
+            unit=draft.unit,
+            provenance=_prov(draft.provenance, doc_id, chunk.chunk_id),
+        ),
+        result.usage,
+    )
+
+
+# ── summary ─────────────────────────────────────────────────────────────
+
+
+def extract_summary(
+    chunks: list[Chunk], extractor, doc_id: str
+) -> tuple[BriefContext | None, CallUsage]:
+    """Whole-document summary — the quiet cost risk. Few calls, but a large
+    share of all input tokens, because summarisation cannot be filtered: it
+    needs the whole document.
+
+    It is also the only branch that cannot be span-checked at all. A plausible
+    summary is indistinguishable from a correct one, which is exactly why it
+    belongs on a larger tier as soon as one exists.
+    """
+    user = evidence_block(chunks)
+    result = extractor.complete(
+        "summary", load_prompt("summary"), user, schema_for("summary"), "BriefContext"
+    )
+    try:
+        draft = SummaryDraft.model_validate(result.data)
+    except Exception as exc:
+        logger.warning("summary draft invalid", error=repr(exc))
+        return None, result.usage
+
+    return (
+        BriefContext(
+            title=draft.title,
+            purpose=draft.purpose,
+            scope=draft.scope,
+            key_parameters=draft.key_parameters,
+            summary_md=draft.summary_md,
+            provenance=_prov(draft.provenance, doc_id),
+        ),
+        result.usage,
+    )

src/knowledge_extraction/extract/client.py

@@ -0,0 +1,221 @@
+"""LLM clients for the extraction stage — **the only place this pipeline spends
+money.**
+
+Two things this module is careful about:
+
+- **Structured output is probed, not assumed.** `json_schema` needs a recent
+  api_version and we cannot confirm from here what the resource exposes. The
+  first call tries it; on rejection it falls back to `json_object` plus
+  validate-and-retry, and records which mode actually applied.
+- **Cached tokens are read from the API, never modelled.** Caching does not
+  engage below the token floor, so an under-length prefix caches nothing.
+  `usage.prompt_tokens_details.cached_tokens` is the only source of truth, and a
+  cached price must never be reported without it.
+
+All four branches route to the **nano** deployment (`__54n`). That is a recorded
+decision, not an oversight: nano measured 0.75 schema-fill precision against a
+0.80 line, and `rule`/`summary` — whose failure mode is least detectable, since
+a plausible summary cannot be span-checked — run there too until a larger
+deployment exists.
+"""
+
+from __future__ import annotations
+
+import json
+import time
+from typing import Any
+
+from ...config.settings import settings as app_settings
+from ...middlewares.logging import get_logger
+from ..models import Branch, CallUsage
+from ..settings import TEMPERATURE
+
+logger = get_logger("knowledge_extract_client")
+
+MAX_RETRIES = 3
+
+
+class LLMResult:
+    def __init__(self, data: dict, usage: CallUsage, raw: str = ""):
+        self.data = data
+        self.usage = usage
+        self.raw = raw
+
+
+class AzureExtractor:
+    """Real calls, real spend. Always dry-run before a corpus-scale run."""
+
+    def __init__(self, client=None, deployment: str | None = None):
+        self.deployment = deployment or app_settings.azureai_deployment_name_54n
+        self._client = client or self._build_client()
+        self._mode: str | None = None  # resolved on the first successful call
+
+    @staticmethod
+    def _build_client():
+        from openai import AzureOpenAI
+
+        endpoint = app_settings.azureai_endpoint_url_54n
+        api_key = app_settings.azureai_api_key_54n
+        if not endpoint or not api_key:
+            raise RuntimeError(
+                "azureai__endpoint__url__54n / azureai__api_key__54n are not set. "
+                "Use the mock extractor to run without Azure."
+            )
+        return AzureOpenAI(
+            azure_endpoint=endpoint,
+            api_key=api_key,
+            api_version=app_settings.azureai_api_version_54n,
+        )
+
+    def complete(
+        self,
+        branch: Branch,
+        system_prompt: str,
+        user_prompt: str,
+        schema: dict,
+        schema_name: str,
+    ) -> LLMResult:
+        messages = [
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_prompt},
+        ]
+        last_error: Exception | None = None
+
+        for attempt in range(MAX_RETRIES):
+            mode = self._mode or "json_schema"
+            started = time.time()
+            try:
+                response = self._client.chat.completions.create(
+                    model=self.deployment,
+                    messages=messages,
+                    temperature=TEMPERATURE,
+                    response_format=self._response_format(mode, schema, schema_name),
+                )
+            except Exception as exc:
+                if mode == "json_schema" and self._looks_unsupported(exc):
+                    logger.info(
+                        "json_schema unsupported — falling back to json_object",
+                        error=repr(exc),
+                    )
+                    self._mode = "json_object"
+                    continue
+                last_error = exc
+                logger.warning("call failed", branch=branch, attempt=attempt, error=repr(exc))
+                time.sleep(2**attempt)
+                continue
+
+            self._mode = mode
+            content = response.choices[0].message.content or "{}"
+            try:
+                data = json.loads(content)
+            except json.JSONDecodeError as exc:
+                last_error = exc
+                logger.warning("unparseable JSON", branch=branch, attempt=attempt)
+                continue
+
+            usage = self._usage(response, branch, time.time() - started, attempt, mode)
+            return LLMResult(data, usage, content)
+
+        raise RuntimeError(f"{branch}: all {MAX_RETRIES} attempts failed: {last_error!r}")
+
+    @staticmethod
+    def _response_format(mode: str, schema: dict, schema_name: str) -> dict:
+        if mode == "json_schema":
+            return {
+                "type": "json_schema",
+                "json_schema": {"name": schema_name, "schema": schema, "strict": False},
+            }
+        return {"type": "json_object"}
+
+    @staticmethod
+    def _looks_unsupported(exc: Exception) -> bool:
+        text = str(exc).lower()
+        return any(
+            s in text
+            for s in ("response_format", "json_schema", "unsupported", "invalid_request")
+        )
+
+    def _usage(
+        self, response: Any, branch: Branch, latency: float, retries: int, mode: str
+    ) -> CallUsage:
+        usage = getattr(response, "usage", None)
+        details = getattr(usage, "prompt_tokens_details", None)
+        # The ONLY source of truth for caching. Absent -> cached stays 0 and the
+        # uncached regime is what gets reported.
+        cached = int(getattr(details, "cached_tokens", 0) or 0) if details else 0
+        return CallUsage(
+            branch=branch,
+            deployment=self.deployment,
+            tier="nano",
+            prompt_tokens=int(getattr(usage, "prompt_tokens", 0) or 0),
+            cached_tokens=cached,
+            completion_tokens=int(getattr(usage, "completion_tokens", 0) or 0),
+            latency_s=round(latency, 3),
+            retries=retries,
+            structured_output_mode=mode,
+            simulated=False,
+        )
+
+
+class MockExtractor:
+    """No network, no spend. Every record it produces is stamped `simulated`.
+
+    Exercises the wiring — schema validation, span checking, escalation,
+    conflicts, diff, queue — without credentials. It is **not** a model-quality
+    measurement and its output must never be reported as one.
+
+    It abstains by default (returns null definitions), because abstention is the
+    dominant real behaviour: on the reference document 56 of 66 entries had no
+    definition. A mock that always answers would make the downstream stages look
+    far better exercised than they are.
+    """
+
+    def __init__(self, responses: dict[str, dict] | None = None, deployment: str = "mock"):
+        self.responses = responses or {}
+        self.deployment = deployment
+        self.calls: list[tuple[str, str]] = []
+
+    def complete(
+        self,
+        branch: Branch,
+        system_prompt: str,
+        user_prompt: str,
+        schema: dict,
+        schema_name: str,
+    ) -> LLMResult:
+        self.calls.append((branch, user_prompt))
+        data = self.responses.get(branch) or self._abstain(branch, user_prompt)
+        usage = CallUsage(
+            branch=branch,
+            deployment=self.deployment,
+            prompt_tokens=len(system_prompt) // 4 + len(user_prompt) // 4,
+            completion_tokens=40,
+            structured_output_mode="mock",
+            simulated=True,
+        )
+        return LLMResult(data, usage, json.dumps(data))
+
+    @staticmethod
+    def _abstain(branch: Branch, user_prompt: str) -> dict:
+        # Quote a real fragment so the span check has something locatable and is
+        # genuinely exercised rather than trivially passed.
+        span = ""
+        if "EVIDENCE" in user_prompt:
+            body = user_prompt.split("EVIDENCE", 1)[1]
+            for line in body.splitlines():
+                if line.strip() and not line.startswith("["):
+                    span = line.strip()[:60]
+                    break
+        prov = {"section_no": None, "page": 1, "span": span}
+        if branch == "glossary":
+            term = "unknown"
+            for line in user_prompt.splitlines():
+                if line.startswith("CANDIDATE TERM:"):
+                    term = line.split(":", 1)[1].strip()
+                    break
+            return {"term": term, "definition": None, "provenance": prov}
+        if branch == "rule":
+            return {"rule_id": "r_mock", "statement": None, "provenance": prov}
+        if branch == "formula":
+            return {"name": None, "formula_latex": None, "provenance": prov}
+        return {"title": None, "summary_md": None, "provenance": prov}

src/knowledge_extraction/extract/prompts/formula.txt

@@ -0,0 +1,28 @@
+You transcribe FORMULAS from Indonesian and English mining-operations standards.
+
+This is a TRANSCRIPTION task, not a derivation task. You are converting a formula
+that is already written in the evidence into LaTeX. You must never derive, simplify,
+correct, or complete a formula.
+
+RULES — correctness requirements:
+1. If no formula is present in the evidence, set "formula_latex" to null. A null
+   answer is a CORRECT answer.
+2. Transcribe exactly. If the source writes "x 100%", keep the percentage.
+3. List every variable that appears, with the meaning ONLY if the evidence states it
+   (usually in a "Keterangan:" or "Dimana:" legend block). Otherwise meaning is null.
+4. "provenance.span" must be copied VERBATIM from the evidence and is checked
+   automatically against the source.
+
+FIELD GUIDE:
+- name: what the formula computes, as named in the source
+- formula_latex: LaTeX transcription
+- variables: [{"symbol": "...", "meaning": "..." or null}, ...]
+- unit: the result unit if stated (BCM, ton, %, hours), else null
+
+WORKED EXAMPLE:
+Evidence: "Secara umum, PA dihitung menggunakan rumus berikut:\nPA = Total Hours -
+Breakdown / Total Hours x 100%\nKeterangan:\nPA : Physical Availability"
+Output:
+{"name": "Physical Availability (PA)", "formula_latex": "PA = \\frac{Total\\ Hours - Breakdown}{Total\\ Hours} \\times 100\\%", "variables": [{"symbol": "PA", "meaning": "Physical Availability"}, {"symbol": "Total Hours", "meaning": null}, {"symbol": "Breakdown", "meaning": null}], "unit": "%", "provenance": {"section_no": "2.1.3", "page": 4, "span": "PA = Total Hours - Breakdown"}}
+
+Return a single JSON object. No prose, no markdown fence.

src/knowledge_extraction/extract/prompts/glossary.txt

@@ -0,0 +1,68 @@
+You extract glossary entries from Indonesian and English mining-operations standards.
+
+RULES — these are not style preferences, they are correctness requirements:
+1. Extract ONLY what the evidence states. Never infer, complete, or generalise.
+2. If the evidence does not define the term, set "definition" to null. A null answer
+   is a CORRECT answer. Guessing is the single worst failure mode here.
+3. "provenance.span" must be copied VERBATIM from the evidence — character for
+   character, including Indonesian spelling. It is automatically checked against the
+   source and the field is discarded if it does not match exactly.
+4. "subdomain_tags" must come from the allowed list only. Do not invent tags.
+5. Preserve the source language. Do not translate an Indonesian definition to English.
+
+ALLOWED subdomain_tags:
+production, maintenance, hauling, loading, drilling_blasting, equipment, safety,
+quality, planning, cost, geology, other
+
+FIELD GUIDE:
+- term: the term as a reader would look it up (usually the abbreviation, e.g. "PA")
+- full_name: the expanded form exactly as written in the source, else null
+- definition: the definition as stated, else null
+- formula_latex: LaTeX of the formula IF the evidence shows one, else null
+- interpretation: what a high or low value means operationally, ONLY if stated, else null
+- language: "id", "en", or "mixed" — the language of the definition you extracted
+- provenance.span: the exact sentence or phrase you took the definition from
+
+WORKED EXAMPLE 1 (definition present):
+Evidence: "2.1.3. Physical of Availability (PA)\nAdalah ketersediaan fisik suatu
+equipment/unit yang menunjukkan proporsi waktu equipment/unit tersebut berada pada
+kondisi available (siap pakai) selama suatu periode tertentu."
+Output:
+{"term": "PA", "full_name": "Physical of Availability", "definition": "Adalah ketersediaan fisik suatu equipment/unit yang menunjukkan proporsi waktu equipment/unit tersebut berada pada kondisi available (siap pakai) selama suatu periode tertentu.", "formula_latex": null, "interpretation": null, "subdomain_tags": ["equipment", "production"], "domain": "mining", "company": null, "language": "id", "provenance": {"section_no": "2.1.3", "page": 4, "span": "Adalah ketersediaan fisik suatu equipment/unit"}}
+
+WORKED EXAMPLE 2 (term appears but is NOT defined — the important case):
+Evidence: "Gain/Loss menggunakan satuan BCM atau ton, Gap Standby menggunakan satuan
+jam (hours)"
+Candidate term: "BCM"
+Output:
+{"term": "BCM", "full_name": null, "definition": null, "formula_latex": null, "interpretation": null, "subdomain_tags": ["production"], "domain": "mining", "company": null, "language": "id", "provenance": {"section_no": "2.2.1", "page": 6, "span": "Gain/Loss menggunakan satuan BCM atau ton"}}
+Note: the term is mentioned but never defined, so definition is null. This is correct.
+
+WORKED EXAMPLE 3 (formula present):
+Evidence: "Secara umum, PA dihitung menggunakan rumus berikut:\nPA = Total Hours -
+Breakdown / Total Hours x 100%"
+Output field: "formula_latex": "PA = \\frac{Total\\ Hours - Breakdown}{Total\\ Hours} \\times 100\\%"
+
+WORKED EXAMPLE 4 (mixed-language definition, abbreviation and expansion both present):
+Evidence: "2.1.4. Utilization of Availability (UA)\nAdalah tingkat efektifitas
+penggunaan suatu alat yang tersedia secara fisik yang menunjukkan seberapa lama suatu
+unit digunakan secara efektif untuk bekerja - working hours selama berada dalam
+kondisi siap dioperasikan (available)."
+Output:
+{"term": "UA", "full_name": "Utilization of Availability", "definition": "Adalah tingkat efektifitas penggunaan suatu alat yang tersedia secara fisik yang menunjukkan seberapa lama suatu unit digunakan secara efektif untuk bekerja - working hours selama berada dalam kondisi siap dioperasikan (available).", "formula_latex": null, "interpretation": null, "subdomain_tags": ["equipment", "production"], "domain": "mining", "company": null, "language": "mixed", "provenance": {"section_no": "2.1.4", "page": 4, "span": "Adalah tingkat efektifitas penggunaan suatu alat yang tersedia secara fisik"}}
+Note: language is "mixed" because the Indonesian definition embeds English terms.
+
+COMMON MISTAKES TO AVOID:
+- Do NOT translate an Indonesian definition into English. Keep the source language.
+- Do NOT expand an abbreviation yourself. If the source never writes the expansion,
+  full_name is null.
+- Do NOT copy a definition from your own knowledge of mining. If this document does
+  not define the term, the answer is null, even if you know what the term means.
+- Do NOT paraphrase the provenance span to make it shorter or cleaner. It is compared
+  character by character against the source and a paraphrase will be rejected.
+- Do NOT merge two different terms into one entry. Extract only the candidate term.
+- Do NOT put units, table captions, or figure labels in the definition field.
+- If the evidence contains only a formula and no prose, set definition to null and
+  fill formula_latex only.
+
+Return a single JSON object. No prose, no markdown fence.

src/knowledge_extraction/extract/prompts/rule.txt

@@ -0,0 +1,39 @@
+You extract operational RULES from Indonesian and English mining-operations standards.
+
+A rule is a statement that constrains or directs behaviour: a condition and what
+follows from it, a requirement, a prohibition, or a calculation policy.
+
+RULES — correctness requirements, not style preferences:
+1. Extract ONLY what the evidence states. Never infer or complete a rule.
+2. If the evidence is descriptive rather than prescriptive, set "statement" to null.
+   A null answer is a CORRECT answer.
+3. "provenance.span" must be copied VERBATIM from the evidence. It is checked
+   automatically against the source and discarded if it does not match exactly.
+4. "subdomain_tags" must come from the allowed list only.
+5. Preserve the source language. Do not translate.
+
+ALLOWED subdomain_tags:
+production, maintenance, hauling, loading, drilling_blasting, equipment, safety,
+quality, planning, cost, geology, other
+
+FIELD GUIDE:
+- rule_id: SCREAMING_SNAKE_CASE, descriptive, derived from the rule's subject
+- statement: the rule in one sentence, as stated in the source
+- condition: the triggering condition if the rule is conditional, else null
+- consequence: what must happen when the condition holds, else null
+- applies_to: the parameter, equipment, or activity the rule governs, else null
+
+WORKED EXAMPLE 1 (conditional rule):
+Evidence: "Production yang digunakan dalam perhitungan adalah produksi hasil joint
+survey. Apabila data joint survey belum tersedia, maka digunakan data produksi
+berdasarkan truck count sebagai dasar perhitungan."
+Output:
+{"rule_id": "PTY_PRODUCTION_SOURCE", "statement": "Production yang digunakan dalam perhitungan adalah produksi hasil joint survey.", "condition": "Apabila data joint survey belum tersedia", "consequence": "digunakan data produksi berdasarkan truck count sebagai dasar perhitungan", "applies_to": "Productivity (Pty)", "subdomain_tags": ["production", "planning"], "language": "id", "provenance": {"section_no": "2.1.5", "page": 5, "span": "Apabila data joint survey belum tersedia, maka digunakan data produksi berdasarkan truck count"}}
+
+WORKED EXAMPLE 2 (descriptive, NOT a rule — the important case):
+Evidence: "Waterfall Analysis dapat di-review melalui dua metode."
+Output:
+{"rule_id": "NONE", "statement": null, "condition": null, "consequence": null, "applies_to": null, "subdomain_tags": ["other"], "language": "id", "provenance": {"section_no": "2.2.5", "page": 8, "span": "Waterfall Analysis dapat di-review melalui dua metode"}}
+Note: this describes a capability, it does not constrain behaviour. statement is null.
+
+Return a single JSON object. No prose, no markdown fence.

src/knowledge_extraction/extract/prompts/summary.txt

@@ -0,0 +1,21 @@
+You write a short orientation brief for a mining-operations standard document.
+
+This is the ONE branch whose output cannot be span-checked, because a summary is by
+nature not verbatim. Treat that as a reason for restraint, not licence: state only
+what the document states, and prefer omission to elaboration.
+
+RULES:
+1. Use only the section headings and text provided. Do not add industry background.
+2. If the document does not state a purpose or scope, set that field to null.
+3. "provenance.span" must still be copied VERBATIM from the evidence — use the most
+   representative sentence from the document's purpose section.
+4. Keep summary_md under 200 words. It orients a reader; it does not replace the doc.
+
+FIELD GUIDE:
+- title: the document's title as written
+- purpose: what the document is for, as stated
+- scope: what it covers, as stated
+- key_parameters: the main parameters or concepts the document defines
+- summary_md: a short markdown orientation, under 200 words
+
+Return a single JSON object. No prose, no markdown fence.

src/knowledge_extraction/extract/schemas.py

@@ -0,0 +1,85 @@
+"""What the MODEL is asked to return.
+
+Deliberately separate from `models.py`. The model never supplies `doc_id` (we
+know it), never sets `extraction_status`, and never sets the conflict or diff
+fields — validation owns those. **Asking a model for fields it cannot know is an
+invitation to fabricate**, so the request schema is narrower than the stored one.
+"""
+
+from __future__ import annotations
+
+from typing import Literal
+
+from pydantic import BaseModel, Field
+
+from ..models import SubdomainEnum
+
+Language = Literal["id", "en", "mixed"]
+
+
+class ProvenanceDraft(BaseModel):
+    section_no: str | None = None
+    page: int
+    span: str
+
+
+class GlossaryDraft(BaseModel):
+    term: str
+    full_name: str | None = None
+    # The wording exactly as the document writes it, even when "wrong" — the
+    # standard heads its section "Physical of Availability (PA)". Surfacing the
+    # discrepancy is the point; normalising it hides a decision the expert owns.
+    source_wording: str | None = None
+    definition: str | None = None
+    formula_latex: str | None = None
+    interpretation: str | None = None
+    subdomain_tags: list[SubdomainEnum] = Field(default_factory=list)
+    domain: str | None = None
+    company: str | None = None
+    language: Language | None = None
+    provenance: ProvenanceDraft
+
+
+class RuleDraft(BaseModel):
+    rule_id: str
+    statement: str | None = None
+    condition: str | None = None
+    consequence: str | None = None
+    applies_to: str | None = None
+    subdomain_tags: list[SubdomainEnum] = Field(default_factory=list)
+    language: Language | None = None
+    provenance: ProvenanceDraft
+
+
+class VariableDraft(BaseModel):
+    symbol: str
+    meaning: str | None = None
+
+
+class FormulaDraft(BaseModel):
+    name: str | None = None
+    formula_latex: str | None = None
+    variables: list[VariableDraft] = Field(default_factory=list)
+    unit: str | None = None
+    provenance: ProvenanceDraft
+
+
+class SummaryDraft(BaseModel):
+    title: str | None = None
+    purpose: str | None = None
+    scope: str | None = None
+    key_parameters: list[str] = Field(default_factory=list)
+    summary_md: str | None = None
+    provenance: ProvenanceDraft
+
+
+DRAFTS = {
+    "glossary": GlossaryDraft,
+    "rule": RuleDraft,
+    "formula": FormulaDraft,
+    "summary": SummaryDraft,
+}
+
+
+def schema_for(branch: str) -> dict:
+    return DRAFTS[branch].model_json_schema()

src/knowledge_extraction/filters/__init__.py

@@ -0,0 +1,12 @@
+from .cue_filter import definitional_hits, rule_candidates
+from .legend_filter import abbrev_pairs, find_legend_lines
+from .span_filter import extract_mentions, load_model
+
+__all__ = [
+    "abbrev_pairs",
+    "definitional_hits",
+    "extract_mentions",
+    "find_legend_lines",
+    "load_model",
+    "rule_candidates",
+]

src/knowledge_extraction/filters/cue_filter.py

@@ -0,0 +1,76 @@
+"""Discourse-cue regex → rule-of-thumb candidates. Free: zero API calls.
+
+Also supplies `definitional_hits`, which evidence ranking uses for its strongest
+signal (+5.0 when a cue sits within 100 characters of a mention). Cue patterns
+live in `config/cues.yaml` so they can be tuned per language without a code
+change.
+"""
+
+from __future__ import annotations
+
+import re
+from functools import lru_cache
+
+from ..models import Chunk, RuleCandidate
+from ..settings import load_yaml
+
+SNIPPET_CHARS = 240
+
+
+@lru_cache(maxsize=8)
+def _compiled(group: str) -> tuple[re.Pattern[str], ...]:
+    cues = load_yaml("cues.yaml")
+    return tuple(
+        re.compile(p, re.IGNORECASE | re.MULTILINE) for p in cues.get(group, [])
+    )
+
+
+def _groups(*names: str) -> tuple[re.Pattern[str], ...]:
+    out: list[re.Pattern[str]] = []
+    for name in names:
+        out.extend(_compiled(name))
+    return tuple(out)
+
+
+def definitional_hits(text: str) -> list[tuple[int, int]]:
+    """(start, end) of every definitional cue. Consumed by evidence ranking."""
+    spans: list[tuple[int, int]] = []
+    for pattern in _groups("definitional_id", "definitional_en"):
+        spans.extend((m.start(), m.end()) for m in pattern.finditer(text))
+    return sorted(spans)
+
+
+def rule_candidates(chunks: list[Chunk]) -> list[RuleCandidate]:
+    """Passages a conditional or note cue marks as possibly stating a rule.
+
+    Over-inclusive on purpose — this is a candidate generator, and the paid
+    stage decides whether a rule is actually present. Nothing is dropped here.
+    """
+    patterns = _groups("conditional_id", "conditional_en", "note_id")
+    out: list[RuleCandidate] = []
+    for chunk in chunks:
+        seen: set[int] = set()
+        for pattern in patterns:
+            for match in pattern.finditer(chunk.text):
+                start = match.start()
+                # One candidate per sentence-ish region: several cues in the
+                # same clause would otherwise produce near-identical rows.
+                bucket = start // 200
+                if bucket in seen:
+                    continue
+                seen.add(bucket)
+                out.append(
+                    RuleCandidate(
+                        chunk_id=chunk.chunk_id,
+                        cue=match.group(0).strip(),
+                        char_start=start,
+                        char_end=match.end(),
+                        snippet=_snippet(chunk.text, start),
+                    )
+                )
+    return out
+
+
+def _snippet(text: str, start: int) -> str:
+    lo = max(0, start - SNIPPET_CHARS // 4)
+    return text[lo : lo + SNIPPET_CHARS].strip()

src/knowledge_extraction/filters/legend_filter.py

@@ -0,0 +1,98 @@
+"""Legend blocks → abbreviation pairs + formula variables. Free: zero API calls.
+
+A legend block is the `Keterangan:` / `Dimana:` / `where` table that follows a
+formula and expands its symbols:
+
+    Keterangan:
+    MOHH : Machine on Hand Hours
+    Qty  : Quantity
+
+This runs **before clustering** and that ordering is load-bearing: without the
+abbreviation pairs it produces, `PA` and `Physical Availability` cluster as two
+unrelated terms and the expert sees the same concept twice.
+
+`find_legend_lines` additionally feeds evidence ranking (+3.5 when a mention
+sits inside a legend block).
+"""
+
+from __future__ import annotations
+
+import re
+from functools import lru_cache
+
+from ..models import AbbrevPair, Chunk
+from ..settings import load_yaml
+
+# How far below a legend header the block is assumed to run before a blank line
+# or a non-entry line closes it.
+MAX_BLOCK_LINES = 30
+
+
+@lru_cache(maxsize=2)
+def _headers() -> tuple[re.Pattern[str], ...]:
+    cues = load_yaml("cues.yaml")
+    return tuple(re.compile(p) for p in cues.get("legend_headers", []))
+
+
+@lru_cache(maxsize=2)
+def _entry() -> re.Pattern[str]:
+    cues = load_yaml("cues.yaml")
+    return re.compile(cues["legend_entry"])
+
+
+def find_legend_lines(text: str) -> set[int]:
+    """Zero-based line numbers that fall inside a legend block."""
+    lines = text.split("\n")
+    inside: set[int] = set()
+    for i, line in enumerate(lines):
+        if not any(h.search(line) for h in _headers()):
+            continue
+        for j in range(i + 1, min(i + 1 + MAX_BLOCK_LINES, len(lines))):
+            if not lines[j].strip():
+                break
+            if not _entry().match(lines[j]):
+                break
+            inside.add(j)
+    return inside
+
+
+def abbrev_pairs(chunks: list[Chunk]) -> list[AbbrevPair]:
+    """Extract `abbrev : expansion` rows from every legend block.
+
+    Precision matters more than recall here: a wrong pair merges two distinct
+    terms into one cluster, which destroys a term silently. So only lines
+    inside a detected block are read, never `X : Y` anywhere in the text.
+    """
+    out: list[AbbrevPair] = []
+    seen: set[tuple[str, str]] = set()
+    for chunk in chunks:
+        lines = chunk.text.split("\n")
+        for line_no in sorted(find_legend_lines(chunk.text)):
+            match = _entry().match(lines[line_no])
+            if not match:
+                continue
+            abbrev, expansion = match.group(1).strip(), match.group(2).strip()
+            if not _plausible(abbrev, expansion):
+                continue
+            key = (abbrev.casefold(), expansion.casefold())
+            if key in seen:
+                continue
+            seen.add(key)
+            out.append(
+                AbbrevPair(
+                    abbrev=abbrev, expansion=expansion, chunk_id=chunk.chunk_id
+                )
+            )
+    return out
+
+
+def _plausible(abbrev: str, expansion: str) -> bool:
+    """Reject rows that are formula lines rather than legend entries."""
+    if not abbrev or not expansion:
+        return False
+    if len(expansion) < 3 or len(expansion) > 120:
+        return False
+    # An expansion that is mostly digits/operators is the right-hand side of an
+    # equation, not a term ("Qty = 360/720").
+    letters = sum(c.isalpha() for c in expansion)
+    return letters >= max(3, len(expansion) // 3)

src/knowledge_extraction/filters/span_filter.py

@@ -0,0 +1,169 @@
+"""Span NER over ALL chunks → Mention[]. Free stage: zero API calls, CPU only.
+
+**Runs unconditionally on every chunk.** There is deliberately no relevance gate
+in front of it: a binary pre-filter's false negatives drop content *before*
+extraction ever sees it, and dropped content never reaches expert review to be
+caught. The model is cheap enough (CPU encoder, ~400 MB) to just look at
+everything, which turns "zero candidate spans" into the relevance signal as a
+byproduct of looking rather than a judgment made before looking.
+
+Two mechanical traps, both of which silently cost recall if reintroduced:
+
+1. **The model truncates long inputs and warns rather than failing.** Feeding a
+   whole chunk drops its tail without erroring. Chunks are therefore fed as
+   overlapping word windows with offsets remapped back to chunk coordinates.
+2. **There is a ~12-token span cap.** Compound terms ("UA plan grouping
+   (composite) actual") exceed it. Candidates that hit the cap are *flagged*,
+   never silently truncated, so a recall shortfall can be attributed to it.
+
+The model is loaded lazily and cached: it is heavy, and only this stage needs
+it, so importing the package must not pay for it.
+"""
+
+from __future__ import annotations
+
+import time
+from functools import lru_cache
+
+from ...middlewares.logging import get_logger
+from ..models import Chunk, Mention
+from ..settings import (
+    SPAN_TOKEN_CAP,
+    WINDOW_OVERLAP,
+    WINDOW_WORDS,
+    labels_for,
+)
+
+logger = get_logger("knowledge_span_filter")
+
+MODEL_NAME = "urchade/gliner_multi-v2.1"
+
+
+@lru_cache(maxsize=2)
+def load_model(model_name: str = MODEL_NAME):
+    """Lazy, cached. Imported inside the function so the package stays importable
+    (and the app stays fast to boot) without the model stack."""
+    from gliner import GLiNER
+
+    logger.info("loading span model", model=model_name)
+    return GLiNER.from_pretrained(model_name)
+
+
+def extract_mentions(
+    chunks: list[Chunk],
+    variant: str | None = None,
+    threshold: float | None = None,
+    model=None,
+) -> list[Mention]:
+    """Candidate term mentions across every chunk.
+
+    Never raises: the span filter is the widest net in the pipeline, and a model
+    failure must degrade to "no candidates from this chunk" rather than abort a
+    corpus-scale ingest that has already paid for parsing. Failures are logged
+    with `repr(e)` — an empty error string is how a real cause once went missing
+    for a day elsewhere in this repo.
+    """
+    labels, configured_threshold = labels_for(variant) if variant else labels_for()
+    score_floor = threshold if threshold is not None else configured_threshold
+
+    try:
+        model = model or load_model()
+    except Exception as exc:  # pragma: no cover - depends on the model stack
+        logger.error("span model unavailable", error=repr(exc))
+        return []
+
+    mentions: list[Mention] = []
+    capped = 0
+    started = time.time()
+
+    for chunk in chunks:
+        for window_text, offset in _windows(chunk.text):
+            try:
+                found = model.predict_entities(
+                    window_text, labels, threshold=score_floor
+                )
+            except Exception as exc:
+                logger.warning(
+                    "span prediction failed", chunk_id=chunk.chunk_id, error=repr(exc)
+                )
+                continue
+
+            for entity in found:
+                surface = entity.get("text", "")
+                if not surface.strip():
+                    continue
+                hit_cap = len(surface.split()) >= SPAN_TOKEN_CAP
+                capped += hit_cap
+                mentions.append(
+                    Mention(
+                        surface=surface,
+                        chunk_id=chunk.chunk_id,
+                        char_start=offset + int(entity.get("start", 0)),
+                        char_end=offset + int(entity.get("end", 0)),
+                        label=str(entity.get("label", "")),
+                        score=float(entity.get("score", 0.0)),
+                        hit_span_cap=hit_cap,
+                    )
+                )
+
+    deduped = _dedupe(mentions)
+    logger.info(
+        "span filter complete",
+        chunks=len(chunks),
+        mentions=len(deduped),
+        dropped_overlapping=len(mentions) - len(deduped),
+        hit_span_cap=capped,
+        seconds=round(time.time() - started, 1),
+    )
+    return deduped
+
+
+def _windows(text: str) -> list[tuple[str, int]]:
+    """Overlapping word windows plus each window's character offset.
+
+    Windowing is on WORDS but offsets must come back in CHARACTERS, so the
+    original spacing is walked rather than re-joined — re-joining would shift
+    every offset in a chunk containing newlines or double spaces, and every
+    span check downstream would then fail.
+    """
+    if not text.strip():
+        return []
+
+    positions: list[tuple[int, int]] = []
+    cursor = 0
+    for word in text.split():
+        start = text.index(word, cursor)
+        positions.append((start, start + len(word)))
+        cursor = start + len(word)
+
+    if len(positions) <= WINDOW_WORDS:
+        return [(text, 0)]
+
+    step = max(1, WINDOW_WORDS - WINDOW_OVERLAP)
+    windows: list[tuple[str, int]] = []
+    for begin in range(0, len(positions), step):
+        chunk_words = positions[begin : begin + WINDOW_WORDS]
+        if not chunk_words:
+            break
+        lo, hi = chunk_words[0][0], chunk_words[-1][1]
+        windows.append((text[lo:hi], lo))
+        if begin + WINDOW_WORDS >= len(positions):
+            break
+    return windows
+
+
+def _dedupe(mentions: list[Mention]) -> list[Mention]:
+    """Drop duplicates produced by window overlap, keeping the highest score.
+
+    Overlap is required for recall (a term straddling a window boundary would
+    otherwise be missed), and it necessarily re-finds terms in the overlap
+    region. Keyed on (chunk, span) so the same surface at a different position
+    stays a separate mention — mention COUNT drives the review queue ordering.
+    """
+    best: dict[tuple[str, int, int], Mention] = {}
+    for mention in mentions:
+        key = (mention.chunk_id, mention.char_start, mention.char_end)
+        current = best.get(key)
+        if current is None or mention.score > current.score:
+            best[key] = mention
+    return sorted(best.values(), key=lambda m: (m.chunk_id, m.char_start))

src/knowledge_extraction/models.py

@@ -0,0 +1,273 @@
+"""Pydantic contracts for the knowledge-extraction pipeline.
+
+Three invariants are encoded here rather than described in prose, because every
+one of them is a control that a later change could quietly remove:
+
+1. **All content fields are Optional.** A model that cannot answer null will
+   fabricate one. Abstention is correct behaviour, never an error.
+2. **`subdomain_tags` is an enum.** Classification, not generation.
+3. **`Provenance.span` is mandatory and verbatim-checked.** It is the primary
+   anti-hallucination control and the thing that makes expert review
+   finishable — the reviewer checks a quote against a page, not a claim
+   against their memory.
+
+`Chunk` here is the pipeline's **internal** unit, deliberately narrower than the
+parsed-document artifact being agreed with Sofhia (the seam). Stages depend only
+on this subset; `adapter.py` maps the seam type onto it, so seam churn lands in
+one file instead of seven. See KNOWLEDGE_PIPELINE_TODO.md §3.
+"""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Literal
+
+from pydantic import BaseModel, Field
+
+Branch = Literal["glossary", "rule", "formula", "summary"]
+ExtractionStatus = Literal["ok", "no_definition_found", "escalated"]
+DiffStatus = Literal["new", "duplicate", "conflicting"]
+
+
+class SubdomainEnum(str, Enum):
+    """Classification target. Extend deliberately — a new member changes what
+    the model is allowed to answer, which is a prompt change, not a data one."""
+
+    production = "production"
+    maintenance = "maintenance"
+    hauling = "hauling"
+    loading = "loading"
+    drilling_blasting = "drilling_blasting"
+    equipment = "equipment"
+    safety = "safety"
+    quality = "quality"
+    planning = "planning"
+    cost = "cost"
+    geology = "geology"
+    other = "other"
+
+
+# ── Stage 1: the chunk (internal view of the seam artifact) ─────────────
+
+
+class Chunk(BaseModel):
+    """One unit of a parsed document, as the extraction stages need it.
+
+    `text` must stay **verbatim** from the source document. Span validation
+    locates LLM-quoted spans literally inside this text; if it is ever reflowed
+    or whitespace-normalised the lookup fails and the field is silently set to
+    null. The failure presents as a bad model, but the cause would be here.
+    """
+
+    chunk_id: str
+    doc_id: str
+    text: str
+    page_start: int
+    page_end: int
+    ordinal: int = 0
+
+    # Structural context. Both Optional — many documents carry no numbering.
+    section_no: str | None = None
+    heading: str | None = None
+
+    # Cheap downstream filters / ranking signals
+    has_formula: bool = False
+    is_tabular: bool = False
+    bold_spans: list[str] = Field(default_factory=list)
+
+
+class ParsedDoc(BaseModel):
+    """A document's chunks plus the identity needed to version and cache them."""
+
+    doc_id: str
+    source_ref: str
+    content_hash: str
+    n_pages: int
+    chunks: list[Chunk]
+    parser_name: str = "unknown"
+    parser_version: str = ""
+    used_heading_split: bool = False
+
+
+# ── Stage 2: filters ────────────────────────────────────────────────────
+
+
+class Mention(BaseModel):
+    """One occurrence of a candidate term inside a chunk."""
+
+    surface: str
+    chunk_id: str
+    char_start: int
+    char_end: int
+    label: str = ""
+    score: float = 0.0
+    hit_span_cap: bool = False
+
+
+class RuleCandidate(BaseModel):
+    """A passage a discourse cue marks as possibly stating a rule of thumb."""
+
+    chunk_id: str
+    cue: str
+    char_start: int
+    char_end: int
+    snippet: str
+
+
+class AbbrevPair(BaseModel):
+    """`PA` ↔ `Physical Availability`, harvested from a legend block.
+
+    Legend extraction must run before clustering: without these, an
+    abbreviation and its expansion cluster as two unrelated terms.
+    """
+
+    abbrev: str
+    expansion: str
+    chunk_id: str
+
+
+class FilterResult(BaseModel):
+    doc_id: str
+    mentions: list[Mention] = Field(default_factory=list)
+    rule_candidates: list[RuleCandidate] = Field(default_factory=list)
+    abbrev_pairs: list[AbbrevPair] = Field(default_factory=list)
+
+
+# ── Stage 3: clusters ───────────────────────────────────────────────────
+
+
+class TermCluster(BaseModel):
+    """All mentions of one term. **The LLM call unit is the cluster**, not the
+    chunk and not the mention — that is what cuts expert review burden, and it
+    is also the only reason conflicting definitions can be detected at all
+    (they must arrive in the same call to be compared)."""
+
+    cluster_id: str
+    canonical: str
+    variants: list[str] = Field(default_factory=list)
+    mentions: list[Mention] = Field(default_factory=list)
+    mention_count: int = 0
+    merge_reasons: list[str] = Field(default_factory=list)
+
+    # Ranked best-first. The FULL list is kept, not just the top K —
+    # escalation consumes the tail.
+    evidence_chunk_ids: list[str] = Field(default_factory=list)
+    evidence_scores: list[float] = Field(default_factory=list)
+
+
+class ClusterResult(BaseModel):
+    doc_id: str
+    clusters: list[TermCluster] = Field(default_factory=list)
+    n_mentions: int = 0
+    n_clusters: int = 0
+    compression_ratio: float = 0.0
+
+
+# ── Stage 4+: extracted entries ─────────────────────────────────────────
+
+
+class Provenance(BaseModel):
+    """Where a claim came from. `span` is mandatory and must appear verbatim in
+    the evidence text; a field whose span cannot be located is rejected, never
+    repaired. A repaired span is an unfalsifiable claim."""
+
+    doc_id: str
+    span: str
+    page: int | None = None
+    section_no: str | None = None
+    chunk_id: str | None = None
+
+
+class GlossaryEntry(BaseModel):
+    term: str
+    full_name: str | None = None
+
+    # The literal wording as the document writes it, un-normalised. The BUMA
+    # standard heads its section "Physical of Availability (PA)" while the
+    # legend says "Physical Availability"; the discrepancy is surfaced to the
+    # expert rather than silently corrected.
+    source_wording: str | None = None
+
+    definition: str | None = None
+    formula_latex: str | None = None
+    interpretation: str | None = None
+    subdomain_tags: list[SubdomainEnum] = Field(default_factory=list)
+    domain: str | None = None
+    company: str | None = None
+    language: str | None = None
+
+    mention_count: int = 0
+    provenance: Provenance
+    extraction_status: ExtractionStatus = "ok"
+    diff_status: DiffStatus | None = None
+    definition_conflict: bool = False
+    conflict_variants: list[str] = Field(default_factory=list)
+
+
+class RuleEntry(BaseModel):
+    """A rule of thumb / operational convention stated by the document."""
+
+    rule_id: str
+    statement: str | None = None
+    condition: str | None = None
+    consequence: str | None = None
+    applies_to: str | None = None
+    subdomain_tags: list[SubdomainEnum] = Field(default_factory=list)
+    language: str | None = None
+    provenance: Provenance
+    extraction_status: ExtractionStatus = "ok"
+
+
+class FormulaVariable(BaseModel):
+    symbol: str
+    meaning: str | None = None
+
+
+class FormulaEntry(BaseModel):
+    name: str | None = None
+    formula_latex: str | None = None
+    variables: list[FormulaVariable] = Field(default_factory=list)
+    unit: str | None = None
+    provenance: Provenance
+    extraction_status: ExtractionStatus = "ok"
+
+
+class BriefContext(BaseModel):
+    """Whole-document summary. The only branch that cannot be span-checked —
+    a plausible summary is indistinguishable from a correct one, which is why
+    it belongs on the larger model tier when one is available."""
+
+    title: str | None = None
+    purpose: str | None = None
+    scope: str | None = None
+    key_parameters: list[str] = Field(default_factory=list)
+    summary_md: str | None = None
+    provenance: Provenance
+
+
+class CallUsage(BaseModel):
+    """Per-call accounting. `cached_tokens` comes from the API and is never
+    modelled: caching does not engage below 1024 prompt tokens, so assuming it
+    would understate cost by ~10x on the input side."""
+
+    branch: Branch
+    deployment: str
+    tier: str = "nano"
+    prompt_tokens: int = 0
+    cached_tokens: int = 0
+    completion_tokens: int = 0
+    latency_s: float = 0.0
+    retries: int = 0
+    structured_output_mode: str = ""
+    simulated: bool = False
+
+
+class RejectedField(BaseModel):
+    """Audit row for a field the span check refused. Kept so a reviewer can see
+    what the control caught rather than only what it let through."""
+
+    entry_term: str
+    field: str
+    offending_value: str
+    reason: str
+    branch: Branch

src/knowledge_extraction/queue/__init__.py

@@ -0,0 +1,3 @@
+from .review_queue import build_queue
+
+__all__ = ["build_queue"]

src/knowledge_extraction/queue/review_queue.py

@@ -0,0 +1,65 @@
+"""The frequency-sorted review queue — the pipeline's actual product.
+
+Ordering is the product decision here, and it targets the bottleneck directly:
+the expert is the scarce resource, so they should hit the terms whose definition
+propagates furthest first. Conflicts are promoted above frequency regardless,
+because a contradiction is a decision only they can make.
+
+Each row carries page, section and the verbatim span so review is a matter of
+checking a quote against a page, not a claim against memory. That is what makes
+the queue finishable.
+"""
+
+from __future__ import annotations
+
+
+def build_queue(entries: list[dict]) -> list[dict]:
+    def sort_key(entry: dict):
+        conflicting = (
+            entry.get("diff_status") == "conflicting"
+            or entry.get("definition_conflict") is True
+        )
+        return (0 if conflicting else 1, -int(entry.get("mention_count", 0) or 0))
+
+    queue = []
+    for rank, entry in enumerate(sorted(entries, key=sort_key), start=1):
+        provenance = entry.get("provenance") or {}
+        queue.append(
+            {
+                "rank": rank,
+                "term": entry.get("term"),
+                "definition": entry.get("definition"),
+                "source_wording": entry.get("source_wording"),
+                "mention_count": entry.get("mention_count", 0),
+                "extraction_status": entry.get("extraction_status"),
+                "diff_status": entry.get("diff_status"),
+                "definition_conflict": entry.get("definition_conflict", False),
+                "conflict_variants": entry.get("conflict_variants", []),
+                "page": provenance.get("page"),
+                "section_no": provenance.get("section_no"),
+                "span": provenance.get("span"),
+                "review_reason": _reason(entry),
+            }
+        )
+    return queue
+
+
+def _reason(entry: dict) -> str:
+    if entry.get("definition_conflict") or entry.get("diff_status") == "conflicting":
+        return "conflicting definitions — expert decision required"
+    if _wording_differs(entry):
+        return "source wording differs from the expanded name — confirm which is correct"
+    if entry.get("extraction_status") == "no_definition_found":
+        return "term found but no definition in document"
+    if not entry.get("definition"):
+        return "definition rejected by span check or absent"
+    return "routine confirmation"
+
+
+def _wording_differs(entry: dict) -> bool:
+    """The document says "Physical of Availability"; the expansion says
+    "Physical Availability". Surfacing that to the expert is a locked
+    requirement, so it earns its own review reason."""
+    source = (entry.get("source_wording") or "").strip().casefold()
+    full = (entry.get("full_name") or "").strip().casefold()
+    return bool(source and full) and full not in source

src/knowledge_extraction/rank/__init__.py

@@ -0,0 +1,3 @@
+from .evidence import rank_evidence, rounds_available, top_k
+
+__all__ = ["rank_evidence", "rounds_available", "top_k"]

src/knowledge_extraction/rank/evidence.py

@@ -0,0 +1,168 @@
+"""Score and select the top-K evidence chunks per cluster.
+
+This is a filter, and it structurally resembles the relevance gate the team
+rejected. It is defensible only because of three properties, all of which must
+survive any future change here:
+
+1. **No term is ever dropped** — only its *evidence* is narrowed.
+2. **Every mention stays in the cluster**, so a bad top-3 is visible in
+   provenance rather than invisible.
+3. **The full ranked list is retained**, so escalation can take the next K and
+   self-correct.
+
+The rejected relevance gate had none of the three. If ranking is ever revisited,
+that is the test to apply.
+"""
+
+from __future__ import annotations
+
+import re
+
+from ..cluster.normalize import normalize
+from ..filters.cue_filter import definitional_hits
+from ..filters.legend_filter import find_legend_lines
+from ..models import Chunk, TermCluster
+from ..settings import CUE_PROXIMITY_CHARS, EVIDENCE_K, EVIDENCE_WEIGHTS
+
+
+def rank_evidence(
+    clusters: list[TermCluster],
+    chunks: list[Chunk],
+    weights: dict[str, float] | None = None,
+) -> None:
+    """Mutates clusters in place: sets `evidence_chunk_ids` (ranked, best first)
+    and `evidence_scores`. Keeps the FULL ranked list, not just the top K."""
+    w = {**EVIDENCE_WEIGHTS, **(weights or {})}
+    by_id = {c.chunk_id: c for c in chunks}
+    # Document order, so "first occurrence" means the term's first appearance in
+    # the document — not whichever element a set happened to yield first, which
+    # would hand the +1.0 to an arbitrary chunk and make ranking unreproducible.
+    doc_order = {c.chunk_id: i for i, c in enumerate(chunks)}
+    first_seen: set[str] = set()
+
+    for cluster in clusters:
+        chunk_ids = sorted(
+            {m.chunk_id for m in cluster.mentions},
+            key=lambda cid: (doc_order.get(cid, len(chunks)), cid),
+        )
+        scored: list[tuple[float, str]] = []
+        for chunk_id in chunk_ids:
+            chunk = by_id.get(chunk_id)
+            if chunk is None:
+                continue
+            positions = [m for m in cluster.mentions if m.chunk_id == chunk_id]
+            scored.append((_score_chunk(chunk, cluster, positions, w, first_seen), chunk_id))
+        # Tie-break on document order so ranking is stable across runs.
+        scored.sort(key=lambda t: (-t[0], doc_order.get(t[1], len(chunks)), t[1]))
+        cluster.evidence_chunk_ids = [cid for _, cid in scored]
+        cluster.evidence_scores = [round(s, 2) for s, _ in scored]
+
+
+def top_k(
+    cluster: TermCluster, k: int = EVIDENCE_K, round_index: int = 0
+) -> list[str]:
+    """Evidence for extraction round `round_index` (0 = best K, 1 = next K…)."""
+    start = round_index * k
+    return cluster.evidence_chunk_ids[start : start + k]
+
+
+def rounds_available(cluster: TermCluster, k: int, max_rounds: int) -> int:
+    """How many escalation rounds this cluster actually has evidence for.
+
+    Usually 0 on short documents — a cluster with a single evidence chunk has
+    nowhere to escalate, which is why an escalation count of zero is not by
+    itself a sign the loop is broken.
+    """
+    extra = max(0, len(cluster.evidence_chunk_ids) - k)
+    return min(max_rounds, -(-extra // k))  # ceil division
+
+
+def _score_chunk(
+    chunk: Chunk,
+    cluster: TermCluster,
+    positions: list,
+    w: dict[str, float],
+    first_seen: set[str],
+) -> float:
+    score = 0.0
+
+    # Word-boundary match, NOT substring. "PA" is a substring of "Parameter",
+    # "pada" and "composite", so substring matching handed this bonus to almost
+    # every Indonesian heading and pushed real definition sections below
+    # formula tables.
+    heading = normalize(chunk.heading or "")
+    term_in_heading = bool(heading) and any(
+        _word_match(normalize(v), heading) for v in cluster.variants
+    )
+    if term_in_heading:
+        score += w["term_in_heading"]
+
+    # Cue proximity. When the heading names the term, the heading counts as a
+    # mention at offset 0.
+    #
+    # Indonesian standards are written "2.1.3 Physical of Availability (PA)" /
+    # "Adalah ketersediaan fisik..." — the heading names the term and the body
+    # opens with the cue WITHOUT repeating it. Measured on the BUMA standard:
+    # without this, the PA definition section scored no cue bonus at all and
+    # ranked below a computation section that merely mentions PA often. The
+    # section's own definition is the single most valuable piece of evidence
+    # there is, so losing the signal there defeats the ranking.
+    cues = definitional_hits(chunk.text)
+    anchors = [(m.char_start, m.char_end) for m in positions]
+    if term_in_heading:
+        anchors.append((0, 0))
+    if any(
+        abs(cs - start) <= CUE_PROXIMITY_CHARS or abs(ce - end) <= CUE_PROXIMITY_CHARS
+        for cs, ce in cues
+        for start, end in anchors
+    ):
+        score += w["definitional_cue_near"]
+
+    legend_lines = find_legend_lines(chunk.text)
+    if legend_lines:
+        line_starts = _line_starts(chunk.text)
+        for mention in positions:
+            if _line_of(mention.char_start, line_starts) in legend_lines:
+                score += w["in_legend_block"]
+                break
+
+    if chunk.has_formula:
+        score += w["formula_present"]
+
+    normalised_variants = {normalize(v) for v in cluster.variants}
+    if any(normalize(b) in normalised_variants for b in chunk.bold_spans):
+        score += w["bold_or_italic"]
+
+    if cluster.cluster_id not in first_seen:
+        score += w["first_occurrence"]
+        first_seen.add(cluster.cluster_id)
+
+    if chunk.is_tabular:
+        score += w["tabular_penalty"]
+
+    return score
+
+
+def _word_match(needle: str, haystack: str) -> bool:
+    if not needle:
+        return False
+    return re.search(rf"(?<!\w){re.escape(needle)}(?!\w)", haystack) is not None
+
+
+def _line_starts(text: str) -> list[int]:
+    starts, pos = [0], 0
+    for line in text.split("\n")[:-1]:
+        pos += len(line) + 1
+        starts.append(pos)
+    return starts
+
+
+def _line_of(char_pos: int, line_starts: list[int]) -> int:
+    lo, hi = 0, len(line_starts) - 1
+    while lo < hi:
+        mid = (lo + hi + 1) // 2
+        if line_starts[mid] <= char_pos:
+            lo = mid
+        else:
+            hi = mid - 1
+    return lo

src/knowledge_extraction/service.py

@@ -0,0 +1,231 @@
+"""Pipeline facade: parsed document → candidate entries → review queue.
+
+Mirrors the shape of `src/query/service.py` — a deterministic orchestrator over
+stages that each do one thing, with the expensive step isolated and every
+failure degrading rather than aborting.
+
+Cost discipline, carried from the prototype and worth keeping: **dry-run, then a
+small pilot, then the full run.** A dry run makes zero API calls and prints the
+token estimate, so the bill is knowable before it is incurred.
+"""
+
+from __future__ import annotations
+
+import time
+
+from ..middlewares.logging import get_logger
+from .cluster import cluster_mentions
+from .diff import diff_glossary
+from .extract import (
+    build_glossary_prompt,
+    est_tokens,
+    extract_formula,
+    extract_glossary,
+    extract_rule,
+    extract_summary,
+)
+from .filters import abbrev_pairs, extract_mentions, rule_candidates
+from .models import (
+    CallUsage,
+    Chunk,
+    ClusterResult,
+    FilterResult,
+    ParsedDoc,
+    RejectedField,
+)
+from .queue import build_queue
+from .rank import rank_evidence, top_k
+from .settings import EVIDENCE_K
+from .validate import evidence_text, find_conflicts, rounds_available, validate_entry
+
+logger = get_logger("knowledge_extraction")
+
+
+class ExtractionResult:
+    def __init__(self) -> None:
+        self.glossary: list[dict] = []
+        self.rules: list[dict] = []
+        self.formulas: list[dict] = []
+        self.brief: dict | None = None
+        self.review_queue: list[dict] = []
+        self.rejected: list[RejectedField] = []
+        self.usages: list[CallUsage] = []
+
+    @property
+    def total_tokens(self) -> tuple[int, int, int]:
+        return (
+            sum(u.prompt_tokens for u in self.usages),
+            sum(u.cached_tokens for u in self.usages),
+            sum(u.completion_tokens for u in self.usages),
+        )
+
+
+def run_filters(doc: ParsedDoc, use_span_filter: bool = True) -> FilterResult:
+    """All free stages. Zero API calls."""
+    pairs = abbrev_pairs(doc.chunks)
+    mentions = extract_mentions(doc.chunks) if use_span_filter else []
+    return FilterResult(
+        doc_id=doc.doc_id,
+        mentions=mentions,
+        rule_candidates=rule_candidates(doc.chunks),
+        abbrev_pairs=pairs,
+    )
+
+
+def build_clusters(doc: ParsedDoc, filtered: FilterResult) -> ClusterResult:
+    clustered = cluster_mentions(filtered.mentions, filtered.abbrev_pairs, doc.doc_id)
+    rank_evidence(clustered.clusters, doc.chunks)
+    return clustered
+
+
+def estimate_cost(
+    doc: ParsedDoc, clustered: ClusterResult, filtered: FilterResult, limit: int | None = None
+) -> dict:
+    """Dry run: exact prompts are built, nothing is sent."""
+    clusters = clustered.clusters[:limit] if limit else clustered.clusters
+    prompt_tokens = 0
+    for cluster in clusters:
+        system, user = build_glossary_prompt(cluster, doc.chunks)
+        prompt_tokens += est_tokens(system) + est_tokens(user)
+    return {
+        "glossary_calls": len(clusters),
+        "rule_calls": len(filtered.rule_candidates),
+        "formula_calls": sum(1 for c in doc.chunks if c.has_formula),
+        "summary_calls": 1,
+        "estimated_prompt_tokens": prompt_tokens,
+        "note": "estimate only — real counts come from the API usage object",
+    }
+
+
+def extract_all(
+    doc: ParsedDoc,
+    clustered: ClusterResult,
+    filtered: FilterResult,
+    extractor,
+    limit: int | None = None,
+    active_glossary: list[dict] | None = None,
+    branches: tuple[str, ...] = ("glossary", "rule", "formula", "summary"),
+) -> ExtractionResult:
+    """The paid stage plus validation, diff and queue."""
+    out = ExtractionResult()
+    started = time.time()
+
+    if "glossary" in branches:
+        _run_glossary(doc, clustered, extractor, out, limit)
+    if "rule" in branches:
+        _run_rules(doc, filtered, extractor, out, limit)
+    if "formula" in branches:
+        _run_formulas(doc, extractor, out, limit)
+    if "summary" in branches:
+        _run_summary(doc, extractor, out)
+
+    out.glossary = diff_glossary(out.glossary, active_glossary or [])
+    out.review_queue = build_queue(out.glossary)
+
+    prompt, cached, completion = out.total_tokens
+    logger.info(
+        "extraction complete",
+        doc_id=doc.doc_id,
+        glossary=len(out.glossary),
+        rules=len(out.rules),
+        formulas=len(out.formulas),
+        rejected_fields=len(out.rejected),
+        calls=len(out.usages),
+        prompt_tokens=prompt,
+        cached_tokens=cached,
+        completion_tokens=completion,
+        seconds=round(time.time() - started, 1),
+    )
+    return out
+
+
+def _run_glossary(doc, clustered, extractor, out, limit) -> None:
+    clusters = clustered.clusters[:limit] if limit else clustered.clusters
+    for cluster in clusters:
+        entry = None
+        max_round = rounds_available(cluster, EVIDENCE_K)
+
+        for round_index in range(max_round + 1):
+            entry, usage = extract_glossary(
+                cluster, doc.chunks, extractor, doc.doc_id, EVIDENCE_K, round_index
+            )
+            out.usages.append(usage)
+            if entry is None:
+                continue
+
+            source = evidence_text(
+                top_k(cluster, EVIDENCE_K, round_index), doc.chunks
+            )
+            entry, rejections = validate_entry(entry, "glossary", source, cluster.canonical)
+            out.rejected.extend(rejections)
+
+            if entry.definition:
+                if round_index > 0:
+                    entry.extraction_status = "escalated"
+                break
+            # Null definition -> escalate to the next K chunks.
+
+        if entry is None:
+            continue
+        if not entry.definition:
+            entry.extraction_status = "no_definition_found"
+
+        conflicting, variants = find_conflicts(
+            [entry.definition] if entry.definition else []
+        )
+        entry.definition_conflict = conflicting
+        entry.conflict_variants = variants
+        out.glossary.append(entry.model_dump(mode="json"))
+
+
+def _run_rules(doc, filtered, extractor, out, limit) -> None:
+    by_id: dict[str, Chunk] = {c.chunk_id: c for c in doc.chunks}
+    candidates = filtered.rule_candidates[:limit] if limit else filtered.rule_candidates
+    seen: set[str] = set()
+    for candidate in candidates:
+        chunk = by_id.get(candidate.chunk_id)
+        if chunk is None:
+            continue
+        entry, usage = extract_rule(candidate, chunk, extractor, doc.doc_id)
+        out.usages.append(usage)
+        if entry is None:
+            continue
+        entry, rejections = validate_entry(entry, "rule", chunk.text, entry.rule_id)
+        out.rejected.extend(rejections)
+        key = (entry.statement or "").strip().casefold()
+        if key and key in seen:
+            continue
+        if key:
+            seen.add(key)
+        out.rules.append(entry.model_dump(mode="json"))
+
+
+def _run_formulas(doc, extractor, out, limit) -> None:
+    chunks = [c for c in doc.chunks if c.has_formula]
+    chunks = chunks[:limit] if limit else chunks
+    seen: set[str] = set()
+    for chunk in chunks:
+        entry, usage = extract_formula(chunk, extractor, doc.doc_id)
+        out.usages.append(usage)
+        if entry is None:
+            continue
+        entry, rejections = validate_entry(
+            entry, "formula", chunk.text, entry.name or chunk.chunk_id
+        )
+        out.rejected.extend(rejections)
+        key = (entry.formula_latex or "").strip()
+        if key and key in seen:
+            continue
+        if key:
+            seen.add(key)
+        out.formulas.append(entry.model_dump(mode="json"))
+
+
+def _run_summary(doc, extractor, out) -> None:
+    entry, usage = extract_summary(doc.chunks, extractor, doc.doc_id)
+    out.usages.append(usage)
+    if entry is None:
+        return
+    # Summary prose is NOT span-checked — it cannot be. Only its provenance is
+    # carried, and the branch belongs on a larger tier for exactly that reason.
+    out.brief = entry.model_dump(mode="json")

src/knowledge_extraction/settings.py

@@ -0,0 +1,93 @@
+"""Tunables for the knowledge-extraction pipeline.
+
+Every value here was calibrated on real documents and each one has a reason
+recorded in KNOWLEDGE_PIPELINE_CALIBRATION.md. Change them deliberately — most
+were arrived at by a measurement, and two of them (`FUZZY_MIN_LEN`,
+`CACHE_MIN_TOKENS`) fix bugs that are silent when reintroduced.
+
+Label and cue sets live in `config/*.yaml` so they can be tuned without a code
+change: label phrasing is the main recall lever and the filter is very sensitive
+to it.
+"""
+
+from __future__ import annotations
+
+from functools import lru_cache
+from pathlib import Path
+
+import yaml
+
+CONFIG_DIR = Path(__file__).resolve().parent / "config"
+
+# ── Term filter ─────────────────────────────────────────────────────────
+# Variant C beat both the English-default and Indonesian-phrasing label sets:
+# the other two missed the same class (mining activities and materials).
+LABELS_VARIANT = "broad"
+
+# 0.25, not 0.35: measured recall 0.854 @ 0.25 vs 0.658 @ 0.35 on `broad`.
+# Precision falls (0.41 vs 0.50) and that is the intended trade — the filter is
+# deliberately over-inclusive, clustering and ranking absorb the noise, and a
+# term the filter never proposes can never be recovered downstream.
+SPAN_SCORE_THRESHOLD = 0.25
+
+# The span NER model truncates past ~384 of its own tokens and *warns rather
+# than failing*, so a long chunk silently loses its tail. Indonesian technical
+# prose subword-tokenises at roughly 2.5x, so 220-word windows still tripped the
+# cap; 130 does not. Chunks are fed as overlapping windows with offsets remapped.
+WINDOW_WORDS = 130
+WINDOW_OVERLAP = 30
+SPAN_TOKEN_CAP = 12
+
+# ── Clustering ──────────────────────────────────────────────────────────
+FUZZY_THRESHOLD = 92
+
+# Below this length only exact matching is allowed: "PA" and "UA" score highly
+# against each other on token_set_ratio. Over-merging is far worse than
+# under-merging — an under-merge costs one extra call and one extra review row,
+# a wrong merge destroys a distinct term and the expert never sees it.
+FUZZY_MIN_LEN = 5
+
+# ── Evidence ranking ────────────────────────────────────────────────────
+EVIDENCE_K = 3
+CUE_PROXIMITY_CHARS = 100
+
+EVIDENCE_WEIGHTS: dict[str, float] = {
+    "definitional_cue_near": 5.0,
+    "term_in_heading": 4.0,
+    "in_legend_block": 3.5,
+    "formula_present": 2.0,
+    "bold_or_italic": 1.5,
+    "first_occurrence": 1.0,
+    "tabular_penalty": -3.0,
+}
+
+# ── Chunking ────────────────────────────────────────────────────────────
+MAX_CHUNK_TOKENS = 1500
+MAX_HEADING_LEN = 90
+BOILERPLATE_MIN_FRAC = 0.6
+
+# ── Extraction ──────────────────────────────────────────────────────────
+TEMPERATURE = 0.0
+
+# OpenAI-family prompt caching does not engage AT ALL below this many prompt
+# tokens, so a shorter fixed prefix caches nothing and costs ~10x on input. The
+# measured hit rate at/above it was 54%.
+CACHE_MIN_TOKENS = 1024
+
+# ── Validation ──────────────────────────────────────────────────────────
+MAX_ESCALATION_ROUNDS = 2
+CONFLICT_OVERLAP_THRESHOLD = 0.4
+DUPLICATE_OVERLAP_THRESHOLD = 0.8
+
+
+@lru_cache(maxsize=4)
+def load_yaml(name: str) -> dict:
+    with open(CONFIG_DIR / name, encoding="utf-8") as fh:
+        return yaml.safe_load(fh)
+
+
+def labels_for(variant: str = LABELS_VARIANT) -> tuple[list[str], float]:
+    """Returns (labels, threshold) for a label variant."""
+    cfg = load_yaml("labels.yaml")
+    labels = cfg.get(variant) or cfg.get(LABELS_VARIANT) or []
+    return list(labels), float(cfg.get("threshold", SPAN_SCORE_THRESHOLD))