docs/knowledge/KNOWLEDGE_PIPELINE_CALIBRATION.md

Knowledge Pipeline — Calibration Reference

🔄 LIVE — tuned constants and the reason behind each. Read before changing any constant in src/knowledge_extraction/settings.py.

Note (2026-09-02): the primary parser is now Mistral OCR (no GPU). The MinerU-specific parsing sections here (§10 quality, §12 render_latex, §13 backend choice) are historical as of 2026-09-07 — MinerU was dropped and is installed nowhere, so those sections record how the choice was made rather than a path you can run; the extraction constants (GLiNER, clustering, ranking, thresholds) are unaffected and current. See eval/parsing/README.md.

Harvested from the kex prototype, 2026-08-19. Every number below cost a run, a bug, or a measurement to find, and none of it is in the spec — it lived only as constants and comments in a codebase that is not being ported.

This file exists because clean rewrites drop accidental knowledge and then rediscover it as production defects. When building extraction v2, take the value and the reason. If you change one, change it deliberately and record why.

Companions: KNOWLEDGE_PIPELINE_TODO.md (plan) · eval/knowledge/ (gold set, scorer, frozen baseline) · knowledge_pipeline_context.md (intent).

---

1. Term filter — GLiNER

Constant	Value	Why this value
Model	urchade/gliner_multi-v2.1	CPU, ~400 MB. Cheap enough to run unconditionally on every chunk, which is what lets us reject a standalone relevance gate
Score threshold	0.25	Not 0.35. The sweep measured recall 0.854 @ 0.25 vs 0.658 @ 0.35 on the broad label set. Precision falls (0.41 vs 0.50) and that is accepted: the filter is meant to be over-inclusive, clustering and evidence ranking absorb the noise, and a term the filter never proposes can never be recovered downstream
Label variant	broad	Two earlier variants both missed the same class — mining activities and materials (coal mining, land clearing, waste removal, joint survey). broad adds mining activity, material or commodity, data source or system, work category, analysis method to cover it
Window size	130 words, overlap 30 (re-confirmed 2026-09-10: 256 truncates 46 of 99 Open Pit windows, 180 truncates 4; 130 is the only size measured safe on both English documents. One BUMA window exceeds the cap at EVERY size — pre-existing)	Not 220. GLiNER truncates past ~384 of its own tokens and warns rather than failing — silently dropping the tail of every long chunk. Indonesian technical prose subword-tokenises at roughly 2.5×, so a 220-word window still tripped the cap. Chunks are fed as overlapping word windows with offsets remapped
Span token cap	12 (GLiNER's own)	Compound terms ("UA plan grouping (composite) actual") exceed it. Candidates that hit the cap are flagged, not silently truncated, so a recall shortfall can be attributed to the cap

Label sets (labels.yaml) — three variants were tried and A/B'd without touching code. Keep that property: label phrasing is the main recall lever and GLiNER is very sensitive to it. The Indonesian-phrasing variant (istilah produksi tambang, …) did not win; broad did.

---

2. Clustering

Constant	Value	Why this value
Match order	exact (normalised) → abbreviation↔expansion → fuzzy	Cheapest and most certain first
Fuzzy threshold	92	⛔ RETIRED 2026-09-10. token_set_ratio returns exactly 100 when one token set is a SUBSET of the other, so this threshold was inert and a head noun always absorbed its compounds. 131 of 140 merges were subset matches; the 6 correct ones were plurals, now handled by normalize(lemma=True). The pass is commented out at its call site; _fuzzy_match is parked. Subset merges went to 0 on all three test documents
Fuzzy disabled below 5 characters	min(len(a), len(b)) < 5 → skip	The one to not lose. PA vs UA scores high on token_set_ratio. Below 5 chars only exact matching is allowed
Canonical surface	shortest variant ≥ 2 chars	That is how a reader looks a term up — PA, not Physical Availability (PA) untuk …

Over-merging is worse than under-merging, and the asymmetry is not close. An under-merge costs one extra LLM call and one extra review row. A wrong merge silently destroys a distinct term and the expert never sees it — there is no downstream mechanism that recovers it.

Legend extraction must run before clustering. Without the abbreviation index, PA and Physical Availability cluster as two unrelated terms.

Stop surfaces — dropped as whole surface forms only (never as substrings, so no term is lost): unit · type · class · equipment · equipment unit · parameter · activity · data · nilai · proses · hasil · total.

Normalisation (clustering only, never for span checking): NFKC → casefold → -/_ → space → strip .’' → non-word to space → collapse whitespace → strip ()/.

---

3. Evidence ranking

Six signals plus one penalty. The full ranked list is retained, not just the top K — escalation needs the tail.

Signal	Weight
Definitional cue within 100 chars of the mention	+5.0
Term appears in the chunk heading	+4.0
Mention sits inside a legend block	+3.5
Chunk contains a formula	+2.0
Mention is bold/italic	+1.5
First occurrence of the cluster	+1.0
Chunk is tabular	−3.0

Constant	Value	Why
evidence_k	5 (raised from 3, 2026-09-10)	Chunks fed per extraction round
Cue proximity	100 chars	Distance from mention to a definitional cue
evidence_link_partners	1 (added 2026-09-11)	Linked partner chunks pulled into the window on top of the ranked K

Linked partners (2026-09-11)

Schema 0.4.0 pairs a table chunk with the prose that introduces it, via referenced_by in both directions. top_k now pulls at most one partner into the evidence window.

Why it is not redundant — the measurement that decided it. Ranking only, on the three v2 artifacts and their recorded filter payloads (free; no GLiNER, no spend):

	clusters	top-5 touches a linked pair	both sides already there	only one side
BUMA	83	31	0	31
Komatsu	93	55	0	55
Open Pit	427	70	0	70

Zero clusters out of 603 saw both sides. The ranker never co-selects a table and the prose that names it — and that prose is frequently the table's only title ("Troubleshooting of engine (S-mode)" above a bare S-1 | 6 grid). Cost at the cap: evidence read +14.5% / +24.5% / +9.9%, which is +1.6% / +4.1% / +4.2% of total prompt spend; Open Pit moves 71% → ~74% of the 2M ceiling.

Two placement rules, both load-bearing. The expansion lives in top_k, not rank_evidence:

1. The ranked list is never mutated, so everything reading evidence_chunk_ids sees the number it saw before.
2. evidence_block (the prompt) and evidence_text (the span check) both take their ids from the same top_k call, so what the model reads is exactly what the span check searches stays true. Expanding in the prompt renderer instead would null every quote from a partner as unlocatable. Verified across all 603 clusters: 0 parity failures.

Scope, stated honestly. The document this helps most (Komatsu, 55 of 93 clusters) is the one that returns 0 definitions for genre reasons. This is groundwork for the table branch (Step 8) rather than a glossary win, and it has not been measured against a paid run — with this corpus's noise a single run could not have shown anything either way. Set EVIDENCE_LINK_PARTNERS = 0 to disable.

Heading matching is word-boundary, never substring. PA is a substring of Parameter, pada and composite — substring matching handed the +4.0 heading bonus to almost every Indonesian heading and pushed real definition sections below formula tables.

Why this filter is legitimate when the relevance gate was rejected: no term is dropped (only its evidence is narrowed), every mention stays in provenance so a bad top-3 is visible, and the full ranked list is retained so the tail stays reachable. The rejected gate had none of those three properties. Preserve all three if ranking is ever revisited. (The third property used to be phrased as "the escalation loop self-corrects"; escalation was retired on 2026-09-11 with MAX_ESCALATION_ROUNDS = 0, but retaining the tail is what made the property true and it still holds.)

---

4. Chunking

Chunking stays section-based. Page-based grouping was tested on 2026-09-02 and did not improve term recall — it lowered it, and the terms it recovered were offset by ones it lost to span-overlap competition. Full numbers and the reasoning in §7. Do not re-propose it without new evidence; the lever it was reaching for is in filters/span_filter.py, not here.

Constant	Value	Why
Max chunk tokens	1500	With paragraph-boundary splitting; sentence boundaries as fallback for a single over-budget paragraph
Max heading length	90 chars	Longer lines are sentences or formula rows, not headings
Boilerplate frequency	≥ 0.6 of pages	A line repeating on 60%+ of pages is a running header/footer. Detected by frequency with digits normalised to # so page counters collapse — never by a hardcoded document-specific string
Tabular detection	short lines > 0.6 and numeric lines > 0.4	Feeds the −3.0 ranking penalty

Three things the real document forced, all of which corrupt every downstream number if missed:

1. Running headers/footers — every page repeats the title block and "Confidential".
2. Breadcrumb headings — pages re-print 2. PENJELASAN PARAMETER / 2.1. Production Parameter / … at the top. A naive splitter re-opens the section and shatters it. Rule used: a heading that is the current section or an ancestor of it, already seen, is a breadcrumb — the section continues and page_end extends.
3. Colon-continuation — legend rows arrive as MOHH then : Machine on Hand Hours on the next line. Rejoined before the legend filter sees them.

Points 1 and 2 apply to MinerU output too: its title blocks repeat the same breadcrumbs on pages 2–8 of the standard.

---

5. Extraction (the paid stage)

Constant	Value	Why
Temperature	0.0	And still not deterministic — see below
Seed	7
Cache minimum	1024 prompt tokens	OpenAI-family prompt caching does not engage at all below this. The glossary prefix is padded past it on purpose. Measured hit rate 54% — 125,184 of 155,313 prompt tokens
Structured output	json_schema, fallback json_object + validate-retry	Which mode was used is recorded in the run manifest
API version	2024-12-01-preview	json_schema needs ≥ 2024-08-01-preview

Prompts live in files, not code, and the fixed prefix must stay byte-identical across calls — any drift and caching stops engaging, silently, at ~10× the input cost.

temperature=0 is not determinism. Consecutive runs of the identical pipeline scored 0.75 and 0.625 on schema fill. Never report a single run as a measurement on a small base.

---

6. Validation

Constant	Value	Why
Span normalisation	whitespace only (+ NFKC)	Not case, not punctuation, not diacritics. Every additional normalisation is a hole a fabrication fits through
LaTeX normalisation	notation-aware canonical form (+ NFKC)	Whitespace removed entirely, LaTeX spacing commands (~, \,, \;, \quad, \ ) dropped, presentational wrappers (\mathrm{}, \text{}, \mathbf{}, \operatorname{}) unwrapped, \cdot→\times, \%→%, \frac{A}{B}→frac(A)(B), remaining braces dropped. A different notion of equality, not a relaxation — the prompt asks the model to transcribe into LaTeX, which is a translation, so identity is the wrong test. Structure is preserved on purpose: dropping braces outright would make \frac{a}{bc} and \frac{ab}{c} normalise alike. Applies only to formula_latex against Chunk.latex — never to prose, where deleting whitespace would make “no data” match “nodata”
Escalation rounds	max 2	Then keep the entry flagged no_definition_found and pass it to review anyway — a term we found but could not define is still useful; dropping it hides a known unknown
Conflict overlap threshold	0.4 token overlap	Token overlap, not embeddings: cheaper, needs no model, and explainable to the reviewer who has to act on it
Duplicate overlap threshold	0.8	Above it, a differing definition is a duplicate; below, a conflict

Guarded fields (span-checked; failure sets the field to None and logs it):

Branch	Fields
glossary	definition, full_name, formula_latex, interpretation
rule	statement, condition, consequence
formula	formula_latex — checked against Chunk.latex, not text
summary	title, purpose_verbatim — changed 2026-09-02, was (none)

If the provenance span itself is not verbatim, every guarded field on the entry is rejected — the entry's only evidence link is broken, so nothing on it can be trusted.

The summary branch became guardable by changing the question, not the model (2026-09-02). It was the one exception here: it asked for a summary, and a summary cannot be located in the source by construction, so guarding it would have rejected every correct answer. It now asks the model to locate the document's own statement of purpose — purpose became purpose_verbatim — and a quote is checkable like any other transcription. Both fields are in _TRANSCRIBED as well as GUARDED_FIELDS; the formula_latex story below is why that distinction matters. Measured on the reference document: 0 rejections on a real run, and a deliberately fabricated purpose is refused. This also weakens the case that the branch needs a larger tier (D3 / risk R5) — locating a sentence is a much smaller ask than composing one.

formula_latex is checked against the source markup, not the rendered text (2026-08-26). It was previously guarded in name only: it sat in GUARDED_FIELDS but not in _TRANSCRIBED, so the LaTeX value itself was never searched — only its provenance.span was. A structurally wrong transcription with a valid prose span passed, which is exactly the shape X21 produced (a \frac that lost its division reads as a product and states something the document does not).

The fix could not simply add the field to _TRANSCRIBED: text carries rendered prose and can never contain LaTeX, so that would have nulled every formula entry and presented as a model failure. Chunk.latex — which the parsing half has always emitted, and which extraction's adapter used to drop on the floor — now crosses the seam and is the haystack instead, compared with whitespace removed entirely (see the normalisation row above). Found by Sofhia; verified on the real MinerU shapes.

An artifact carrying no latex fails open, keeping the old provenance-only guard rather than rejecting. “Cannot verify” must not present as “failed verification” — the alternative silently voids every formula entry on every artifact written before this landed. The consequence to know: on such an artifact formula_latex carries the weaker guarantee, and nothing in the output says which one applied.

Identity was still the wrong test — corrected 2026-08-26. Sofhia ran the first version against real artifacts and found Production = MOHH \times Qty \times … rejected. Probing further: on a source that does carry \times, the guard also rejected a model that dropped \mathrm{}, used \cdot, or wrote idiomatic LaTeX — 4 of 5 plausible correct outputs. It passed only when the model mirrored MinerU's notation character for character. The first version's own passing test passed only because the fixture was hand-written to mirror the source.

That is the same category error as the original defect, one level up: first the haystack was the wrong format, then it was the wrong notion of correctness. The fix is the notation-aware form in the table above. It still catches X21 — a lost \frac is a structural difference, not a notational one.

The pipeline backend cannot express multiplication, and that is reported rather than guessed. pipeline writes Production = MOHH x Qty x PA — a bare letter x, no \times, no \mathrm, no signal at all (§12, Still open). A claim spelling out the operator is therefore unprovable rather than wrong, and takes the same fail-open path as an empty latex. Without that, every product formula on every pipeline artifact would be nulled silently and read as a bad model — and every artifact on Sofhia's machine is pipeline, because she has no GPU.

Resolving x to multiplication inside the guard was rejected: it would silently settle a decision §12 deliberately leaves open, and would let Ma\times Capacity match a source reading MaxCapacity. The narrow exception covers a missing operator only — a claim that also loses its division is still rejected, and there is a test for exactly that.

\mathrm{x} is treated as a multiplication sign, before the wrapper unwrap that would destroy the distinction. That is grounded: \mathrm{x}100\% is the real PA formula's tail in this document.

Measured against real markup, 2026-08-26. Sofhia supplied the same 11 equations as MinerU transcribed them under four backends (44 strings, committed at eval/knowledge/fixtures/latex_backends.json). Agreement with hybrid/high, the production backend:

Backend	Whitespace-only	Notation-aware	After the evidence-led additions
vlm/medium	9/11	11/11	11/11
hybrid/medium	—	10/11	11/11
pipeline	5/11	8/11	10/11

The additions came from the inventory, not from guesswork: \mathsf (equation 7) and \displaystyle (equation 4) occur in this corpus and were missing; the array alignment spec ({l} from one backend, {r} from another) and a trailing row separator before \end{array} are presentational. A separator BETWEEN rows stays structural. \mathbf, \mathit, \textrm and \operatorname were guesses and never occur — kept because they are the same class and cost nothing, but their presence is not evidence of anything.

The single residual, pipeline equation 1, is Production = MOHH x Qty x PA x UA x Pty: the operator is genuinely absent from the source, so it is the marker's job, not the normaliser's. It is reported unverified_operator, which is the designed behaviour.

X20 same-day A/B: it costs a rule (2026-08-27)

The first controlled comparison in this project. Arms alternated within one session so any drift hit both equally; arm B reverts X20's three edits and keeps X23's synthetic examples, so the only variable is X20. Rule branch only, 15 calls per run, 5 rounds each. Served by gpt-5.4-nano-2026-03-17.

Arm	Runs	Result
A — X20 in	5	5/14 every time
B — X20 reverted	5	6/14 every time

Zero variance in both arms, and exactly one rule differs: GAINLOSS_QTY_ZERO_PLAN, 0% with X20 and 100% without it. X20 is a net −1 as written.

The mechanism is identifiable, and it is X20's own wording. That gold rule's fragments are QtyP(M) = 0 and QtyA(M) > 0 — a genuine conditional whose CONDITION is written symbolically (“Jika (Qty_P(M) = 0, Qty_A(M) > 0), maka: …”). X20 closes with “a statement that is mostly symbols, = and numbers is an equation”, which instructs the model to null precisely this. The heuristic cannot tell a computation from a symbolic condition.

And X20 is not buying what it was written to buy — today. GAINLOSS_UNITS, the rule it targets, fires 0% in BOTH arms. Yesterday it fired 6/7 with X20 in place. So X20's apparent win was the day, not the prompt (see the cross-day section below), and with today's model X20 has no measurable upside and one measurable cost.

REVERTED 2026-08-27, on Rifqi's decision. The carve-out idea (jika / apabila / maka → a rule however symbolic its condition) was considered and not taken: X20 had a measured cost and no measured benefit, and refining it would spend more runs defending a change that has yet to earn any. The intent — stop the rule branch duplicating the formula branch — remains valid and is available to anyone who wants to attempt it properly.

The shipped prompt is byte-identical to arm B, verified by hash. That matters for provenance: the state now in main is the exact state measured at 6/14 across 5 same-day runs on gpt-5.4-nano-2026-03-17, not a reconstruction of it. No re-run was needed and none was done.

Scope of that number: arm B measured the rule branch only (15 calls). It does not carry E1, E2 or E3, and no full-run result file reflects the reverted prompt — every committed full run on 26–27 Aug had X20 in. Do not read one of those as the current configuration's baseline.

Cache floor survived the revert: rule.txt is 1097 estimated tokens, still clear of 1024. X20 was what first pushed it over, so this was worth checking rather than assuming.

Method note worth keeping: rule-branch-only runs cost 15 calls against 82 for a full run and take ~35s. A/Bs on this branch are cheap enough that no prompt change to rule.txt should ever again be argued about instead of measured.

⚠️ Scores are not stable across DAYS (2026-08-27)

The most important measurement fact found so far, and it invalidates casual cross-day comparison.

GAINLOSS_UNITS scored 6/7 on 2026-08-26 and 0/12 on 2026-08-27 — 3 eval runs, 1 CLI run and 8 isolated repeats of the single call. Every input was proven identical:

• only ::0016 contains the gold fragments (BCM/ton/jam), so no other chunk could ever have scored it;
• that chunk's candidate is byte-identical with and without the day's cue change — same cue (Catatan), same offset (255), same snippet, verified by building both candidate sets;
• rule.txt was untouched between the two days;
• 0 span rejections, so the field is a genuine model abstention, not a guard kill.

Same input, same prompt, same deployment name, temperature=0 — 6/7 one day, 0/12 the next. The cause is outside this repo (a server-side model or deployment change is the obvious candidate) and cannot be determined from here.

What follows from it, and it is not optional:

1. A committed result file is a measurement of a day, not of the pipeline. Comparing today's run to a file from last week silently mixes in whatever the model did in between.
2. Any A/B must re-run BOTH arms on the same day. Yesterday's median is not a control.
3. temperature=0 was never determinism (§5 already said so within a day); it is not even stability across days.
4. Borderline judgements move most. ::0016 is a tabular chunk whose only prose sits in a Catatan: line — exactly the case rule.txt asks the model to weigh formula against prose. Stable rules (ECA_OTHER_ACTIVITY, QTY_ACTUAL_ZERO_PLAN_ALT) did not move at all.

This retracts yesterday's “X20 confirmed” reading. X20 was called confirmed because GAINLOSS_UNITS — the rule it names — fired 6/7. That rule now fires 0/12 with X20 still in place, so the 6/7 measured a day, not a fix. X20 may still be correct; it is no longer demonstrated, and showing it needs a same-day A/B against a reverted rule.txt.

Rule-branch noise band, and what it revealed (2026-08-26)

Seven runs of the identical pipeline on the same artifact, temperature=0. Run 1 carried the latex-guard defect; it is included because that defect cannot touch the rule branch.

Rule recall	Runs
3/15	1
4/15	5
6/15	1

min 3 · median 4 · max 6 · mean 4.14. The branch is far less noisy than one pair of runs suggested — 5 of 7 land on exactly 4/15. Never quote a single run; quote the median and the band.

Per-rule stability is the useful view, not the total. Three rules fire in 7/7 runs (GAINLOSS_QTY_ZERO_PLAN, QTY_ACTUAL_ZERO_PLAN_ALT, ECA_OTHER_ACTIVITY). Two fire in 1/7 (QTY_PER_ACTIVITY, UA_BY_ACTIVITY) and are the whole of the variance. Nine never fire at all.

X20 is demonstrated, not assumed. GAINLOSS_UNITS — the single rule X20 was written to recover, the units sentence that shared a chunk with a Gain/Loss equation — fires in 6 of 7 runs against 0 at the 3/15 baseline. That is a stable +1, outside the noise, and it is the rule the fix names.

X23's gold question is answered by measurement. WATERFALL_TWO_METHODS fires 0/7 with the few-shot that forbade it removed. The prompt was never the binding constraint: the model judges that sentence descriptive on its own. The gold entry is unreachable with this prompt and model, so the remaining decision is about the gold, not about the prompt.

The real ceiling is upstream, and it is not the prompt. Traced properly on 2026-08-27, which corrected a wrong first reading — an earlier version of this section blamed tabular chunks. It is not tables: 5 of the 8 candidates already come from tabular chunks, so the cue regex reaches them fine. Three separate causes, and only one is the prompt's business:

#	Cause	Costs	Fix lives in
1	Two gold rules have no matching text in ANY chunk — PTY_PRODUCTION_SOURCE, PTY_WORKING_HOURS_DEFINITION	2 rules, permanently	The gold set, or the parse. Not extraction
2	One call returns ONE rule, but a chunk can hold several. ::0006 alone holds 4 gold rules	3 rules	Schema/call unit — a list instead of a single object
3	Four prose chunks are never proposed as candidates (::0008, ::0011, ::0017, ::0024)	4 rules	The discourse-cue regex, on prose

Arithmetic: 13 reachable gold rules live in 9 chunks, and one-rule-per-call caps the branch at 9/15 even with perfect candidates and perfect extraction. Against a median of 4/15, the headroom is 5 rules — and none of it is reachable by tuning rule.txt.

E3 was 1.00 in all seven runs, on a scoreable base of 6–7. Zero variance: when nano answers, it answers correctly; the open question is how rarely it answers, not whether to trust it.

What the LaTeX normalisation assumes, and where it stops

Audited 2026-08-26 against cases the corpus does not contain, because every fixture here is BUMA and a suite built only on BUMA cannot catch a rule that is really a fact about one document.

Held, and these are the ones that matter. Different operands, precedence (\frac{a+b}{c} vs a+\frac{b}{c}), operand order, sub- versus superscript, and + versus - all stay distinct. The guard's discrimination follows from preserving structure, not from anything about this corpus.

Two deliberate over-normalisations. Neither is safe to assume elsewhere.

Assumption	Why it is accepted here	When it breaks
\mathrm{x} is a multiplication sign	It is how MinerU transcribes × in this corpus. Applied to both sides, so a genuine roman x converts identically in claim and source and they still match	A corpus where x is a real variable name — algebra, statistics. Residual risk is a contrived false acceptance, never a false rejection
Whitespace inside \text{} is deleted	In text mode a space is a real word boundary, so \text{Total Hours} and \text{TotalHours} compare equal	Two labels differing only by internal spacing. Tolerated because a fabrication differing from the truth by one space is not a threat worth a mode-aware scanner — but it IS lost discrimination, not a neutral simplification

\cdot is deliberately NOT folded into \times. It was, speculatively — it appears nowhere in the 44 real strings — and on vectors the two are different operations. The cost is real and pinned by a test: a model writing \cdot for scalar multiplication is now rejected. If that shows up in a live run, report it the way unverified_operator is reported; do not re-fold the two.

Known incompleteness, all in the fail-safe direction (a false rejection, never a false acceptance): plain-TeX {a \over b} is infix and unhandled; \mathbf, \mathit, \textrm, \operatorname, \quad and the thin-space commands are handled but never observed, so their presence in the code is not evidence that this corpus emits them.

One boundary bug worth remembering: \limits\b never matched before a subscript, because _ is a word character. That is the same defect X22 found in the eval scorer, in a different file. Where a LaTeX command can be followed by _, \b is the wrong boundary — use (?![A-Za-z]).

Which guarantee applied is recorded on the entry (latex_verification, added 2026-08-26). Three paths leave formula_latex populated — located in the markup, no markup to check against, operator not recoverable — and they are not equally strong. Until this field existed the output could not tell them apart, so a verified formula and an unverifiable one reached the expert as identical rows, and on a pipeline artifact most product formulas are the unverifiable kind. Set by the span check, never by the model; null when there is no claim to characterise. Values and per-field notes are in the output contract §7.

Never repair a failed span. A repaired span is an unfalsifiable claim, which is exactly what the control exists to prevent.

Conflict detection never picks a winner. The expert decides.

---

7. Negative findings — do not re-derive these

Finding	Detail
nano is not sufficient for schema fill	0.75 precision against a 0.80 kill line. Decision 2026-08-19: stay on nano anyway for now; the __54n env quad is provisioned. Revisit if quality blocks the demo
Escalation is unreachable on small documents	54 of 66 clusters had exactly 1 evidence chunk against K=3, so rounds_available() correctly returned 0 and the loop never fired. escalated = 0 is not a bug. It stays unverified until a larger corpus runs
Conflict detection had nothing to find	0 conflicts on a single internally-consistent standard. Untested against real disagreement
85% abstention	56 of 66 entries returned no definition. Correct behaviour, but it means the review queue is mostly "term found, no definition in document". Whether that is useful or noise is a review-experience question for Mas Beta
Diff only ever ran the empty-baseline path	The prototype diffed against the file it then overwrote, so every entry came back new. v2 needs a real active-version pointer
Literal source wording was normalised away	The standard's heading reads "Physical of Availability (PA)"; the entry carried full_name: "Physical Availability". The gold set records the literal form correctly, so gold was right and the pipeline was wrong. Mechanism: the heading is a separate field from the chunk text, and the chunk text does not contain the phrase, so the literal wording never reached the model. Fix: feed the heading verbatim + a span-checked source_wording field
A glob('*.pdf') picked the wrong document once	Scored the textbook against the standard's gold set → recall 0.05. Document selection must be explicit and error on ambiguity
Page-based chunking does not improve term recall (2026-09-02)	Tested directly: the 31 section chunks of the BUMA fixture regrouped into 9 page chunks, everything else held constant. It fixed the structural symptom — chunks under GLiNER's 130-word window went 28/31 → 2/9 — and E1 fell 0.8293 → 0.8049 (34/41 → 33/41). The net −1 hides the real result, which is a swap of 7 terms: page-grouping recovered Fleet management, Mineplan, Uncontrollable (the terms the short-context theory predicts) but lost Coal barging, Model Unit, Standby, Total Hours, and the filter's dropped_overlapping rose 7 → 16. So chunk size is not the binding constraint — span-overlap resolution is, and that lives in filters/span_filter.py, not in the parser. Read this as no evidence page-based helps rather than page-based is worse: n=1 document, 41 gold terms, and a 1-term delta is inside the noise band §6 documents. Runs: results/v2_filters_2026-09-02_092638.json (control) and _092714.json (experiment)
torch wheels install corrupt	torchgen missing, version reports None. Fix is delete site-packages/torch* and reinstall. Needs torch ≥ 2.6. Documented so nobody debugs it as a GLiNER problem

---

8. What was measured, so v2 can be compared

Frozen in eval/knowledge/results/baseline_prototype_2026-08-13_145132.json.

Funnel on the 9-page standard: 9 pages → 13 chunks → 169 mentions (195 before noise filtering) → 66 clusters → 83 LLM calls → 66 entries → 66 review-queue items.

E1 recall 0.854 · E2 compression 2.56× · E3 schema-fill precision 0.75 (FAIL) · E4 fabrication rejection 1.00 with 0.00 false rejections. Cost $0.0069, 242 s wall clock, 3 fields rejected by span check.

When quoting E2 externally: the context doc's 6.2× is a 1,000-page projection; 2.56× is the 9-page measurement. Compression grows as terms repeat across a corpus. Both are true — use the right one for the audience.

---

9. Deliberate deviations from the prototype (v2)

Changes made while rebuilding, with the reason. Each one alters behaviour, so each is listed rather than folded silently into the tables above.

#	Change	Why	Measured?
V1	A heading that names the term counts as a mention at offset 0 for cue-proximity scoring	Indonesian standards are written "2.1.3 Physical of Availability (PA)" / "Adalah ketersediaan fisik…" — the heading names the term and the body opens with the definitional cue without repeating it. The prototype therefore awarded no cue bonus on the definition section itself. Measured on the BUMA standard: PA's own definition section (§2.1.3) scored 7.5 and ranked third, below a computation section (§2.2.4, 8.5) that merely mentions PA often. After the fix, all five terms rank their own definition section first — PA→2.1.3, UA→2.1.4, Qty→2.1.2, Pty→2.1.5, MOHH→2.1.1	Structurally verified on the 9-page standard. Not yet scored — needs the paid stage to show up in E3
V2	first_occurrence follows document order, and ranking ties break on document order	The prototype awarded the +1.0 to whichever chunk a set iteration yielded first, making the bonus arbitrary and the ranking unreproducible across runs. "First occurrence" should mean the term's first appearance in the document	Determinism locked by test; verified stable across PYTHONHASHSEED 0/1/7/42/99
V3	source_wording field added to the glossary entry	The control for the literal-source-wording defect (§7). Not yet populated — needs the seam to carry the heading verbatim and the extraction prompt to fill it	No

Both V1 and V2 change which chunks reach the paid stage, so they can move E3 when the extraction branch lands. Re-score against eval/knowledge/results/baseline_prototype_2026-08-13_145132.json at that point and record the delta rather than assuming the improvement carries.

V1 and the R1 literal-wording defect share a root cause: the heading is a separate field from the chunk text, so anything that reasons over "the term and its definition together" has to be told to look at both.

| V4 | source_wording is taken from the section heading, not from the model | Asked to quote the source wording, the model returned "Physical Availability (PA)" — a genuine verbatim quote from the page-1 intro, but not the §2.1.3 heading "Physical of Availability (PA)". Both occur in the document; only the heading form reveals the disagreement. Taking it deterministically means it cannot be normalised away | Live-verified on the pilot: the entry now carries both forms and the queue flags the mismatch | | V5 | evidence_text includes chunk headings | The heading is part of the source document and is often the only place a term is formally named. Excluding it rejected a correct verbatim quotation of the document's own section title (2 rejections on a 3-entry pilot, both spurious) | Live-verified: 0 rejections after the change | | V6 | full_name and source_wording are span-checked against the source themselves | Both claim to be literal transcriptions. The provenance span passing does not prove the transcription is faithful, and a normalised "full name" is exactly the silent correction this pipeline exists to surface | Unit-tested |

Cache-floor finding (measured on the real prompts): only the glossary prefix clears the 1024-token floor at 1,401 tokens. rule (731), formula (459) and summary (288) are not cacheable and cache nothing today. Glossary is also the branch with by far the most calls (66 of 83 on the reference document), so the padding is on the branch that matters — but padding the other three is free money if their call counts ever grow.

---

10. Parser comparison — MinerU vs PyMuPDF (2026-08-19)

First run of the extraction half on MinerU-parsed input, same document, same gold set, same pipeline code. Result: eval/knowledge/results/v2_parser_comparison_2026-08-19_152842.json.

Input	E1 recall	Terms found
PyMuPDF (what the frozen baseline used)	0.8537	35 / 41
MinerU, raw markup	0.7561	31 / 41
MinerU, formulas + tables rendered to prose	0.8293	34 / 41

The span model reads prose, not markup. MinerU's structure is an asset for the formula branch and a liability for the filter unless it is rendered first:

• Terms wrapped in LaTeX are invisible. \text {Total Hours} - \text {Breakdown} yields no mentions for either term, though both are gold terms and both are found in the plain-text parse. Unwrapping \text{}/\mathrm{}/\frac{}{} and collapsing MinerU's character-spaced output recovers them.
• Raw <table> HTML in the chunk text is noise the model has to read past. Eight of fourteen chunks carried it before rendering.

The residual ~2.4 points are chunking, not content. MinerU marks non-section labels — Keterangan:, Keterangan grafik: — as text_level titles. Opening a new section at each one splits a legend away from the figure it explains, and Other Activity / Uncontrollable are then missed despite being present as prose in a chunk the filter processed. Only numbered headings should open a section, which is consistent with §9's finding that MinerU flattens every heading to level: 2 — numbering is the only reliable section signal it gives us.

Caveat on attribution: the MinerU artifacts here were built by a scratch converter, because section-aware chunking belongs to the parsing half. These are findings about what that chunker must handle, not a measurement of MinerU itself.

Re-measured through the production module — B3 (2026-08-26)

The caveat above is now discharged: src/knowledge_parsing produced the artifact, not a scratch converter. eval/parsing/run_eval.py, result in eval/parsing/results/.

Input	E1 recall	Terms
PyMuPDF — frozen baseline	0.8537	35 / 41
MinerU hybrid/high, production module	0.8293	34 / 41
scratch converter (above)	0.8293	34 / 41

Parity with the scratch converter, from code that ships. The miss lists differ by one term in each direction — the module finds Other Activity, which the converter missed, and misses Breakdown, which it found. The other six are identical.

Getting there required fixing a real defect. The first run scored 0.7073, and the cause was not MinerU: normalize.py silently dropped every type: "list" item. MinerU keeps list content in list_items and gives those items no text field, so they fell through the generic text path, came back empty, and were skipped with no warning — the same shape of bug the chart/image branch was written to prevent. Two dropped blocks took five gold terms with them (Joint survey, Truck count, EWH, Controllable, Hierarki Level Analysis). Fixing it moved recall +0.1220.

Note what this says about §10's own conclusion: the scratch converter did handle list blocks, so Uncontrollable really was "present as prose but missed" in that run, exactly as recorded. §10's attribution was right. The production module had regressed against it, and B3 is what surfaced that.

The residual 0.0244 is not parsing's to close. Breakdown appears three times in the text and sits in the same chunks as Total Hours, which the filter does find — same chunk, same sentence, ( Total Hours - Breakdown ), one tagged and one not. Chunk arrangement cannot explain it, so it is filter behaviour. Chunk counts confirm there is no over-splitting to blame: 31 chunks = 15 prose (11 headed, 9 distinct sections) + 14 tables + 2 images, against the converter's 14 prose chunks. On prose granularity the two agree.

Trap for anyone re-running this. A harness that re-normalises a cached content_list must pass the page vocabulary, as run.py does. Without it, word-boundary recovery has no arbiter, INPR Hours fuses into INPRHours, and the scorer's word-boundary match fails — costing a full term. That mis-measurement (0.8049) was produced and discarded during B3; the module was never at fault.

---

11. Full-document run, both halves (2026-08-24)

First run with the parsing module building the artifact and extraction running every branch. Result: eval/knowledge/results/v2_full_document_2026-08-24_093051.json.

Metric	v2	Prototype	Verdict
E1 term-filter recall	0.8049	0.8537	PASS (kill 0.70)
E3 schema fill	0.90	0.75	PASS (kill 0.80) — the prototype FAILED this
Rule-branch recall	0.0667	0.2667	REGRESSION

77 calls, 139,283 prompt tokens with 110,848 cached (80%), 2 fields rejected by the span check, 48 of 58 entries abstained.

Read E3 with the same care the prototype's number needed. The scoreable base is 10 entries, so one adjudication moves it ~10 points, and nano is not deterministic at temperature=0. The single failure — produksi — is the same one the prototype recorded, and looks like a gold-adjudication case rather than a model error: the document carries a broader definition than the gold set names.

E1 is below the baseline because the chunking differs, not because the filter regressed. The parsing module splits to a token budget (31 chunks) where the earlier comparison used 14, so the span model sees different windows. Same code, same gold set.

The rule branch went backwards and is not yet diagnosed (checklist X19). Candidate generation is not the cause: 8 cue passages against the prototype's 9.

Bug this run surfaced — blank lines closed the legend block

The legend filter returned zero abbreviation pairs on the parsing half's output. Cause: that module separates blocks with a blank line — ordinary prose formatting — and find_legend_lines treated the first blank as the end of the block.

The consequence is exactly what the filter exists to prevent: PA and Physical Availability clustered as two unrelated terms and reached the expert's queue as two rows. Fixed by skipping blank lines and closing the block on two consecutive blanks.

	Before	After
Abbreviation pairs	0	5
Clusters	61	58
LLM calls	81	77
Cache hit rate	57%	80%
PA rows in the queue	2	1

The lesson generalises: the two halves agree on the contract, and this still slipped through — a formatting choice that is completely reasonable on one side silently broke a regex on the other. Whitespace is part of the interface whether or not the schema says so.

---

12. Rule-branch regression — diagnosed (2026-08-24)

The full run scored the rule branch at 1/15, against the prototype's 4/15. Diagnosed to three separate causes, only one of which is an actual quality loss.

Cause 1 — two of the three losses are a scoring artifact, not a regression

The gold set's statement_contains strings were written against the PyMuPDF-era notation, which ran subscripts together: QtyA, QtyP. The rendered LaTeX writes them with an underscore: Qty_A, Qty_P. The scorer normalises punctuation but _ is a word character, so qty_a never contains qtya and the match fails on notation alone.

Scoring	Rules matched
As committed	1/15
Collapsing _ to nothing	3/15

GAINLOSS_QTY_ZERO_PLAN and QTY_ACTUAL_ZERO_PLAN_ALT are both correct extractions scored as misses. The true comparison is 4/15 → 3/15, a loss of one rule. Fixing this means changing either the scorer or the gold set — both are the frozen measuring instrument, so it is Rifqi's call, not a silent edit.

Cause 2 — the real loss: rendered formulas now compete with prose

4 of 8 v2 rule statements are bare equations. In the prototype, 0 of 7 were.

Run	Formula statements	Prose statements
v2	4	4
Prototype	0	7

Before rendering, formulas reached the model as mangled character-spaced LaTeX and did not read as statements. Rendered to clean prose they read exactly like prescriptive rules, and the model takes them. rule.txt never excludes them — it defines a rule as "a statement that constrains or directs behaviour", which a formula arguably is.

The lost rule shows the mechanism exactly: chunk ::0015 contains both the units sentence ("Gain/Loss menggunakan satuan BCM atau ton…" — gold GAINLOSS_UNITS) and a Gain/Loss equation. The chunk was a cue candidate, so the model saw it, and returned the equation.

This is pure loss, because the formula branch already captures those equations — Qty = (I NPRHour s) / (T otalHour s) appears in both branches of the same run. The rule branch is spending calls duplicating the formula branch and dropping the operational prose it exists for.

Fix direction: rule.txt should exclude bare equations explicitly ("an equation is the formula branch's job; a rule is prose that constrains behaviour"). That is a prompt change and therefore owes an eval run per CLAUDE.md §7B — not to be done on vibes.

X20 — the fix, applied 2026-08-26 (296cdd7). NOT YET EVALUATED.

Three edits to rule.txt:

1. A rule is now prose, not "a statement", with a paragraph stating that an equation belongs to the formula branch and that "calculation policy" means prose saying how, when or on what basis a calculation applies — never the calculation itself.
2. New correctness requirement 3: evidence whose only prescriptive content is a formula returns statement: null.
3. The load-bearing clause: evidence carrying both a formula and constraining prose yields the prose. This is the one that targets the measured loss — chunk ::0015 held both the units sentence (gold GAINLOSS_UNITS) and a Gain/Loss equation, and the model returned the equation.

Worked example 3 is deliberately synthetic (MTTR, absent from all 15 gold rules). Using the real GAINLOSS_UNITS case would have leaked a gold answer into the prompt and made the eval score it trivially. Worth knowing: worked example 1 already is gold rule PTY_PRODUCTION_SOURCE verbatim, so that contamination pre-exists — the new example was kept clean rather than adding to it.

Committed without the eval §7B requires, on Rifqi's explicit call, because the dev box has no source document, no parsed artifact and no MinerU install (data/knowledge_docs/, data/knowledge_cache/, data/knowledge_runs/ are gitignored and absent; there is also no eval/knowledge/run_eval.py — the 2026-08-24 run was assembled ad-hoc). Restoring one parsed artifact unblocks the measurement. The measurement is still owed.

Compare against 3/15, not 1/15. X22 landed on 2026-08-24 (abae344) after this regression was diagnosed: the scorer now deletes underscores, so the notation artifact in Cause 1 is gone. The 1/15 in v2_full_document_2026-08-24_093051.json was measured with the old instrument and is no longer the number to beat.

Side effect to verify on the first paid run. rule.txt grew from ~731 to ~1140 estimated tokens and crossed the 1024 prompt-cache floor, so cacheable('rule') now returns True for the first time — the "free money" §9 predicted. But est_tokens is len/3.6; only the API's cached_tokens proves a hit.

Found while doing this, not actioned — the rule branch cannot score better than 14/15 today. Worked example 2 teaches that "Waterfall Analysis dapat di-review melalui dua metode" is not a rule and must return null; the gold set records that exact sentence as WATERFALL_TWO_METHODS. The prompt and the frozen gold contradict each other, so one gold rule is unwinnable by construction. Both are measuring instruments — Rifqi's call, tracked as X23.

Cause 3 — three distinct render_latex faults (FIXED 2026-08-24)

Two corrections, both from re-measuring against the document's own 11 equations.

An earlier version of this section said render_latex "eats the first character". That was retracted — the repro was mangled by shell escaping, not by the code.

The retraction then over-corrected. It claimed Production = MOHH × Qty × PA × UA × Pty "renders perfectly". It does not. The document's raw content_list.json contains no \times and no \mathrm — just a literal x, spaced like every other character:

'P r o d u c t i o n = M O H H x Q t y x P A x U A x P t y'
  ->  'Production = MOHHxQtyxPAxUAxPty'

MOHH, Qty, PA, UA and Pty — the five terms the glossary is built around — fuse into one token. See Still open below.

(A) A spaced run touching a bracket is only half-collapsed. _SPACED_RUN was bounded by (?<!\S) / (?!\S), so a run could not match if it touched ( or ). The \frac{a}{b} → (a) / (b) rewrite inserts exactly those brackets, stranding a character at each end:

'I N P R H o u r s'      ->  'INPRHours'      # correct, no brackets
'(I N P R H o u r s)'    ->  '(I NPRHour s)'  # first and last stranded

(B) The subscript fold ran before the collapse and split the run. Q t y _ {a c t i v i t y} became Qt y_a ctivity — the _ glued mid-run, so the collapse saw broken fragments.

(C) The serious one — fraction structure was silently dropped.

\mathrm { P A } = { \frac { \mathrm { T o t a l } ~ H o u r s - B r e a k d o w n }
                          { \mathrm { T o t a l } ~ H o u r s } } \mathrm { x } 1 0 0 \%
  ->  'PA = Total Hours - BreakdownTotal Hoursx 100 %'

The \frac rewrite needed \{([^{}]*)\}, which cannot match a numerator containing nested braces. The unmatched \frac then fell through to the generic "drop any leftover command" pass, so the division was deleted rather than rendered. This is not a spacing blemish — the formula reads as a product and states something the document does not.

The nesting comes from \mathrm{…} and \sum _{ij}, not \text{…}. The BUMA standard never uses \text{}. A fix written against \text{} would repair a synthetic example and no real equation.

Measured tally, 11 equations:

	Before	After
Division lost	4 — eq 6, 7, 9, 11 (PA, PA_ij, UA_ij, PTY_ij)	0
Stranded characters	4	0
Clean	5	9 + 2 carrying only the document's own literal x

Not two equations and not only PA/UA: every composite (_ij) form was affected too.

The fix (src/knowledge_parsing/render.py):

1. Braced wrappers are unwrapped before \frac, consuming their braces. The old _WRAPPERS deleted only the command name and left { } behind — those leftover braces were what stopped \frac from matching. This is the ordering decision the diagnosis asked for.
2. \frac uses a balanced-brace scanner, not a regex. [^{}]* can never handle a nested argument, and it fails silently rather than raising.
3. Run boundaries are word-character classes, and subscript contents are joined at fold time (_ { i j } → _ij).

Six regression cases hold unchanged: C_p, C 5, C_p × T, \frac{a}{b}, Purchasing costs, and a bare spaced run.

Still open — a decision, not a defect. After MinerU spaces every character, a multiplication x is indistinguishable from the letter x: no \times, no \mathrm, no signal at all. Treating it as an operator is right for engineering formulas and wrong for a variable named x. The _SYMBOLS comment shows the author already weighed this trade for \times. Left unfixed deliberately, because it changes interpretation rather than mechanics.

Impact of what remains: a term occurring only inside Production = … cannot be separated from its neighbours, which costs the term filter five mentions on the standard's flagship formula.

Owner: Sofhia — src/knowledge_parsing/render.py.

---

13. Backend comparison — all four measured (2026-08-24)

Every backend, one Colab GPU, the same two documents (BUMA standard 9 pages + McGraw-Hill handbook 20 pages = 29 pages). Until this run, hybrid had never been measured at all, and every effort figure in circulation came from an older Colab session on different hardware.

Backend	s/page	Image descriptions	Multiplication sign
hybrid / high ⭐	4.85	3 · 1	\times
hybrid / medium	1.88	0 · 0 — force-disabled	\times
pipeline	2.67	0 · 0 — never analyses	letter x
vlm	6.96	3 · 1	\times

Formula transcription overlap, across the standard's 11 equations:

Pair	Identical
pipeline vs everything else	0%
vlm vs hybrid/high	100%
medium vs high	70%

Two findings decide the backend. vlm and hybrid/high are the same output — same LaTeX, same descriptions — so vlm buys nothing for its extra time. And pipeline is the only backend that copies the multiplication sign as a literal x, fusing MOHH x Qty x PA into one token and taking five glossary terms with it; it is also slower than hybrid/medium, so it loses on both counts.

The choice therefore narrows to hybrid/medium vs hybrid/high, i.e. whether image descriptions are wanted. Across the whole 6,800-page BUMA corpus that is $2.67 against $6.89 — a $4 difference, too small to trade the feature for. Decided: hybrid + effort=high.

Rifqi's claim that medium and high capture only ~30% of the same formulas did not reproduce — measured at 70%.

Image descriptions are gated by sub_type, and coverage is partial

MinerU only describes a picture it has first classified. The pattern held 6 of 6 across both documents:

sub_type	content
line · bar · flowchart	filled
absent	empty

Coverage was 4 of 6 (67%), and the miss is not explained by image quality: Gambar 2.1 is a plainly legible waterfall chart with seven labelled bars, and got no sub_type at all. Meanwhile Gambar 2.2 was labelled flowchart while the description it produced calls it "This waterfall chart…" — so the classifier is shaky in both directions.

Consequence for planning: the parsing stage cannot fully replace a dedicated image-description pipeline. It supplies some descriptions free, but a clear chart can be skipped with no warning. A fallback path is easy to target, though — chunks with images populated and generated_description null are exactly the gap.

Determinism — settled (2026-08-26)

hybrid/high is deterministic. Two runs over the same two documents, with the parse cache moved aside in between so both genuinely re-parsed (from_cache: false on all four records), produced 0 differences across 50 chunks — every one of chunk_id, kind, page_idx, section_no, heading, text, latex, table_html and generated_description identical.

That result is not vacuous: all four image descriptions were present in both runs and character-for-character identical (192, 256, 77 and 146 chars). Those are the only model-written content in the artifact and were the likeliest thing to wobble; they did not.

This closes the strongest objection to calibrating on the VLM family — that a frozen baseline would drift between runs and no eval number could be trusted. It does not. Existing baselines stay valid and need no re-measurement on this account, and the backend decision is final rather than provisional.

Parse time does vary — 167.4 s against 65.2 s for the same 20-page document, which is model loading on the first run. The output does not.

---

14. Word-boundary recovery — reading the answer from the source (2026-08-24)

MinerU writes formulas one character at a time (T o t a l H o u r s). Once those are joined the boundaries are gone: Total Hours and TotalHours become indistinguishable, and a multiplication written as the letter x fuses in too, because it looks like any other spaced character.

Two obvious fixes were tried and rejected. Rewriting a lone x to a multiplication sign guesses, and guesses wrong — silently — on any document with a variable named x; it also leaves TotalHours untouched, which is the larger half of the damage. Emitting a warning instead fires on 7 of 11 equations: a warning lighting up on 64% of items is not actionable and recovers nothing.

The source PDF still holds the boundaries — its text layer reads MOHH x Qty x PA x UA x Pty, properly spaced. So the answer is read from the document. A token not already a word on that page is split only if EVERY piece is a word on that page, and only if exactly one shortest such split exists. Ambiguity is left alone; so is anything unsegmentable. It fails closed.

NFKC normalisation is required — formulas use a mathematical font, so the page literally contains U+1D440 and friends, and without it nothing matches.

Measured across three documents, two of them an English textbook with nothing in common with the BUMA standard:

Document	Text-layer quality	Equations repaired
BUMA standard, 9 pages	99.6%	6 of 11
Handbook, 20 pages	96.0%	1 of 3
Handbook, 5 pages	98.1%	8 of 20

MOHHxQtyxPAxUAxPty  ->  MOHH x Qty x PA x UA x Pty
TotalHours          ->  Total Hours
newwrenchtime       ->  new wrench time
areaofroom, sqft    ->  area of room , sq ft

Glossary terms readable as whole tokens went 320 -> 397 on the standard, across 9 of 31 chunks. The second test document was unchanged — as it should be, having no such damage.

The guard that makes it safe on documents nobody has seen

Trusting the text layer is safe when it is intact and dangerous when it is HALF there: a legitimate word starts looking unknown and gets split. So each page has to prove itself — how many of MinerU's own prose words on that page are actually found in its text layer. A page scoring below 90%, or carrying too little prose to judge, loses its vocabulary and skips recovery: damaged but honest, rather than repaired into something the document never said.

That replaces a constant tuned against one document with a measure that assesses each page on its own evidence. A scanned PDF with no text layer is already safe — no vocabulary, no recovery. The untested territory is a text layer well below 96%, which none of the three documents provides.

Threshold measured rather than picked: a minimum token length of 6, 4 and 3 give identical results on equations, and 3 additionally recovers MOHHx / xPA_A inside table cells. 2 starts doing damage, splitting PA_ij into PA_i j because i and j are both valid words on that page.