Commit History
-
[NOTICKET] fix(knowledge_extraction): formula_latex is checked against Chunk.latex
Rifqi Hafizuddin +1: Claude Opus 5The field was guarded in name only. It sat in GUARDED_FIELDS but not in _TRANSCRIBED, so its value was never searched in the source -- only its provenance.span was. A structurally wrong transcription with a valid prose span passed, which is exactly the shape X21 produced: a \frac that lost its division reads as a product and states something the document does not. Adding the field to _TRANSCRIBED was not the fix. `text` carries rendered prose and can never contain LaTeX, so that would have nulled every formula entry and presented as a model failure rather than a validation bug. The haystack was the problem, not the check. Chunk.latex is now the haystack. The parsing half has always emitted it; extraction's adapter dropped it on the floor, which is what left the claim unprovable. It now crosses the seam and is compared with whitespace removed ENTIRELY rather than collapsed. That is a different notion of equality, not a weaker one. In LaTeX, whitespace outside \text{} is not significa…9c8a6e4
-
[NOTICKET] feat(knowledge_extraction): paid extraction stage + validate, diff, queue
Rifqi Hafizuddin +1: Claude Fable 5Completes every stage that does not depend on the seam. The pipeline now runs end to end: artifact -> filters -> cluster -> rank -> LLM -> validate -> diff -> frequency-sorted review queue. extract/ client on the nano __54n quad; structured output is PROBED not assumed (json_schema, falling back to json_object + validate-retry, recording which applied); cached tokens read from the API, never modelled. Four branches, one call per TERM CLUSTER. Prompts stay as files so the fixed prefix is byte-identical and caching keeps engaging. validate/ span check (whitespace-only normalisation, never repairs, unlocatable provenance rejects every guarded field), escalation, conflict detection by token overlap — explainable to the reviewer, and it never picks a winner. diff/ new / duplicate / conflicting against an explicitly supplied active glossary, so the duplicate and conflicting paths are reachable. The …ab5ea78