tools/_reextract_contract_v3.md

Re-Sourcing Contract v3 (2026-05-16) — exhaustive verbatim sweep

v2 fixed only cells whose source_quote matched a narrow provenance-note list. The hardened verifier proved ~947 cells still carry a NON-verbatim source_quote that slipped that list: heuristic/placeholder notes ("classified as X from PDF heuristics", "Default IRDAI 24-month", "limit: …", "(standard … per Policy Schedule)", "No mandatory copay extracted") and human paraphrases/summaries. v3 uses an OPERATIONAL definition — no pattern list — so nothing slips.

Qualifying rule (OPERATIONAL — applies to every value-bearing cell)

For each cell in your assigned insurers' 40-data/policy_facts/*.json with:

• a non-null value (ignore null/""/[]; ignore 999/9999), AND
• NOT max_renewal_age (skip — removed), AND
• NO source_url (url-sourced day_care/network are fine — skip), AND
• source_quote is NOT a "not stated …/image-only scan …" sourced-null,

→ open the cell's source_pdf_path PDF (PyMuPDF/fitz, column-aware) and TEST: does the source_quote actually occur in the PDF text (whitespace- normalised, case-insensitive — a clause is "present" if a majority of its 8-word shingles are in the text)?

• Present verbatim → leave it (already good).
• NOT present (paraphrase, summary, heuristic note, placeholder, inferred, "limit:…", "Default IRDAI…", "classified as…") → it qualifies; FIX it.

Fixing a qualifying cell (same as v2 rules)

1. Verbatim clause in the PDF supports the value → set source_quote to that exact clause (≤300 chars), keep value, _confidence high/medium.
2. PDF states a different value → correct value, with the verbatim clause.
3. Field genuinely absent from the PDF → value:null, source_quote:"not stated in <file>.pdf", _confidence:"low".
4. Source PDF image-only (<400 extractable chars) & no text sibling → drop: value:null, source_quote:"source document is an image-only scan; not text-extractable (no OCR available)", _confidence:"low". (If a text- bearing sibling doc for the same policy exists, source from it + update source_pdf_path.)

Hard rules

• NEVER keep a non-verbatim source_quote on a non-null value. NEVER fabricate, paraphrase, summarise, or infer a quote — copy exact PDF text only.
• NEVER invent a number. NEVER 999/9999.
• Edit ONLY assigned insurers' policy_facts files. No code. Valid JSON via json.dump(d,f,ensure_ascii=False,indent=2), key order preserved.
• An independent adversarial re-audit WILL re-open the PDFs — every quote must survive a fresh shingle/semantic check.

Output

{"insurers":["..."],"files_processed":N,"cells_checked":N,
 "already_verbatim_left":N,"reverbatim_fixed":N,"values_corrected":N,
 "nulled_absent":N,"dropped_imageonly":N,"anomalies":["..."]}