"""Pure PDF helpers — text, per-page text, page count, and bookmark outline. These are the format-agnostic PDF primitives the app builds on: they take raw PDF bytes and return text / structure, with no dependency on any agenda source. They were extracted from the (now-removed) ``civicclerk.api`` package when the app pivoted from the CivicClerk OData API to user-uploaded agenda-packet PDFs. Kept at the repo root (not under ``webapp/``) so both ``webapp`` and ``chroma`` can import them without a ``chroma -> webapp`` layering cycle. PDFium note: :func:`extract_pdf_outline` uses pypdfium2, which is **not** thread safe. Callers that run under a threadpool (the Gradio worker) must serialize every pypdfium2 call behind a single global lock — see ``webapp/backend._PDFIUM_LOCK``. """ from __future__ import annotations import io import pdfplumber def extract_pdf_text(data: bytes) -> str: """Extract text from PDF bytes using pdfplumber, joining all pages.""" parts: list[str] = [] with pdfplumber.open(io.BytesIO(data)) as pdf: for page in pdf.pages: text = page.extract_text() if text: parts.append(text) return "\n\n".join(parts) def extract_pdf_pages(data: bytes) -> list[str]: """Extract text from PDF bytes, returning one string **per page**. Like :func:`extract_pdf_text` but it keeps page boundaries (empty string for a page with no extractable text), so callers can slice a packet by page range. """ pages: list[str] = [] with pdfplumber.open(io.BytesIO(data)) as pdf: for page in pdf.pages: pages.append(page.extract_text() or "") return pages def _pdf_page_count(data: bytes) -> int: with pdfplumber.open(io.BytesIO(data)) as pdf: return len(pdf.pages) def extract_pdf_outline(data: bytes) -> list[dict]: """Extract a PDF's bookmark outline as a flat list of ``{page, level, title}``. Government agenda packets are typically compiled with a bookmark outline that mirrors the agenda: top-level bookmarks are sections, the next level are the agenda items (their title prefixed by the outline number), and the deepest level are the item's own attachments/reports. ``page`` is the 0-indexed page the bookmark targets (``None`` if it has no page destination); ``level`` is the nesting depth (0 = top). This is the deterministic, packet-authored map of where each item's content lives -- far more reliable than re-deriving it from page text. Returns ``[]`` when the PDF carries no outline (some packets don't), so callers can fall back. Uses pypdfium2 (already a dependency for page rendering); imported lazily so the rest of this module needs only pdfplumber. """ import pypdfium2 as pdfium out: list[dict] = [] pdf = pdfium.PdfDocument(data) try: for bm in pdf.get_toc(): dest = bm.get_dest() page = dest.get_index() if dest is not None else None out.append({"page": page, "level": bm.level, "title": bm.get_title() or ""}) finally: pdf.close() return out