docs/ARCHITECTURE.md

Architecture

Design principle

Financial conversion must fail visibly when it cannot prove a result. Language models and OCR may propose structure, but arithmetic, data typing, totals and release decisions are deterministic.

Processing graph

Upload
  -> document profiler
  -> native text or OCR extraction
  -> report-family router
  -> positioned table parser
  -> normalized domain model
  -> financial validator
  -> workbook planner/composer
  -> XLSX self-check
  -> download

Components

Extraction

finreport_agent.extraction opens PDF and image files with PyMuPDF. Pages with enough native text use the PDF text layer. Image-only pages use PyMuPDF's Tesseract integration with Turkish and English language packs.

Coordinates are normalized to a 950 × 1200 reference canvas. Parsers therefore work with PDFs and rendered versions of the same layout without binding themselves to a single DPI. The dominant text direction is detected per page and 90°, 180° and 270° content is transformed into reading coordinates before table discovery.

Parsing

finreport_agent.parsing currently implements the Turkish investment-fund portfolio report family. It uses column geometry, semantic markers and data-type patterns together:

• ISIN and date patterns identify real data rows.
• X coordinates assign tokens to financial columns.
• Section markers distinguish equity, repo, derivative and collateral tables.
• Multi-line issuer names are assembled only inside the current record boundary.

For other financial documents, finreport_agent.generic clusters numeric right-edges into dynamic columns, assembles multi-line row labels, infers hierarchy and joins compatible continuation pages. The normalized Pydantic models store typed decimals, table structure and source references.

Standard monthly fund reports are handled by finreport_agent.fund_generic. It segments pages at the source document's I–VI section headings, extracts introduction and performance data as key-value tables, maps portfolio rows into canonical code/currency/issuer/ISIN/ date/value columns and keeps totals, expenses and period transactions separate. Source headings override model-generated worksheet names for this family.

Bounded planning agent

finreport_agent.agent_planner sends only structural metadata—front matter, detected headings, column labels and sample row labels—to a Hugging Face Inference Provider model. Strict JSON Schema constrains its response to document classification, descriptions and worksheet names. It cannot change extracted numbers or create arbitrary cells. A deterministic planner provides the same contract whenever inference is unavailable.

Validation

finreport_agent.validation runs deterministic controls:

• nominal × current price versus position value;
• detail totals versus PDF-declared totals;
• equity + repo + cash collateral versus fund portfolio value;
• FPD percentage reconciliation;
• ISIN formatting;
• explicit treatment of derivative exposure.

Errors, warnings and information findings are separate. A rounding warning is never hidden or silently converted into a successful reconciliation.

Excel composition

finreport_agent.excel provides the specialist fund workbook and finreport_agent.generic_excel composes dynamic workbooks with:

• typed dates and numbers;
• cached values for formulas;
• filters, frozen panes and Excel tables;
• Turkish financial formats;
• print-ready A4 landscape layouts;
• negative-position highlighting;
• validation and source-provenance worksheets.

Adding another report family

1. Add domain models when the financial concepts differ.
2. Implement a parser that consumes ExtractedDocument.
3. Add a report-family signature to the router.
4. Add deterministic validation rules.
5. Implement the workbook plan for the detected sections.
6. Add manually verified golden PDF/JSON/XLSX fixtures.
7. Require zero silent numeric errors before enabling the family in the UI.

Next extensions

• table-level accounting equation checks for additional report families;
• ambiguity-triggered second extraction for low-confidence cells;
• human review gates for unresolved numeric errors;
• more manually verified golden PDF/XLSX fixtures.