docs/ARCHITECTURE.md

# Architecture

## Design principle

Financial conversion must fail visibly when it cannot prove a result. Language
models and OCR may propose structure, but arithmetic, data typing, totals and
release decisions are deterministic.

## Processing graph

```text
Upload
  -> document profiler
  -> native text or OCR extraction
  -> report-family router
  -> positioned table parser
  -> normalized domain model
  -> financial validator
  -> workbook planner/composer
  -> XLSX self-check
  -> download
```

## Components

### Extraction

`finreport_agent.extraction` opens PDF and image files with PyMuPDF. Pages with
enough native text use the PDF text layer. Image-only pages use PyMuPDF's
Tesseract integration with Turkish and English language packs.

Coordinates are normalized to a 950 × 1200 reference canvas. Parsers therefore
work with PDFs and rendered versions of the same layout without binding
themselves to a single DPI. The dominant text direction is detected per page and
90°, 180° and 270° content is transformed into reading coordinates before table
discovery.

### Parsing

`finreport_agent.parsing` currently implements the Turkish investment-fund
portfolio report family. It uses column geometry, semantic markers and data-type
patterns together:

- ISIN and date patterns identify real data rows.
- X coordinates assign tokens to financial columns.
- Section markers distinguish equity, repo, derivative and collateral tables.
- Multi-line issuer names are assembled only inside the current record boundary.

For other financial documents, `finreport_agent.generic` clusters numeric
right-edges into dynamic columns, assembles multi-line row labels, infers
hierarchy and joins compatible continuation pages. The normalized Pydantic
models store typed decimals, table structure and source references.

Standard monthly fund reports are handled by
`finreport_agent.fund_generic`. It segments pages at the source document's
`I`–`VI` section headings, extracts introduction and performance data as
key-value tables, maps portfolio rows into canonical code/currency/issuer/ISIN/
date/value columns and keeps totals, expenses and period transactions separate.
Source headings override model-generated worksheet names for this family.

### Bounded planning agent

`finreport_agent.agent_planner` sends only structural metadata—front matter,
detected headings, column labels and sample row labels—to a Hugging Face
Inference Provider model. Strict JSON Schema constrains its response to document
classification, descriptions and worksheet names. It cannot change extracted
numbers or create arbitrary cells. A deterministic planner provides the same
contract whenever inference is unavailable.

### Validation

`finreport_agent.validation` runs deterministic controls:

- nominal × current price versus position value;
- detail totals versus PDF-declared totals;
- equity + repo + cash collateral versus fund portfolio value;
- FPD percentage reconciliation;
- ISIN formatting;
- explicit treatment of derivative exposure.

Errors, warnings and information findings are separate. A rounding warning is
never hidden or silently converted into a successful reconciliation.

### Excel composition

`finreport_agent.excel` provides the specialist fund workbook and
`finreport_agent.generic_excel` composes dynamic workbooks with:

- typed dates and numbers;
- cached values for formulas;
- filters, frozen panes and Excel tables;
- Turkish financial formats;
- print-ready A4 landscape layouts;
- negative-position highlighting;
- validation and source-provenance worksheets.

## Adding another report family

1. Add domain models when the financial concepts differ.
2. Implement a parser that consumes `ExtractedDocument`.
3. Add a report-family signature to the router.
4. Add deterministic validation rules.
5. Implement the workbook plan for the detected sections.
6. Add manually verified golden PDF/JSON/XLSX fixtures.
7. Require zero silent numeric errors before enabling the family in the UI.

## Next extensions

- table-level accounting equation checks for additional report families;
- ambiguity-triggered second extraction for low-confidence cells;
- human review gates for unresolved numeric errors;
- more manually verified golden PDF/XLSX fixtures.