File size: 4,284 Bytes
9703837 23b8566 9703837 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | # Architecture
## Design principle
Financial conversion must fail visibly when it cannot prove a result. Language
models and OCR may propose structure, but arithmetic, data typing, totals and
release decisions are deterministic.
## Processing graph
```text
Upload
-> document profiler
-> native text or OCR extraction
-> report-family router
-> positioned table parser
-> normalized domain model
-> financial validator
-> workbook planner/composer
-> XLSX self-check
-> download
```
## Components
### Extraction
`finreport_agent.extraction` opens PDF and image files with PyMuPDF. Pages with
enough native text use the PDF text layer. Image-only pages use PyMuPDF's
Tesseract integration with Turkish and English language packs.
Coordinates are normalized to a 950 × 1200 reference canvas. Parsers therefore
work with PDFs and rendered versions of the same layout without binding
themselves to a single DPI. The dominant text direction is detected per page and
90°, 180° and 270° content is transformed into reading coordinates before table
discovery.
### Parsing
`finreport_agent.parsing` currently implements the Turkish investment-fund
portfolio report family. It uses column geometry, semantic markers and data-type
patterns together:
- ISIN and date patterns identify real data rows.
- X coordinates assign tokens to financial columns.
- Section markers distinguish equity, repo, derivative and collateral tables.
- Multi-line issuer names are assembled only inside the current record boundary.
For other financial documents, `finreport_agent.generic` clusters numeric
right-edges into dynamic columns, assembles multi-line row labels, infers
hierarchy and joins compatible continuation pages. The normalized Pydantic
models store typed decimals, table structure and source references.
Standard monthly fund reports are handled by
`finreport_agent.fund_generic`. It segments pages at the source document's
`I`–`VI` section headings, extracts introduction and performance data as
key-value tables, maps portfolio rows into canonical code/currency/issuer/ISIN/
date/value columns and keeps totals, expenses and period transactions separate.
Source headings override model-generated worksheet names for this family.
### Bounded planning agent
`finreport_agent.agent_planner` sends only structural metadata—front matter,
detected headings, column labels and sample row labels—to a Hugging Face
Inference Provider model. Strict JSON Schema constrains its response to document
classification, descriptions and worksheet names. It cannot change extracted
numbers or create arbitrary cells. A deterministic planner provides the same
contract whenever inference is unavailable.
### Validation
`finreport_agent.validation` runs deterministic controls:
- nominal × current price versus position value;
- detail totals versus PDF-declared totals;
- equity + repo + cash collateral versus fund portfolio value;
- FPD percentage reconciliation;
- ISIN formatting;
- explicit treatment of derivative exposure.
Errors, warnings and information findings are separate. A rounding warning is
never hidden or silently converted into a successful reconciliation.
### Excel composition
`finreport_agent.excel` provides the specialist fund workbook and
`finreport_agent.generic_excel` composes dynamic workbooks with:
- typed dates and numbers;
- cached values for formulas;
- filters, frozen panes and Excel tables;
- Turkish financial formats;
- print-ready A4 landscape layouts;
- negative-position highlighting;
- validation and source-provenance worksheets.
## Adding another report family
1. Add domain models when the financial concepts differ.
2. Implement a parser that consumes `ExtractedDocument`.
3. Add a report-family signature to the router.
4. Add deterministic validation rules.
5. Implement the workbook plan for the detected sections.
6. Add manually verified golden PDF/JSON/XLSX fixtures.
7. Require zero silent numeric errors before enabling the family in the UI.
## Next extensions
- table-level accounting equation checks for additional report families;
- ambiguity-triggered second extraction for low-confidence cells;
- human review gates for unresolved numeric errors;
- more manually verified golden PDF/XLSX fixtures.
|