--- title: IRIS IR Platform emoji: 🏦 colorFrom: blue colorTo: indigo sdk: docker pinned: false short_description: Multimodal RAG platform for financial document intelligence thumbnail: https://huggingface.co/spaces/rajvivan/iris-ir-platform/resolve/main/screenshot.png --- # IRIS β€” Investor Relations Intelligence System IRIS is a state-of-the-art, fully local, multimodal Retrieval-Augmented Generation (RAG) platform for Investor Relations teams. It enables financial analysts to query dense IR presentations, quarterly reports, and earnings statements β€” returning grounded, IR-quality answers with exact page visual evidence in **under 50ms** for all common financial queries. --- ## πŸš€ What Makes IRIS Different | Problem | IRIS Solution | |---|---| | Standard RAG misses charts & tables | Hybrid retrieval: text + table + **ColPali visual** legs fused via RRF | | LLMs hallucinate financial numbers | **KPI Ground Truth** layer: all numbers pinned to verified PDF values | | Slow responses (60s+ LLM calls) | **3-Layer Smart Engine**: intent β†’ KPI context β†’ template fill β†’ <50ms | | Cache only works for one document | **Universal optimizer**: auto-generates cache for ANY trained PDF | | New PDF = start from scratch | **Auto-training pipeline**: upload PDF, all responses generated automatically | --- ## πŸ›οΈ System Architecture ```mermaid flowchart TD S1["1. User opens HF Space or local app"] S2["2. Next.js UI renders chat, document viewer, KPI cards, and evidence panels"] S3["3. Browser calls relative API routes under /api/proxy/*"] S4["4. Next.js proxy checks internal FastAPI at BACKEND_INTERNAL_URL, default 127.0.0.1:8000"] S5{"5. Is FastAPI live?"} S6["6A. Live path: FastAPI receives /api/health, /api/documents, /api/chat/query, /api/visuals/*"] S7["6B. Fallback path: Next.js serves the same API from backend/data cache and page screenshots"] S8["7. Guardrail and intent detection classify unsupported, insufficient, or IR financial query"] S9["8. Retrieval/answer layer loads verified KPI cache, response_cache JSON, tables, and exact page images"] S10["9. UI returns grounded answer with executive summary, KPI rows, key drivers, source footnotes, and visual evidence"] S1 --> S2 --> S3 --> S4 --> S5 S5 -- yes --> S6 --> S8 S5 -- "no / warming / restarting" --> S7 --> S8 S8 --> S9 --> S10 ``` ### 9-Step Runtime Flow 1. User opens the Hugging Face Space or local app. 2. Next.js renders the chat workspace, document panels, source viewer, and evidence cards. 3. The browser calls only relative `/api/proxy/*` routes, so office networks never need direct access to `localhost:8000`. 4. The Next.js proxy tries the private in-container FastAPI backend at `BACKEND_INTERNAL_URL`, defaulting to `http://127.0.0.1:8000`. 5. If FastAPI is live, requests are streamed through to the backend. 6. If FastAPI is warming, blocked, or restarting, Next.js serves the same public API contract from checked-in cache files under `backend/data`. 7. Domain guardrails and intent detection decide whether the query is unsupported, insufficient, or a valid Investor Relations financial question. 8. The answer layer loads verified KPI data, precomputed `response_cache` JSON, source metadata, and exact PDF page screenshots. 9. The UI returns an evidence-grounded response with executive summary, KPI rows, key drivers, source footnotes, and visual evidence. --- ## End-to-End Architecture Data Flow This diagram shows the complete IRIS process from PDF ingestion through runtime answer generation. The key design principle is that financial answers must be grounded in three evidence types at the same time: narrative text, exact financial tables, and visual slide/page evidence. ```mermaid flowchart TD A0["Source PDF
Investor presentation, annual report, disclosure, earnings deck"] subgraph TRAIN["Document Training and Indexing"] A1["1. Parse PDF
Extract text, tables, page order, metadata"] A2["2. Build page map
Map pages to sections, KPIs, topics, synonyms, visual layout"] A3["3. Extract KPI ground truth
Verify values, periods, units, and source pages"] A4["4. Chunk narrative text
Section-aware chunks with doc/page/KPI metadata"] A5["5. Embed text chunks
Local BGE embeddings"] A6["6. Store text vectors
ChromaDB persistent local vector store"] A7["7. Index structured tables
Rows, headers, metrics, values, periods, units"] A8["8. Render PDF pages
High-quality page screenshots for visual evidence"] A9["9. Build ColPali visual index
Patch embeddings per page for chart/page retrieval"] A10["10. Generate smart response cache
Pre-filled responses for common IR intents"] end subgraph DATA["Persistent Data Layer"] D1["kpi_ground_truth.json
Verified financial numbers"] D2["backend/data/chroma
Dense text vectors"] D3["tables.json
Structured table evidence"] D4["backend/data/pages
Rendered page screenshots"] D5["backend/data/colpali_index
Visual page patch embeddings"] D6["response_cache/{doc_id}
Millisecond answers for known intents"] D7["documents.json + slide index
Document metadata and page directory"] end subgraph RUNTIME["Question Runtime"] R0["User asks question in Next.js UI"] R1["Next.js API proxy
/api/proxy/*"] R2{"FastAPI live?"} R3["Fallback cache path
Serve checked-in backend/data if FastAPI is warming"] R4["Domain guardrails
Reject non-IR / unsafe / unsupported questions"] R5["Intent classifier
Map common financial questions to known intent"] R6{"Known intent
with cache/template?"} R7["Smart response engine
Load response_cache or fill template from KPI ground truth"] R8["Hybrid retrieval for novel queries"] R9["Text leg
ChromaDB semantic search"] R10["Table leg
Metric/table keyword + metadata search"] R11["Visual leg
ColPali MaxSim page retrieval"] R12["RRF fusion + optional rerank
Merge text/table/visual evidence"] R13["Financial analyst generator
Ollama LLM or template fallback"] R14["Grounded IRIS response
Executive summary, KPIs, drivers, sources, visuals"] R15["UI renders answer
Footnotes + PDF viewer + page screenshots"] end A0 --> A1 --> A2 --> A3 --> A4 --> A5 --> A6 A3 --> D1 A6 --> D2 A2 --> D7 A7 --> D3 A8 --> D4 A9 --> D5 A10 --> D6 A4 --> A7 A7 --> A8 --> A9 --> A10 R0 --> R1 --> R2 R2 -- "no / warming" --> R3 --> R14 R2 -- yes --> R4 --> R5 --> R6 R6 -- yes --> R7 D1 --> R7 D6 --> R7 R7 --> R14 R6 -- no --> R8 D2 --> R9 D3 --> R10 D5 --> R11 R8 --> R9 --> R12 R8 --> R10 --> R12 R8 --> R11 --> R12 D4 --> R14 R12 --> R13 --> R14 --> R15 ``` ### Step-by-Step Data Flow | Step | Layer | What Happens | Main Files / Services | Output | |---|---|---|---|---| | 1 | Ingestion | A PDF is uploaded or placed in `documents/`. The watcher or manual command starts training. | `backend/train_document.py`, `backend/ingest.py` | Document enters the indexing pipeline. | | 2 | Parsing | Text, tables, page order, and document metadata are extracted. | `services/ingestion/pdf_parser.py` | Raw extraction JSON under `backend/data/processed/`. | | 3 | Page intelligence | Each page is mapped to section, slide number, KPI family, synonyms, topic, and visual layout. | `slide_directory_index.json`, page map builders | Retrieval can connect questions to likely pages. | | 4 | KPI verification | Important financial values are extracted and pinned to source pages. | `kpi_ground_truth.json` | Numeric answers use verified values instead of generated guesses. | | 5 | Text indexing | Narrative text is chunked, embedded with BGE, and stored in ChromaDB. | `text_chunker.py`, `text_retriever.py`, `backend/data/chroma` | Semantic search over commentary and explanatory text. | | 6 | Table indexing | Extracted tables are converted into searchable metric/value records. | `table_retriever.py`, `tables.json` | Precise retrieval for period comparisons, ratios, and KPI rows. | | 7 | Visual indexing | PDF pages are rendered to PNG and embedded with ColPali patch vectors. | `page_renderer.py`, `colpali_indexer.py`, `colpali_index/` | Visual retrieval for charts, graphs, slide layouts, and page screenshots. | | 8 | Smart cache | Common IR intent responses are pre-generated from verified KPI context. | `response_cache//*.json` | Known financial questions return in milliseconds. | | 9 | Runtime proxy | Browser calls Next.js relative API routes, which proxy to private FastAPI. | `src/app/api/proxy/[...path]/route.ts` | Stable public API for local and Hugging Face deployments. | | 10 | Guardrails | Non-IR, unsafe, unsupported, or insufficient questions are stopped early. | `guardrails.py`, client-side guard patterns | Keeps IRIS focused on Investor Relations evidence. | | 11 | Fast path | Known intents load cache/template responses from verified KPI data. | `intent_classifier.py`, `kpi_context_builder.py`, `smart_response_engine.py` | Fast grounded response without LLM inference. | | 12 | Novel path | Uncached questions use hybrid retrieval across text, tables, and visuals. | `hybrid_retriever.py` | Fused evidence package ranked by relevance. | | 13 | Generation | The financial analyst agent formats the answer using retrieved evidence only. | `financial_analyst_agent.py`, Ollama or template fallback | IR-style answer with sources and visual pages. | | 14 | UI rendering | Next.js renders executive summary, KPI tables, drivers, sources, and PDF screenshots. | `ResponseWorkspace.tsx`, `RightEvidencePanel.tsx` | Analyst-ready answer with traceable page evidence. | ### Why These Components Are Used | Component | Why IRIS Uses It | What It Solves | |---|---|---| | **ChromaDB** | Provides a simple persistent local vector database for dense text embeddings. It runs on-device, has no external service dependency, and can filter by document metadata. | Finds semantically related management commentary even when the user does not use the exact wording from the PDF. | | **BGE text embeddings** | BGE is lightweight enough for local use and strong for retrieval-style semantic matching. | Converts narrative financial text into vectors that can match phrases like β€œprofitability improved” with β€œnet profit YoY performance.” | | **Structured table index** | Financial questions often depend on exact table rows, periods, ratios, and units rather than prose. | Prevents the system from relying only on fuzzy semantic search for numbers. It retrieves exact KPI evidence such as CET1, NIM, LCR, loans, deposits, and cost of risk. | | **KPI ground truth layer** | Verified KPI values are stored separately from generated answers. | Reduces hallucination risk by forcing common financial answers to use pinned numbers and page references. | | **ColPali** | ColPali retrieves pages visually using late-interaction MaxSim over page patch embeddings. It can match a query to a chart, waterfall, or slide even when OCR text is incomplete or the chart labels are tiny. | Standard text RAG misses visual-only evidence. ColPali finds charts, graphs, page layouts, and slide screenshots that explain financial movement. | | **Rendered page screenshots** | IR users need to see the exact source page, not just a text citation. | Powers the PDF viewer, visual evidence cards, and β€œopen page” source workflow. | | **Reciprocal Rank Fusion (RRF)** | Text, table, and visual retrievers produce different score scales. RRF merges ranked lists without assuming scores are directly comparable. | A page can rank highly because it has relevant commentary, exact table values, or a chart; RRF lets all three evidence types contribute fairly. | | **Smart response cache** | Most IR questions are repeated: profitability, NIM, capital, liquidity, deposits, loans, ESG, segment performance. | Delivers common answers in milliseconds and avoids unnecessary LLM calls. | | **Ollama / local LLM fallback** | Some questions are novel and need synthesis across retrieved evidence. | Keeps complex answer generation local while preserving a strict financial analyst prompt and evidence-only behavior. | | **Next.js proxy fallback** | Hugging Face exposes only the web app; FastAPI may still be warming up or restarting. | Keeps the UI usable by serving the same API contract from checked-in cache files when the backend is unavailable. | The result is not a single-vector RAG system. IRIS is a financial evidence system: ChromaDB handles semantic prose, the table index handles exact numeric facts, ColPali handles visual/chart evidence, and the smart cache handles repeated IR questions with verified KPI values. --- ## Hugging Face Runtime Architecture The Hugging Face Space runs as a Docker app on port `7860`. Only the Next.js server is exposed publicly; FastAPI is private inside the same container on port `8000`. Step-by-step request flow: 1. Browser opens `https://rajvivan-iris-ir-platform.hf.space`. 2. The Next.js app calls relative URLs such as `/api/proxy/api/health`, `/api/proxy/api/chat/query`, `/api/proxy/api/documents`, and `/api/proxy/api/visuals/...`. 3. The Next.js proxy first tries `BACKEND_INTERNAL_URL`, defaulting to `http://127.0.0.1:8000`. 4. If FastAPI is live, the proxy streams the FastAPI response back to the browser. 5. If FastAPI is still warming up, blocked, or restarting, the proxy serves the same public API from checked-in cache files under `backend/data`. 6. `/api/health` still returns `200` from the proxy fallback, so the UI does not incorrectly show β€œbackend not live”. 7. Page screenshots are served from `backend/data/pages/...` through both `/api/visuals/...` and legacy `/pages/...` paths. 8. `docker-entrypoint.sh` starts FastAPI in a restart loop, then starts Next.js. If FastAPI exits, it is restarted without taking down the public web app. This means office networks only need to reach the single Hugging Face HTTPS URL. The browser never needs direct access to `localhost:8000`. --- ## Change Safety Contract Before changing runtime, sync, proxy, cache, or evidence-rendering behavior, read [PRESERVE_EXISTING_FUNCTIONALITY.md](./PRESERVE_EXISTING_FUNCTIONALITY.md). It lists protected files, required checks, and the expected HF-safe deployment behavior. --- ## ⚑ Smart Response Optimizer (New) The core performance innovation. All common financial questions are handled before any LLM inference: ### How it works ``` User asks: "How did Net Profit perform year-on-year?" β”‚ β–Ό <1ms IntentClassifier β†’ PROFITABILITY (confidence: 1.00) β†’ matched: ["net profit", "year-on-year"] β”‚ β–Ό <3ms KPIContextBuilder β†’ loads kpi_ground_truth.json for active document β†’ verified: net_profit = 6.4 bn (Page 16) β†’ relevant pages: [16, 30] β†’ slide refs + visual URLs built β”‚ β–Ό <5ms SmartResponseEngine β†’ tries response_cache//profitability.json first β†’ if not found, fills PROFITABILITY template with KPI values β†’ auto-saves to cache for future reuse β”‚ β–Ό Complete IR-quality response returned in <10ms βœ… ``` ### Intent Coverage (16 financial & corporate topics) | Intent | Example queries | |---|---| | `PROFITABILITY` | Net profit, PAT, PBT, ROE, earnings, bottom line | | `NET_INTEREST_MARGIN` | NIM, interest margin, spread, EIBOR, yield | | `INCOME` | Total income, NII, revenue, top-line | | `NON_FUNDED_INCOME` | NFI, fee income, net fee & commission, wealth fees | | `CREDIT_QUALITY` | NPL ratio, coverage ratio, ECL, provisioning, IFRS 9 | | `CAPITAL` | CET-1, CAR, RWA, Tier 1, regulatory capital | | `LIQUIDITY` | LCR, ADR, NSFR, liquid assets | | `LOANS_SECTOR` | Gross loans by sector, sector breakdown/allocation | | `LOANS` | Gross loans, loan book, lending growth, advances | | `DEPOSITS` | Customer deposits, CASA, funding mix | | `COST_EFFICIENCY` | Cost-to-income, operating expenses, OPEX, jaws | | `SEGMENT` | RBWM, CIB, GM&T, DenizBank, divisional performance | | `HYPERINFLATION` | IAS 29, DenizBank inflation, TRY monetary correction | | `ECL_SCENARIO` | Model-driven ECL, MEV weights, downside scenarios | | `MACRO` | UAE GDP, operating environment, interest rates | | `ESG` | 2030 ESG objectives, decarbonization targets, sustainable finance framework, TNFD adopter | ### ESG 2030 Objectives Coverage For ESG-related queries (such as *"what is my 2030 ESG objectives?"*), the platform includes a dedicated `esg.json` cache response that returns verified ESG key objectives and metrics in <10ms: - **Sustainable Finance:** Mobilizing USD 30 billion by 2030 (with USD 26.1 billion already mobilized β€” 87%+ of target). - **Decarbonization:** Achieving a 30% reduction in Scope 1 & Scope 2 GHG emissions by 2030 (from a 2020 baseline), on path to Net Zero 2050. - **Disclosures & Frameworks:** First MENA bank to be a TNFD early adopter (disclosing by 2025), first bank globally to publish an ISSB report combining TCFD and IFRS S1/S2 frameworks. - **Leadership Diversity:** Target of 25% female leadership by 2027 (reaching 20% in 2025). ### New PDF = Instant Cache When `train_document.py` runs on a new PDF, step **8b** auto-generates pre-filled response JSONs for every supported intent into `data/response_cache//`. First request serves from template; subsequent requests hit cache at <5ms. --- ## πŸ€– Multimodal Financial Analyst Agent For complex or novel questions not covered by the 16 pre-defined intents, IRIS falls back to a local Multimodal RAG pipeline: 1. **Hybrid Retrieval:** Blends ChromaDB dense text embeddings, structured keyword table indexes, and **ColPali MaxSim page patch embeddings** via Reciprocal Rank Fusion (RRF). 2. **Local LLM Execution:** Calls a local Ollama model (defaulting to `mistral` or `llama3.2`) using a strict financial analyst persona system prompt that forces AED bn formatting, prevents hallucinations, and excludes RAG jargon. 3. **Template Fallback:** If Ollama is not running locally, the agent uses a template fallback to extract KPIs directly from table rows and summarize retrieved text chunks in <15ms. --- ## πŸ”„ Local HuggingFace Sync To deploy the code to a Hugging Face Space, use the Hugging Face Hub upload path for binary assets and avoid plain Git pushes for page screenshots: - **Code Isolation:** Automatically isolates Next.js frontend code and FastAPI Python backend code into `/tmp/hf-push/`. - **Runtime Data:** Keeps the compact cache files and page screenshots needed by the HF fallback API. - **Large Local Artifacts:** Excludes `node_modules/`, `.next/`, `.venv/`, `documents/`, vector databases, and embedding arrays. - **Recommended Upload:** Use `hf upload ... --repo-type space` for commits that include binary screenshots, because HF rejects normal Git pushes containing binary files unless Xet/LFS is configured. You can run it in your project root: ```bash ./sync-to-hf.sh ``` For targeted fixes, upload only the changed files: ```bash hf upload rajvivan/iris-ir-platform src/app/api/proxy/[...path]/route.ts src/app/api/proxy/[...path]/route.ts --repo-type space hf upload rajvivan/iris-ir-platform docker-entrypoint.sh docker-entrypoint.sh --repo-type space ``` --- ## πŸ“ Repository Structure ``` finbot-ir-platform/ β”œβ”€β”€ README.md β”œβ”€β”€ AGENTS.md β”œβ”€β”€ start.sh # Start backend + frontend together β”œβ”€β”€ ingest.sh # PDF ingestion runner β”œβ”€β”€ documents/ # Drop PDFs here for auto-indexing β”‚ └── emiratesnbd_investor_presentation_2026_q1.pdf β”œβ”€β”€ src/ # Next.js Frontend β”‚ β”œβ”€β”€ app/ β”‚ β”‚ β”œβ”€β”€ layout.tsx β”‚ β”‚ β”œβ”€β”€ page.tsx β”‚ β”‚ └── globals.css β”‚ β”œβ”€β”€ components/ β”‚ β”‚ β”œβ”€β”€ chat/ # Chat cards, KPI tables, source panels β”‚ β”‚ β”œβ”€β”€ layout/ # Shell, sidebars, PDF viewer β”‚ β”‚ └── ui/ # Accordion, badges β”‚ └── lib/ β”‚ β”œβ”€β”€ api.ts # API client β”‚ └── mockData.ts └── backend/ # FastAPI Backend β”œβ”€β”€ requirements.txt β”œβ”€β”€ train_document.py # Full training pipeline (9 steps + auto-cache) β”œβ”€β”€ app/ β”‚ β”œβ”€β”€ main.py # FastAPI app, CORS, background watcher β”‚ └── api/ β”‚ β”œβ”€β”€ chat.py # 3-layer response pipeline β”‚ β”œβ”€β”€ documents.py # Document management β”‚ └── pages.py # Page image serving β”œβ”€β”€ services/ β”‚ β”œβ”€β”€ classification/ # NEW: Smart Engine Layer 1 & 2 β”‚ β”‚ β”œβ”€β”€ intent_classifier.py # Zero-latency keyword intent detector β”‚ β”‚ └── kpi_context_builder.py# Per-document KPI ground truth loader β”‚ β”œβ”€β”€ generation/ β”‚ β”‚ β”œβ”€β”€ smart_response_engine.py # NEW: Template filler + cache manager β”‚ β”‚ └── financial_analyst_agent.py# Ollama LLM fallback β”‚ β”œβ”€β”€ retrieval/ β”‚ β”‚ β”œβ”€β”€ hybrid_retriever.py # RRF fusion across all 3 legs β”‚ β”‚ β”œβ”€β”€ text_retriever.py # ChromaDB dense search β”‚ β”‚ β”œβ”€β”€ table_retriever.py # Structured financial table search β”‚ β”‚ └── visual_retriever_colpali.py # ColPali MaxSim visual search β”‚ β”œβ”€β”€ ingestion/ β”‚ β”‚ β”œβ”€β”€ pdf_parser.py # pdfplumber extraction β”‚ β”‚ β”œβ”€β”€ text_chunker.py # Section-aware chunking β”‚ β”‚ β”œβ”€β”€ page_renderer.py # PDF β†’ PNG screenshots β”‚ β”‚ └── colpali_indexer.py # ColPali page embedding β”‚ └── validation/ β”‚ └── guardrails.py # Domain guardrail filter └── data/ β”œβ”€β”€ kpi_ground_truth.json # Verified KPI values per document β”œβ”€β”€ response_rules.json # IR formatting & language rules β”œβ”€β”€ retrieval_config.json # Calibrated retrieval weights β”œβ”€β”€ slide_directory_index.json# Per-slide topic/KPI index β”œβ”€β”€ response_cache/ # NEW: Auto-generated smart response cache β”‚ └── / β”‚ β”œβ”€β”€ profitability.json β”‚ β”œβ”€β”€ net-interest-margin.json β”‚ β”œβ”€β”€ capital.json β”‚ └── ... β”œβ”€β”€ chroma/ # ChromaDB vector store β”œβ”€β”€ colpali_index/ # ColPali patch embeddings β”œβ”€β”€ pages/ # Rendered PDF page images └── tables.json # Extracted financial tables ``` --- ## πŸ› οΈ Installation & Setup ### Prerequisites - **Node.js** v18+ - **Python** v3.10+ - **Ollama** installed β€” pull the default model: ```bash ollama pull mistral ``` ### Quick Start ```bash chmod +x start.sh ./start.sh ``` This launches: 1. Python `.venv` setup + FastAPI Uvicorn server β†’ `http://localhost:8000` 2. Next.js development server β†’ `http://localhost:3000` In production/Hugging Face, browser requests should go through `/api/proxy/*`; the browser should not call `localhost:8000` directly. --- ## πŸ“‚ Adding a New Document ### Option A β€” Auto-scan (background) Drop a PDF into the `documents/` folder. The background watcher detects it every 30 seconds and runs the full ingestion pipeline automatically. ### Option B β€” Manual training (recommended for first run) ```bash cd backend source .venv/bin/activate python train_document.py --pdf documents/your_report.pdf ``` The training pipeline has 9 numbered stages. Stage 8 is split into `8A` and `8B` because table indexing and smart-response cache generation happen together before final visual rendering. ```mermaid flowchart TD T0["Input PDF
Investor presentation, annual report, or financial disclosure"] T1["1. Parse PDF
Extract text, tables, page structure, and raw document metadata"] T2["2. Build page to section to KPI map
Connect each slide/page to topics, sections, KPI families, and business context"] T3["3. Extract KPI ground truth
Pin verified financial numbers, periods, units, and source pages"] T4["4. Chunk text with section metadata
Create retrieval-ready chunks with page, section, KPI, and document labels"] T5["5. Generate training pairs and calibrate retrieval weights
Create query-document examples and tune text/table/visual retrieval balance"] T6["6. Optional embedding fine-tune
Improve domain matching for bank-specific IR terminology when training data is available"] T7["7. Embed and index text chunks
Store dense vectors in ChromaDB for semantic retrieval"] T8A["8A. Index tables
Store enriched table rows, metrics, periods, and interpretation metadata"] T8B["8B. Auto-generate smart response cache
Pre-fill intent templates for profitability, income, NIM, liquidity, capital, ESG, segments, and more"] T9["9. Render pages and build ColPali visual index
Create exact page screenshots and visual embeddings for chart/page evidence"] T10["Ready for IRIS runtime
Questions return grounded answers with KPI values, footnotes, and page screenshots"] T0 --> T1 --> T2 --> T3 --> T4 --> T5 --> T6 --> T7 T7 --> T8A --> T8B --> T9 --> T10 ``` | Stage | What Happens | Output Created | Why It Matters | |---|---|---|---| | 1. Parse PDF | Reads PDF text, tables, page order, and raw structure. | Extracted text/table JSON. | Gives the system machine-readable source material. | | 2. Page β†’ section β†’ KPI mapping | Maps each page to sections, topics, KPI families, synonyms, and visual layout. | Page map and slide directory metadata. | Lets retrieval know which page answers which type of question. | | 3. KPI ground truth | Extracts and verifies important numbers such as net profit, total income, NIM, LCR, CET1, cost of risk, loans, deposits, and segment values. | `kpi_ground_truth.json`. | Prevents hallucinated financial figures. | | 4. Chunk text | Splits content into retrieval chunks while preserving page, section, KPI, and document metadata. | Section-aware text chunks. | Keeps answers tied to source pages and context. | | 5. Training pairs and retrieval calibration | Builds sample query-to-source examples and tunes retrieval weights across text, tables, and visual evidence. | Training examples and retrieval config. | Improves ranking quality for financial questions. | | 6. Optional embedding fine-tune | Fine-tunes embeddings when enough domain examples exist. | Optional improved embedding model. | Helps with bank-specific terminology and phrasing. | | 7. ChromaDB text index | Embeds chunks and stores them in a dense vector index. | ChromaDB vector store. | Enables semantic search over narrative text. | | 8A. Table index | Indexes rows, metrics, periods, values, units, and table interpretations. | `tables.json` and table metadata. | Ensures numeric/table-heavy questions retrieve precise evidence. | | 8B. Smart response cache | Generates pre-filled response JSON for supported intents. | `backend/data/response_cache//*.json`. | Makes common IR questions return in milliseconds without LLM inference. | | 9. Page rendering and ColPali indexing | Renders exact PDF page screenshots and builds visual embeddings. | `backend/data/pages/**` and ColPali index metadata. | Powers screenshot evidence and chart/page retrieval. | After training, all supported financial intents are instantly cached for the new document. First question β†’ template generation (<10ms). Subsequent questions β†’ cache hit (<5ms). --- ## πŸ“Š Performance | Query type | Method | Latency | |---|---|---| | Known financial intent (any trained PDF) | Smart Engine β†’ Template | <10ms | | Previously asked intent (any trained PDF) | Smart Engine β†’ Cache | <5ms | | Emirates NBD Q1 2026 (static fallback) | Static cache | ~10ms | | Unknown/novel question | Ollama RAG pipeline | 30–120s | **Tested: 20/20 financial queries return in <10ms** without any LLM inference. --- ## πŸ›‘οΈ Key Features - **Multimodal Retrieval**: Text + Tables + ColPali visual page embeddings, fused via Reciprocal Rank Fusion - **KPI Ground Truth**: All financial numbers pinned to verified PDF values β€” no hallucination - **Universal Smart Cache**: Document-agnostic intent templates work for any bank, any quarter - **Auto-Training Pipeline**: Upload PDF β†’ verified KPI extraction, retrieval indexes, smart response cache, and exact page screenshots - **100% Local**: No cloud API calls β€” all inference on-device (Mac MPS / CUDA) - **IR-Quality Formatting**: Professional Investor Relations language, AED bn formatting, YoY/QoQ labels - **Visual Evidence**: Every response includes rendered PDF page screenshots as evidence - **Domain Guardrail**: Filters non-financial queries before any processing --- ## πŸ“ Financial Intents Reference The `IntentClassifier` recognises 200+ phrase variants across 15 intents. Some examples: ``` "How did Net Profit perform?" β†’ PROFITABILITY "What is the NIM compression trend?" β†’ NET_INTEREST_MARGIN "Show me loans by sector" β†’ LOANS_SECTOR "Tell me about ECL scenario weights" β†’ ECL_SCENARIO "What is the CET-1 ratio?" β†’ CAPITAL "How did fee income grow?" β†’ NON_FUNDED_INCOME "What is the cost-to-income ratio?" β†’ COST_EFFICIENCY "What is the hyperinflation impact?" β†’ HYPERINFLATION "How did DenizBank perform?" β†’ SEGMENT ``` New synonyms can be added to `services/classification/intent_classifier.py` without any retraining.