README.md

metadata

title: IRIS IR Platform
emoji: 🏦
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
short_description: Multimodal RAG platform for financial document intelligence
thumbnail: >-
  https://huggingface.co/spaces/rajvivan/iris-ir-platform/resolve/main/screenshot.png

IRIS — Investor Relations Intelligence System

IRIS is a state-of-the-art, fully local, multimodal Retrieval-Augmented Generation (RAG) platform for Investor Relations teams. It enables financial analysts to query dense IR presentations, quarterly reports, and earnings statements — returning grounded, IR-quality answers with exact page visual evidence in under 50ms for all common financial queries.

---

🚀 What Makes IRIS Different

Problem	IRIS Solution
Standard RAG misses charts & tables	Hybrid retrieval: text + table + ColPali visual legs fused via RRF
LLMs hallucinate financial numbers	KPI Ground Truth layer: all numbers pinned to verified PDF values
Slow responses (60s+ LLM calls)	3-Layer Smart Engine: intent → KPI context → template fill → <50ms
Cache only works for one document	Universal optimizer: auto-generates cache for ANY trained PDF
New PDF = start from scratch	Auto-training pipeline: upload PDF, all responses generated automatically

---

🏛️ System Architecture

flowchart TD
  S1["1. User opens HF Space or local app"]
  S2["2. Next.js UI renders chat, document viewer, KPI cards, and evidence panels"]
  S3["3. Browser calls relative API routes under /api/proxy/*"]
  S4["4. Next.js proxy checks internal FastAPI at BACKEND_INTERNAL_URL, default 127.0.0.1:8000"]
  S5{"5. Is FastAPI live?"}
  S6["6A. Live path: FastAPI receives /api/health, /api/documents, /api/chat/query, /api/visuals/*"]
  S7["6B. Fallback path: Next.js serves the same API from backend/data cache and page screenshots"]
  S8["7. Guardrail and intent detection classify unsupported, insufficient, or IR financial query"]
  S9["8. Retrieval/answer layer loads verified KPI cache, response_cache JSON, tables, and exact page images"]
  S10["9. UI returns grounded answer with executive summary, KPI rows, key drivers, source footnotes, and visual evidence"]

  S1 --> S2 --> S3 --> S4 --> S5
  S5 -- yes --> S6 --> S8
  S5 -- "no / warming / restarting" --> S7 --> S8
  S8 --> S9 --> S10

9-Step Runtime Flow

1. User opens the Hugging Face Space or local app.
2. Next.js renders the chat workspace, document panels, source viewer, and evidence cards.
3. The browser calls only relative /api/proxy/* routes, so office networks never need direct access to localhost:8000.
4. The Next.js proxy tries the private in-container FastAPI backend at BACKEND_INTERNAL_URL, defaulting to http://127.0.0.1:8000.
5. If FastAPI is live, requests are streamed through to the backend.
6. If FastAPI is warming, blocked, or restarting, Next.js serves the same public API contract from checked-in cache files under backend/data.
7. Domain guardrails and intent detection decide whether the query is unsupported, insufficient, or a valid Investor Relations financial question.
8. The answer layer loads verified KPI data, precomputed response_cache JSON, source metadata, and exact PDF page screenshots.
9. The UI returns an evidence-grounded response with executive summary, KPI rows, key drivers, source footnotes, and visual evidence.

---

End-to-End Architecture Data Flow

This diagram shows the complete IRIS process from PDF ingestion through runtime answer generation. The key design principle is that financial answers must be grounded in three evidence types at the same time: narrative text, exact financial tables, and visual slide/page evidence.

flowchart TD
  A0["Source PDF<br/>Investor presentation, annual report, disclosure, earnings deck"]

  subgraph TRAIN["Document Training and Indexing"]
    A1["1. Parse PDF<br/>Extract text, tables, page order, metadata"]
    A2["2. Build page map<br/>Map pages to sections, KPIs, topics, synonyms, visual layout"]
    A3["3. Extract KPI ground truth<br/>Verify values, periods, units, and source pages"]
    A4["4. Chunk narrative text<br/>Section-aware chunks with doc/page/KPI metadata"]
    A5["5. Embed text chunks<br/>Local BGE embeddings"]
    A6["6. Store text vectors<br/>ChromaDB persistent local vector store"]
    A7["7. Index structured tables<br/>Rows, headers, metrics, values, periods, units"]
    A8["8. Render PDF pages<br/>High-quality page screenshots for visual evidence"]
    A9["9. Build ColPali visual index<br/>Patch embeddings per page for chart/page retrieval"]
    A10["10. Generate smart response cache<br/>Pre-filled responses for common IR intents"]
  end

  subgraph DATA["Persistent Data Layer"]
    D1["kpi_ground_truth.json<br/>Verified financial numbers"]
    D2["backend/data/chroma<br/>Dense text vectors"]
    D3["tables.json<br/>Structured table evidence"]
    D4["backend/data/pages<br/>Rendered page screenshots"]
    D5["backend/data/colpali_index<br/>Visual page patch embeddings"]
    D6["response_cache/{doc_id}<br/>Millisecond answers for known intents"]
    D7["documents.json + slide index<br/>Document metadata and page directory"]
  end

  subgraph RUNTIME["Question Runtime"]
    R0["User asks question in Next.js UI"]
    R1["Next.js API proxy<br/>/api/proxy/*"]
    R2{"FastAPI live?"}
    R3["Fallback cache path<br/>Serve checked-in backend/data if FastAPI is warming"]
    R4["Domain guardrails<br/>Reject non-IR / unsafe / unsupported questions"]
    R5["Intent classifier<br/>Map common financial questions to known intent"]
    R6{"Known intent<br/>with cache/template?"}
    R7["Smart response engine<br/>Load response_cache or fill template from KPI ground truth"]
    R8["Hybrid retrieval for novel queries"]
    R9["Text leg<br/>ChromaDB semantic search"]
    R10["Table leg<br/>Metric/table keyword + metadata search"]
    R11["Visual leg<br/>ColPali MaxSim page retrieval"]
    R12["RRF fusion + optional rerank<br/>Merge text/table/visual evidence"]
    R13["Financial analyst generator<br/>Ollama LLM or template fallback"]
    R14["Grounded IRIS response<br/>Executive summary, KPIs, drivers, sources, visuals"]
    R15["UI renders answer<br/>Footnotes + PDF viewer + page screenshots"]
  end

  A0 --> A1 --> A2 --> A3 --> A4 --> A5 --> A6
  A3 --> D1
  A6 --> D2
  A2 --> D7
  A7 --> D3
  A8 --> D4
  A9 --> D5
  A10 --> D6
  A4 --> A7
  A7 --> A8 --> A9 --> A10

  R0 --> R1 --> R2
  R2 -- "no / warming" --> R3 --> R14
  R2 -- yes --> R4 --> R5 --> R6
  R6 -- yes --> R7
  D1 --> R7
  D6 --> R7
  R7 --> R14
  R6 -- no --> R8
  D2 --> R9
  D3 --> R10
  D5 --> R11
  R8 --> R9 --> R12
  R8 --> R10 --> R12
  R8 --> R11 --> R12
  D4 --> R14
  R12 --> R13 --> R14 --> R15

Step-by-Step Data Flow

Step	Layer	What Happens	Main Files / Services	Output
1	Ingestion	A PDF is uploaded or placed in documents/. The watcher or manual command starts training.	backend/train_document.py, backend/ingest.py	Document enters the indexing pipeline.
2	Parsing	Text, tables, page order, and document metadata are extracted.	services/ingestion/pdf_parser.py	Raw extraction JSON under backend/data/processed/.
3	Page intelligence	Each page is mapped to section, slide number, KPI family, synonyms, topic, and visual layout.	slide_directory_index.json, page map builders	Retrieval can connect questions to likely pages.
4	KPI verification	Important financial values are extracted and pinned to source pages.	kpi_ground_truth.json	Numeric answers use verified values instead of generated guesses.
5	Text indexing	Narrative text is chunked, embedded with BGE, and stored in ChromaDB.	text_chunker.py, text_retriever.py, backend/data/chroma	Semantic search over commentary and explanatory text.
6	Table indexing	Extracted tables are converted into searchable metric/value records.	table_retriever.py, tables.json	Precise retrieval for period comparisons, ratios, and KPI rows.
7	Visual indexing	PDF pages are rendered to PNG and embedded with ColPali patch vectors.	page_renderer.py, colpali_indexer.py, colpali_index/	Visual retrieval for charts, graphs, slide layouts, and page screenshots.
8	Smart cache	Common IR intent responses are pre-generated from verified KPI context.	response_cache/<doc_id>/*.json	Known financial questions return in milliseconds.
9	Runtime proxy	Browser calls Next.js relative API routes, which proxy to private FastAPI.	src/app/api/proxy/[...path]/route.ts	Stable public API for local and Hugging Face deployments.
10	Guardrails	Non-IR, unsafe, unsupported, or insufficient questions are stopped early.	guardrails.py, client-side guard patterns	Keeps IRIS focused on Investor Relations evidence.
11	Fast path	Known intents load cache/template responses from verified KPI data.	intent_classifier.py, kpi_context_builder.py, smart_response_engine.py	Fast grounded response without LLM inference.
12	Novel path	Uncached questions use hybrid retrieval across text, tables, and visuals.	hybrid_retriever.py	Fused evidence package ranked by relevance.
13	Generation	The financial analyst agent formats the answer using retrieved evidence only.	financial_analyst_agent.py, Ollama or template fallback	IR-style answer with sources and visual pages.
14	UI rendering	Next.js renders executive summary, KPI tables, drivers, sources, and PDF screenshots.	ResponseWorkspace.tsx, RightEvidencePanel.tsx	Analyst-ready answer with traceable page evidence.

Why These Components Are Used

Component	Why IRIS Uses It	What It Solves
ChromaDB	Provides a simple persistent local vector database for dense text embeddings. It runs on-device, has no external service dependency, and can filter by document metadata.	Finds semantically related management commentary even when the user does not use the exact wording from the PDF.
BGE text embeddings	BGE is lightweight enough for local use and strong for retrieval-style semantic matching.	Converts narrative financial text into vectors that can match phrases like “profitability improved” with “net profit YoY performance.”
Structured table index	Financial questions often depend on exact table rows, periods, ratios, and units rather than prose.	Prevents the system from relying only on fuzzy semantic search for numbers. It retrieves exact KPI evidence such as CET1, NIM, LCR, loans, deposits, and cost of risk.
KPI ground truth layer	Verified KPI values are stored separately from generated answers.	Reduces hallucination risk by forcing common financial answers to use pinned numbers and page references.
ColPali	ColPali retrieves pages visually using late-interaction MaxSim over page patch embeddings. It can match a query to a chart, waterfall, or slide even when OCR text is incomplete or the chart labels are tiny.	Standard text RAG misses visual-only evidence. ColPali finds charts, graphs, page layouts, and slide screenshots that explain financial movement.
Rendered page screenshots	IR users need to see the exact source page, not just a text citation.	Powers the PDF viewer, visual evidence cards, and “open page” source workflow.
Reciprocal Rank Fusion (RRF)	Text, table, and visual retrievers produce different score scales. RRF merges ranked lists without assuming scores are directly comparable.	A page can rank highly because it has relevant commentary, exact table values, or a chart; RRF lets all three evidence types contribute fairly.
Smart response cache	Most IR questions are repeated: profitability, NIM, capital, liquidity, deposits, loans, ESG, segment performance.	Delivers common answers in milliseconds and avoids unnecessary LLM calls.
Ollama / local LLM fallback	Some questions are novel and need synthesis across retrieved evidence.	Keeps complex answer generation local while preserving a strict financial analyst prompt and evidence-only behavior.
Next.js proxy fallback	Hugging Face exposes only the web app; FastAPI may still be warming up or restarting.	Keeps the UI usable by serving the same API contract from checked-in cache files when the backend is unavailable.

The result is not a single-vector RAG system. IRIS is a financial evidence system: ChromaDB handles semantic prose, the table index handles exact numeric facts, ColPali handles visual/chart evidence, and the smart cache handles repeated IR questions with verified KPI values.

---

Hugging Face Runtime Architecture

The Hugging Face Space runs as a Docker app on port 7860. Only the Next.js server is exposed publicly; FastAPI is private inside the same container on port 8000.

Step-by-step request flow:

1. Browser opens https://rajvivan-iris-ir-platform.hf.space.
2. The Next.js app calls relative URLs such as /api/proxy/api/health, /api/proxy/api/chat/query, /api/proxy/api/documents, and /api/proxy/api/visuals/....
3. The Next.js proxy first tries BACKEND_INTERNAL_URL, defaulting to http://127.0.0.1:8000.
4. If FastAPI is live, the proxy streams the FastAPI response back to the browser.
5. If FastAPI is still warming up, blocked, or restarting, the proxy serves the same public API from checked-in cache files under backend/data.
6. /api/health still returns 200 from the proxy fallback, so the UI does not incorrectly show “backend not live”.
7. Page screenshots are served from backend/data/pages/... through both /api/visuals/... and legacy /pages/... paths.
8. docker-entrypoint.sh starts FastAPI in a restart loop, then starts Next.js. If FastAPI exits, it is restarted without taking down the public web app.

This means office networks only need to reach the single Hugging Face HTTPS URL. The browser never needs direct access to localhost:8000.

---

Change Safety Contract

Before changing runtime, sync, proxy, cache, or evidence-rendering behavior, read PRESERVE_EXISTING_FUNCTIONALITY.md. It lists protected files, required checks, and the expected HF-safe deployment behavior.

---

⚡ Smart Response Optimizer (New)

The core performance innovation. All common financial questions are handled before any LLM inference:

How it works

User asks: "How did Net Profit perform year-on-year?"
     │
     ▼  <1ms
IntentClassifier
  → PROFITABILITY (confidence: 1.00)
  → matched: ["net profit", "year-on-year"]
     │
     ▼  <3ms
KPIContextBuilder
  → loads kpi_ground_truth.json for active document
  → verified: net_profit = 6.4 bn (Page 16)
  → relevant pages: [16, 30]
  → slide refs + visual URLs built
     │
     ▼  <5ms
SmartResponseEngine
  → tries response_cache/<doc_id>/profitability.json first
  → if not found, fills PROFITABILITY template with KPI values
  → auto-saves to cache for future reuse
     │
     ▼
Complete IR-quality response returned in <10ms ✅

Intent Coverage (16 financial & corporate topics)

Intent	Example queries
PROFITABILITY	Net profit, PAT, PBT, ROE, earnings, bottom line
NET_INTEREST_MARGIN	NIM, interest margin, spread, EIBOR, yield
INCOME	Total income, NII, revenue, top-line
NON_FUNDED_INCOME	NFI, fee income, net fee & commission, wealth fees
CREDIT_QUALITY	NPL ratio, coverage ratio, ECL, provisioning, IFRS 9
CAPITAL	CET-1, CAR, RWA, Tier 1, regulatory capital
LIQUIDITY	LCR, ADR, NSFR, liquid assets
LOANS_SECTOR	Gross loans by sector, sector breakdown/allocation
LOANS	Gross loans, loan book, lending growth, advances
DEPOSITS	Customer deposits, CASA, funding mix
COST_EFFICIENCY	Cost-to-income, operating expenses, OPEX, jaws
SEGMENT	RBWM, CIB, GM&T, DenizBank, divisional performance
HYPERINFLATION	IAS 29, DenizBank inflation, TRY monetary correction
ECL_SCENARIO	Model-driven ECL, MEV weights, downside scenarios
MACRO	UAE GDP, operating environment, interest rates
ESG	2030 ESG objectives, decarbonization targets, sustainable finance framework, TNFD adopter

ESG 2030 Objectives Coverage

For ESG-related queries (such as "what is my 2030 ESG objectives?"), the platform includes a dedicated esg.json cache response that returns verified ESG key objectives and metrics in <10ms:

• Sustainable Finance: Mobilizing USD 30 billion by 2030 (with USD 26.1 billion already mobilized — 87%+ of target).
• Decarbonization: Achieving a 30% reduction in Scope 1 & Scope 2 GHG emissions by 2030 (from a 2020 baseline), on path to Net Zero 2050.
• Disclosures & Frameworks: First MENA bank to be a TNFD early adopter (disclosing by 2025), first bank globally to publish an ISSB report combining TCFD and IFRS S1/S2 frameworks.
• Leadership Diversity: Target of 25% female leadership by 2027 (reaching 20% in 2025).

New PDF = Instant Cache

When train_document.py runs on a new PDF, step 8b auto-generates pre-filled response JSONs for every supported intent into data/response_cache/<doc_id>/. First request serves from template; subsequent requests hit cache at <5ms.

---

🤖 Multimodal Financial Analyst Agent

For complex or novel questions not covered by the 16 pre-defined intents, IRIS falls back to a local Multimodal RAG pipeline:

1. Hybrid Retrieval: Blends ChromaDB dense text embeddings, structured keyword table indexes, and ColPali MaxSim page patch embeddings via Reciprocal Rank Fusion (RRF).
2. Local LLM Execution: Calls a local Ollama model (defaulting to mistral or llama3.2) using a strict financial analyst persona system prompt that forces AED bn formatting, prevents hallucinations, and excludes RAG jargon.
3. Template Fallback: If Ollama is not running locally, the agent uses a template fallback to extract KPIs directly from table rows and summarize retrieved text chunks in <15ms.

---

🔄 Local HuggingFace Sync

To deploy the code to a Hugging Face Space, use the Hugging Face Hub upload path for binary assets and avoid plain Git pushes for page screenshots:

• Code Isolation: Automatically isolates Next.js frontend code and FastAPI Python backend code into /tmp/hf-push/.
• Runtime Data: Keeps the compact cache files and page screenshots needed by the HF fallback API.
• Large Local Artifacts: Excludes node_modules/, .next/, .venv/, documents/, vector databases, and embedding arrays.
• Recommended Upload: Use hf upload ... --repo-type space for commits that include binary screenshots, because HF rejects normal Git pushes containing binary files unless Xet/LFS is configured.

You can run it in your project root:

./sync-to-hf.sh

For targeted fixes, upload only the changed files:

hf upload rajvivan/iris-ir-platform src/app/api/proxy/[...path]/route.ts src/app/api/proxy/[...path]/route.ts --repo-type space
hf upload rajvivan/iris-ir-platform docker-entrypoint.sh docker-entrypoint.sh --repo-type space

---

📁 Repository Structure

finbot-ir-platform/
├── README.md
├── AGENTS.md
├── start.sh                          # Start backend + frontend together
├── ingest.sh                         # PDF ingestion runner
├── documents/                        # Drop PDFs here for auto-indexing
│   └── emiratesnbd_investor_presentation_2026_q1.pdf
├── src/                              # Next.js Frontend
│   ├── app/
│   │   ├── layout.tsx
│   │   ├── page.tsx
│   │   └── globals.css
│   ├── components/
│   │   ├── chat/                     # Chat cards, KPI tables, source panels
│   │   ├── layout/                   # Shell, sidebars, PDF viewer
│   │   └── ui/                       # Accordion, badges
│   └── lib/
│       ├── api.ts                    # API client
│       └── mockData.ts
└── backend/                          # FastAPI Backend
    ├── requirements.txt
    ├── train_document.py             # Full training pipeline (9 steps + auto-cache)
    ├── app/
    │   ├── main.py                   # FastAPI app, CORS, background watcher
    │   └── api/
    │       ├── chat.py               # 3-layer response pipeline
    │       ├── documents.py          # Document management
    │       └── pages.py              # Page image serving
    ├── services/
    │   ├── classification/           # NEW: Smart Engine Layer 1 & 2
    │   │   ├── intent_classifier.py  # Zero-latency keyword intent detector
    │   │   └── kpi_context_builder.py# Per-document KPI ground truth loader
    │   ├── generation/
    │   │   ├── smart_response_engine.py  # NEW: Template filler + cache manager
    │   │   └── financial_analyst_agent.py# Ollama LLM fallback
    │   ├── retrieval/
    │   │   ├── hybrid_retriever.py   # RRF fusion across all 3 legs
    │   │   ├── text_retriever.py     # ChromaDB dense search
    │   │   ├── table_retriever.py    # Structured financial table search
    │   │   └── visual_retriever_colpali.py  # ColPali MaxSim visual search
    │   ├── ingestion/
    │   │   ├── pdf_parser.py         # pdfplumber extraction
    │   │   ├── text_chunker.py       # Section-aware chunking
    │   │   ├── page_renderer.py      # PDF → PNG screenshots
    │   │   └── colpali_indexer.py    # ColPali page embedding
    │   └── validation/
    │       └── guardrails.py         # Domain guardrail filter
    └── data/
        ├── kpi_ground_truth.json     # Verified KPI values per document
        ├── response_rules.json       # IR formatting & language rules
        ├── retrieval_config.json     # Calibrated retrieval weights
        ├── slide_directory_index.json# Per-slide topic/KPI index
        ├── response_cache/           # NEW: Auto-generated smart response cache
        │   └── <doc_id>/
        │       ├── profitability.json
        │       ├── net-interest-margin.json
        │       ├── capital.json
        │       └── ...
        ├── chroma/                   # ChromaDB vector store
        ├── colpali_index/            # ColPali patch embeddings
        ├── pages/                    # Rendered PDF page images
        └── tables.json               # Extracted financial tables

---

🛠️ Installation & Setup

Prerequisites

• Node.js v18+
• Python v3.10+
• Ollama installed — pull the default model:

  ollama pull mistral
  

Quick Start

chmod +x start.sh
./start.sh

This launches:

1. Python .venv setup + FastAPI Uvicorn server → http://localhost:8000
2. Next.js development server → http://localhost:3000

In production/Hugging Face, browser requests should go through /api/proxy/*; the browser should not call localhost:8000 directly.

---

📂 Adding a New Document

Option A — Auto-scan (background)

Drop a PDF into the documents/ folder. The background watcher detects it every 30 seconds and runs the full ingestion pipeline automatically.

Option B — Manual training (recommended for first run)

cd backend
source .venv/bin/activate
python train_document.py --pdf documents/your_report.pdf

The training pipeline has 9 numbered stages. Stage 8 is split into 8A and 8B because table indexing and smart-response cache generation happen together before final visual rendering.

flowchart TD
  T0["Input PDF<br/>Investor presentation, annual report, or financial disclosure"]
  T1["1. Parse PDF<br/>Extract text, tables, page structure, and raw document metadata"]
  T2["2. Build page to section to KPI map<br/>Connect each slide/page to topics, sections, KPI families, and business context"]
  T3["3. Extract KPI ground truth<br/>Pin verified financial numbers, periods, units, and source pages"]
  T4["4. Chunk text with section metadata<br/>Create retrieval-ready chunks with page, section, KPI, and document labels"]
  T5["5. Generate training pairs and calibrate retrieval weights<br/>Create query-document examples and tune text/table/visual retrieval balance"]
  T6["6. Optional embedding fine-tune<br/>Improve domain matching for bank-specific IR terminology when training data is available"]
  T7["7. Embed and index text chunks<br/>Store dense vectors in ChromaDB for semantic retrieval"]
  T8A["8A. Index tables<br/>Store enriched table rows, metrics, periods, and interpretation metadata"]
  T8B["8B. Auto-generate smart response cache<br/>Pre-fill intent templates for profitability, income, NIM, liquidity, capital, ESG, segments, and more"]
  T9["9. Render pages and build ColPali visual index<br/>Create exact page screenshots and visual embeddings for chart/page evidence"]
  T10["Ready for IRIS runtime<br/>Questions return grounded answers with KPI values, footnotes, and page screenshots"]

  T0 --> T1 --> T2 --> T3 --> T4 --> T5 --> T6 --> T7
  T7 --> T8A --> T8B --> T9 --> T10

Stage	What Happens	Output Created	Why It Matters
1. Parse PDF	Reads PDF text, tables, page order, and raw structure.	Extracted text/table JSON.	Gives the system machine-readable source material.
2. Page → section → KPI mapping	Maps each page to sections, topics, KPI families, synonyms, and visual layout.	Page map and slide directory metadata.	Lets retrieval know which page answers which type of question.
3. KPI ground truth	Extracts and verifies important numbers such as net profit, total income, NIM, LCR, CET1, cost of risk, loans, deposits, and segment values.	kpi_ground_truth.json.	Prevents hallucinated financial figures.
4. Chunk text	Splits content into retrieval chunks while preserving page, section, KPI, and document metadata.	Section-aware text chunks.	Keeps answers tied to source pages and context.
5. Training pairs and retrieval calibration	Builds sample query-to-source examples and tunes retrieval weights across text, tables, and visual evidence.	Training examples and retrieval config.	Improves ranking quality for financial questions.
6. Optional embedding fine-tune	Fine-tunes embeddings when enough domain examples exist.	Optional improved embedding model.	Helps with bank-specific terminology and phrasing.
7. ChromaDB text index	Embeds chunks and stores them in a dense vector index.	ChromaDB vector store.	Enables semantic search over narrative text.
8A. Table index	Indexes rows, metrics, periods, values, units, and table interpretations.	tables.json and table metadata.	Ensures numeric/table-heavy questions retrieve precise evidence.
8B. Smart response cache	Generates pre-filled response JSON for supported intents.	backend/data/response_cache/<doc_id>/*.json.	Makes common IR questions return in milliseconds without LLM inference.
9. Page rendering and ColPali indexing	Renders exact PDF page screenshots and builds visual embeddings.	backend/data/pages/** and ColPali index metadata.	Powers screenshot evidence and chart/page retrieval.

After training, all supported financial intents are instantly cached for the new document. First question → template generation (<10ms). Subsequent questions → cache hit (<5ms).

---

📊 Performance

Query type	Method	Latency
Known financial intent (any trained PDF)	Smart Engine → Template	<10ms
Previously asked intent (any trained PDF)	Smart Engine → Cache	<5ms
Emirates NBD Q1 2026 (static fallback)	Static cache	~10ms
Unknown/novel question	Ollama RAG pipeline	30–120s

Tested: 20/20 financial queries return in <10ms without any LLM inference.

---

🛡️ Key Features

• Multimodal Retrieval: Text + Tables + ColPali visual page embeddings, fused via Reciprocal Rank Fusion
• KPI Ground Truth: All financial numbers pinned to verified PDF values — no hallucination
• Universal Smart Cache: Document-agnostic intent templates work for any bank, any quarter
• Auto-Training Pipeline: Upload PDF → verified KPI extraction, retrieval indexes, smart response cache, and exact page screenshots
• 100% Local: No cloud API calls — all inference on-device (Mac MPS / CUDA)
• IR-Quality Formatting: Professional Investor Relations language, AED bn formatting, YoY/QoQ labels
• Visual Evidence: Every response includes rendered PDF page screenshots as evidence
• Domain Guardrail: Filters non-financial queries before any processing

---

📝 Financial Intents Reference

The IntentClassifier recognises 200+ phrase variants across 15 intents. Some examples:

"How did Net Profit perform?"         → PROFITABILITY
"What is the NIM compression trend?"  → NET_INTEREST_MARGIN
"Show me loans by sector"             → LOANS_SECTOR
"Tell me about ECL scenario weights"  → ECL_SCENARIO
"What is the CET-1 ratio?"            → CAPITAL
"How did fee income grow?"            → NON_FUNDED_INCOME
"What is the cost-to-income ratio?"   → COST_EFFICIENCY
"What is the hyperinflation impact?"  → HYPERINFLATION
"How did DenizBank perform?"          → SEGMENT

New synonyms can be added to services/classification/intent_classifier.py without any retraining.