| --- |
| title: Multimodal Financial RAG |
| emoji: 🏦 |
| colorFrom: indigo |
| colorTo: blue |
| sdk: gradio |
| sdk_version: 5.33.0 |
| app_file: app.py |
| pinned: true |
| python_version: "3.11" |
| license: mit |
| short_description: Production RAG for financial documents |
| tags: |
| - finance |
| - rag |
| - nlp |
| - document-question-answering |
| - gradio |
| - multimodal |
| - openai |
| - gemini |
| --- |
| |
| # 🏦 Multimodal Financial RAG |
|
|
| **Production-grade document intelligence** — chart understanding · hybrid RRF retrieval · numeric guardrails · source citations |
|
|
| This Space is a faithful demo of the enterprise system at |
| [github.com/Mattral/RAG-Multimodal-Financial-Doc-Analysis-and-Recall](https://github.com/Mattral/RAG-Multimodal-Financial-Doc-Analysis-and-Recall). |
|
|
| ## Features |
|
|
| - **👁️ Vision chart extraction** — GPT-4o / Gemini 2.5 Flash describe charts and graphs into searchable text |
| - **⚡ Hybrid RRF retrieval** — dense (BAAI/bge-small-en-v1.5) + BM25 fused with Reciprocal Rank Fusion (k=60) |
| - **🔢 Numeric grounding** — every number in the answer cross-checked against source context |
| - **🔒 PII + injection protection** — SSNs, IBANs, CUSIPs redacted; prompt injection patterns blocked |
| - **📍 Page-level citations** — every claim attributed to a specific document and page |
| - **🔬 Full pipeline transparency** — see every retrieval score, RRF weight, generation cost |
|
|
| ## Models (v2.0 — current) |
|
|
| | Provider | Text generation | Vision | |
| |---|---|---| |
| | Google Gemini | gemini-2.5-flash (default), gemini-2.5-pro | gemini-2.5-flash | |
| | OpenAI | gpt-4o-mini, gpt-4o | gpt-4o | |
|
|
| ## How to Use |
|
|
| 1. Enter your API key (Gemini free tier at [aistudio.google.com](https://aistudio.google.com)) |
| 2. Upload a 10-K, 10-Q, or earnings release PDF |
| 3. Click **Process Document** |
| 4. Ask a question |
|
|
| ## Architecture |
|
|
| ``` |
| PDF → pdfplumber (text + tables) + Vision LLM (charts) |
| → Semantic chunking (≤800 chars, 100-char overlap, paragraph-boundary split) |
| → BAAI/bge-small-en-v1.5 embeddings → FAISS IndexFlatIP |
| → BM25Okapi keyword index |
| → RRF fusion: 0.7/(60+dense_rank+1) + 0.3/(60+bm25_rank+1) |
| → Top-k chunks → GPT-4o-mini / Gemini 2.5 Flash generation |
| → Guardrails: injection check → PII redaction → numeric grounding |
| → Grounded answer + page-level citations |
| ``` |
|
|
| Mirrors the production pipeline in |
| [`src/rag_system/`](https://github.com/Mattral/RAG-Multimodal-Financial-Doc-Analysis-and-Recall/tree/main/src/rag_system). |
|
|
| ## Privacy |
|
|
| API keys are never stored. Uploaded PDFs are processed in-memory and not persisted. |
|
|
| ## License |
|
|
| MIT |
|
|
