README.md

metadata

title: Multimodal Financial RAG
emoji: 🏦
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: 5.33.0
app_file: app.py
pinned: true
python_version: '3.11'
license: mit
short_description: Production RAG for financial documents
tags:
  - finance
  - rag
  - nlp
  - document-question-answering
  - gradio
  - multimodal
  - openai
  - gemini

🏦 Multimodal Financial RAG

Production-grade document intelligence — chart understanding · hybrid RRF retrieval · numeric guardrails · source citations

This Space is a faithful demo of the enterprise system at github.com/Mattral/RAG-Multimodal-Financial-Doc-Analysis-and-Recall.

Features

• 👁️ Vision chart extraction — GPT-4o / Gemini 2.5 Flash describe charts and graphs into searchable text
• ⚡ Hybrid RRF retrieval — dense (BAAI/bge-small-en-v1.5) + BM25 fused with Reciprocal Rank Fusion (k=60)
• 🔢 Numeric grounding — every number in the answer cross-checked against source context
• 🔒 PII + injection protection — SSNs, IBANs, CUSIPs redacted; prompt injection patterns blocked
• 📍 Page-level citations — every claim attributed to a specific document and page
• 🔬 Full pipeline transparency — see every retrieval score, RRF weight, generation cost

Models (v2.0 — current)

Provider	Text generation	Vision
Google Gemini	gemini-2.5-flash (default), gemini-2.5-pro	gemini-2.5-flash
OpenAI	gpt-4o-mini, gpt-4o	gpt-4o

How to Use

1. Enter your API key (Gemini free tier at aistudio.google.com)
2. Upload a 10-K, 10-Q, or earnings release PDF
3. Click Process Document
4. Ask a question

Architecture

PDF → pdfplumber (text + tables) + Vision LLM (charts)
    → Semantic chunking (≤800 chars, 100-char overlap, paragraph-boundary split)
    → BAAI/bge-small-en-v1.5 embeddings → FAISS IndexFlatIP
    → BM25Okapi keyword index
    → RRF fusion: 0.7/(60+dense_rank+1) + 0.3/(60+bm25_rank+1)
    → Top-k chunks → GPT-4o-mini / Gemini 2.5 Flash generation
    → Guardrails: injection check → PII redaction → numeric grounding
    → Grounded answer + page-level citations

Mirrors the production pipeline in src/rag_system/.

Privacy

API keys are never stored. Uploaded PDFs are processed in-memory and not persisted.

License

MIT