| --- |
| title: FinSight |
| emoji: 📊 |
| colorFrom: blue |
| colorTo: green |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| --- |
| |
| # FinSight — Grounded Financial Filing Analyzer |
|
|
| Upload an annual report or 10-K. Ask questions. Get answers where **every number is traceable to the document** — with a confidence label, a sector-aware red-flag scan, and an auditable BUY / HOLD / AVOID / SKIP call. Built for the case where the filing is confidential and cloud AI is not an option. |
|
|
| **Design thesis: numbers never come from the LLM.** Deterministic extraction owns every figure; the LLM only narrates. That is what makes answers grounded — and it is why a small local model on a CPU is enough. |
|
|
| ## Architecture |
|
|
| ```mermaid |
| flowchart LR |
| A[PDF / HTML filing] --> B[Parser<br/>PyMuPDF + chunking] |
| B --> C[Sector detector<br/>BANK / IT / PHARMA / ENERGY / MFG] |
| C --> D[Metric extractor<br/>multi-candidate, confidence-scored,<br/>currency-aware, subsidiary-boundary checks] |
| B --> E[Entity resolver<br/>alias table + fuzzy matching] |
| E --> F[Knowledge graph — NetworkX<br/>companies, filings, chunks, typed relations] |
| D --> F |
| B --> G[ChromaDB<br/>local MiniLM embeddings] |
| F --> H[Red-flag engine<br/>sector thresholds, zero LLM] |
| H --> I[Recommendation engine<br/>BUY / HOLD / AVOID / SKIP, zero LLM] |
| F --> J[Query router] |
| G --> J |
| J -->|metric question| K[Direct graph lookup<br/>ZERO LLM tokens] |
| J -->|open question| L[LLM narration<br/>local Ollama default / Groq opt-in] |
| ``` |
|
|
| ## What makes it different |
|
|
| - **Confidence is a first-class value.** Every extracted metric is `{value, currency, confidence, alternatives, needs_clarification}` — scanned as *all* plausible candidates, not first-match-wins. Low-confidence numbers are surfaced for clarification, never silently trusted. |
| - **Wrong-entity protection.** Indian annual reports often bundle a subsidiary's full financial statements inside the parent's PDF. FinSight detects section ownership headings and demotes numbers that belong to a different entity — a wrong-*entity* bug is worse than a wrong-*number* bug. |
| - **Indian-filing realities handled.** ₹ crore/lakh units, comma-grouped Indian digit formats, and PDF font encodings that render ₹ as `` ` ``, `J`, `C`, or `H` (each confirmed in real filings) — recovered via a unit-confirmation heuristic instead of glyph whack-a-mole. |
| - **Token efficiency by architecture, not compression.** Metric questions are answered by direct graph lookup — zero LLM tokens. Only open-ended questions reach the LLM, and then with top-3 chunks plus a compact metrics header, not the whole document. |
| - **Post-generation numeric verification.** A prompt saying "use only the context" is a request, not a guarantee — so every number in a generated answer is checked against the source filing (unit-normalized across ₹ crore/lakh/$B/M formats). Unverified numbers are flagged in the response, never silently shipped. Real run: a full performance summary generated by a local 1.5B model had all 23 of its numbers traced back to the filing. |
| - **Page-level citations.** Chunks carry the PDF page they came from; answers cite them (`HDFC Bank 2025 filing, p.210, p.392, p.410`), and every extracted metric records the page it was read from. |
| - **Analysis without any LLM ("Mode C").** The red-flag engine (sector-specific thresholds: NPA%, CASA, attrition, D/E, YoY declines, severity-weighted risk score) and the recommendation engine (explicit, auditable decision tree — `SKIP` is an honest answer when data is thin) are pure rules over extracted metrics. They work with no model running at all. |
| - **Typed relationship graph.** SUPPLIER_TO / COMPETITOR_OF / SUBSIDIARY_OF / ACQUIRED edges extracted by sentence-level patterns first, with a local-LLM fallback only for ambiguous co-occurring pairs. |
| |
| ## Run it |
| |
| ### Backend |
| |
| ```bash |
| python -m venv venv |
| venv\Scripts\activate # Windows (source venv/bin/activate on Unix) |
| pip install -r requirements.txt |
| python -m spacy download en_core_web_sm |
| copy .env.example .env # then edit as needed |
| uvicorn backend.main:app --reload --port 8000 |
| ``` |
| |
| ### Frontend (React + TypeScript + Vite) |
| |
| ```bash |
| cd frontend |
| npm install |
| npm run dev # http://localhost:5173, proxies /api -> :8000 |
| ``` |
| |
| ### Local LLM (optional but recommended) |
| |
| Install [Ollama](https://ollama.com), then: |
| |
| ```bash |
| ollama pull mistral:7b-instruct-q4_K_M # or any instruct model; set OLLAMA_MODEL |
| ``` |
| |
| No Ollama and no Groq key? Metric Q&A, red flags, and recommendations still work — only free-text narration is disabled. |
| |
| ## API |
| |
| | Endpoint | What it does | |
| |---|---| |
| | `POST /upload` | Parse a filing (PDF/HTML), extract confidence-scored metrics, index into graph + vector store | |
| | `POST /query` | Ask a question — direct zero-token lookup when possible, RAG otherwise | |
| | `GET /metrics/{company}` | All extracted metrics by year, with confidence and alternatives | |
| | `GET /red_flags/{company}/{year}` | Rule-based red-flag scan with severity-weighted risk score | |
| | `GET /recommendation/{company}/{year}` | BUY / HOLD / AVOID / SKIP with explicit reasons | |
| | `GET /report/{company}/{year}` | Structured narrative report (LLM, grounded on extracted metrics) | |
| | `POST /compare` | Compare a metric across companies | |
| | `GET /companies` | List indexed companies | |
|
|
| ## Deploy (Hugging Face Spaces, free) |
|
|
| The repo doubles as a Docker Space — the YAML header above is its config. One container builds the React frontend, installs the backend, and **parses the committed demo filing at image-build time**, so the Space boots with HDFC Bank's annual report already indexed. |
|
|
| ```bash |
| # once: create a Docker-SDK Space at huggingface.co/new-space, then |
| git remote add space https://huggingface.co/spaces/<user>/FinSight |
| git push space main |
| ``` |
|
|
| In the Space settings add secrets: `GROQ_API_KEY` (narration) — `LLM_BACKEND=groq` is already set in the image. The hosted demo uses cloud narration by design; the private, fully-local mode is what you get by cloning the repo and running Ollama. Red flags, recommendations, and direct metric lookup work on the Space even with no key set. |
|
|
| ## Measured results |
|
|
| From `python -m eval.run_eval` against hand-labeled ground truth (every value read from the actual filing, page-referenced — see `eval/ground_truth.json`): |
|
|
| | Metric | Result | |
| |---|---| |
| | Extraction recall (true value found) | 6/6 (100%) | |
| | Top-candidate accuracy | 4/6 (67%) — misses carry `needs_clarification: true` and the correct value in `alternatives` | |
| | Prompt-token reduction vs naive top-20 RAG | **80.4%** (~2,077 vs ~10,616 tokens per question, len/4 approximation) | |
| | Metric questions answered with zero LLM tokens | 5/5 | |
|
|
| Current eval set is one real 590-page Indian bank annual report; the harness is built for more — add a PDF + labeled entry and re-run. |
|
|
| ## Honest limitations |
|
|
| - Extraction is heuristic (regex + NER + context windows), not a table parser — a statement-table tier is planned. Treat outputs as assistance, not audited figures; the numbers above quantify exactly how far to trust it. |
| - Ratio metrics (NPA%, attrition, …) don't carry confidence scores yet — only money metrics do. Ratios use majority voting across the filing's repeated statements instead. |
| - The verifier confirms numbers exist in the source; it does not (yet) confirm they're attached to the right claim in the sentence. |
| - English filings only; scanned/image PDFs need OCR, which isn't wired in. |
| - Local narration on CPU is slow (a 1.5B model takes ~2–3 minutes for a long answer). That's the privacy trade; Groq mode is instant. |
|
|
