File size: 7,638 Bytes
055323f d4f8959 055323f d4f8959 055323f d4f8959 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | ---
title: FinSight
emoji: 📊
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false
---
# FinSight — Grounded Financial Filing Analyzer
Upload an annual report or 10-K. Ask questions. Get answers where **every number is traceable to the document** — with a confidence label, a sector-aware red-flag scan, and an auditable BUY / HOLD / AVOID / SKIP call. Built for the case where the filing is confidential and cloud AI is not an option.
**Design thesis: numbers never come from the LLM.** Deterministic extraction owns every figure; the LLM only narrates. That is what makes answers grounded — and it is why a small local model on a CPU is enough.
## Architecture
```mermaid
flowchart LR
A[PDF / HTML filing] --> B[Parser<br/>PyMuPDF + chunking]
B --> C[Sector detector<br/>BANK / IT / PHARMA / ENERGY / MFG]
C --> D[Metric extractor<br/>multi-candidate, confidence-scored,<br/>currency-aware, subsidiary-boundary checks]
B --> E[Entity resolver<br/>alias table + fuzzy matching]
E --> F[Knowledge graph — NetworkX<br/>companies, filings, chunks, typed relations]
D --> F
B --> G[ChromaDB<br/>local MiniLM embeddings]
F --> H[Red-flag engine<br/>sector thresholds, zero LLM]
H --> I[Recommendation engine<br/>BUY / HOLD / AVOID / SKIP, zero LLM]
F --> J[Query router]
G --> J
J -->|metric question| K[Direct graph lookup<br/>ZERO LLM tokens]
J -->|open question| L[LLM narration<br/>local Ollama default / Groq opt-in]
```
## What makes it different
- **Confidence is a first-class value.** Every extracted metric is `{value, currency, confidence, alternatives, needs_clarification}` — scanned as *all* plausible candidates, not first-match-wins. Low-confidence numbers are surfaced for clarification, never silently trusted.
- **Wrong-entity protection.** Indian annual reports often bundle a subsidiary's full financial statements inside the parent's PDF. FinSight detects section ownership headings and demotes numbers that belong to a different entity — a wrong-*entity* bug is worse than a wrong-*number* bug.
- **Indian-filing realities handled.** ₹ crore/lakh units, comma-grouped Indian digit formats, and PDF font encodings that render ₹ as `` ` ``, `J`, `C`, or `H` (each confirmed in real filings) — recovered via a unit-confirmation heuristic instead of glyph whack-a-mole.
- **Token efficiency by architecture, not compression.** Metric questions are answered by direct graph lookup — zero LLM tokens. Only open-ended questions reach the LLM, and then with top-3 chunks plus a compact metrics header, not the whole document.
- **Post-generation numeric verification.** A prompt saying "use only the context" is a request, not a guarantee — so every number in a generated answer is checked against the source filing (unit-normalized across ₹ crore/lakh/$B/M formats). Unverified numbers are flagged in the response, never silently shipped. Real run: a full performance summary generated by a local 1.5B model had all 23 of its numbers traced back to the filing.
- **Page-level citations.** Chunks carry the PDF page they came from; answers cite them (`HDFC Bank 2025 filing, p.210, p.392, p.410`), and every extracted metric records the page it was read from.
- **Analysis without any LLM ("Mode C").** The red-flag engine (sector-specific thresholds: NPA%, CASA, attrition, D/E, YoY declines, severity-weighted risk score) and the recommendation engine (explicit, auditable decision tree — `SKIP` is an honest answer when data is thin) are pure rules over extracted metrics. They work with no model running at all.
- **Typed relationship graph.** SUPPLIER_TO / COMPETITOR_OF / SUBSIDIARY_OF / ACQUIRED edges extracted by sentence-level patterns first, with a local-LLM fallback only for ambiguous co-occurring pairs.
## Run it
### Backend
```bash
python -m venv venv
venv\Scripts\activate # Windows (source venv/bin/activate on Unix)
pip install -r requirements.txt
python -m spacy download en_core_web_sm
copy .env.example .env # then edit as needed
uvicorn backend.main:app --reload --port 8000
```
### Frontend (React + TypeScript + Vite)
```bash
cd frontend
npm install
npm run dev # http://localhost:5173, proxies /api -> :8000
```
### Local LLM (optional but recommended)
Install [Ollama](https://ollama.com), then:
```bash
ollama pull mistral:7b-instruct-q4_K_M # or any instruct model; set OLLAMA_MODEL
```
No Ollama and no Groq key? Metric Q&A, red flags, and recommendations still work — only free-text narration is disabled.
## API
| Endpoint | What it does |
|---|---|
| `POST /upload` | Parse a filing (PDF/HTML), extract confidence-scored metrics, index into graph + vector store |
| `POST /query` | Ask a question — direct zero-token lookup when possible, RAG otherwise |
| `GET /metrics/{company}` | All extracted metrics by year, with confidence and alternatives |
| `GET /red_flags/{company}/{year}` | Rule-based red-flag scan with severity-weighted risk score |
| `GET /recommendation/{company}/{year}` | BUY / HOLD / AVOID / SKIP with explicit reasons |
| `GET /report/{company}/{year}` | Structured narrative report (LLM, grounded on extracted metrics) |
| `POST /compare` | Compare a metric across companies |
| `GET /companies` | List indexed companies |
## Deploy (Hugging Face Spaces, free)
The repo doubles as a Docker Space — the YAML header above is its config. One container builds the React frontend, installs the backend, and **parses the committed demo filing at image-build time**, so the Space boots with HDFC Bank's annual report already indexed.
```bash
# once: create a Docker-SDK Space at huggingface.co/new-space, then
git remote add space https://huggingface.co/spaces/<user>/FinSight
git push space main
```
In the Space settings add secrets: `GROQ_API_KEY` (narration) — `LLM_BACKEND=groq` is already set in the image. The hosted demo uses cloud narration by design; the private, fully-local mode is what you get by cloning the repo and running Ollama. Red flags, recommendations, and direct metric lookup work on the Space even with no key set.
## Measured results
From `python -m eval.run_eval` against hand-labeled ground truth (every value read from the actual filing, page-referenced — see `eval/ground_truth.json`):
| Metric | Result |
|---|---|
| Extraction recall (true value found) | 6/6 (100%) |
| Top-candidate accuracy | 4/6 (67%) — misses carry `needs_clarification: true` and the correct value in `alternatives` |
| Prompt-token reduction vs naive top-20 RAG | **80.4%** (~2,077 vs ~10,616 tokens per question, len/4 approximation) |
| Metric questions answered with zero LLM tokens | 5/5 |
Current eval set is one real 590-page Indian bank annual report; the harness is built for more — add a PDF + labeled entry and re-run.
## Honest limitations
- Extraction is heuristic (regex + NER + context windows), not a table parser — a statement-table tier is planned. Treat outputs as assistance, not audited figures; the numbers above quantify exactly how far to trust it.
- Ratio metrics (NPA%, attrition, …) don't carry confidence scores yet — only money metrics do. Ratios use majority voting across the filing's repeated statements instead.
- The verifier confirms numbers exist in the source; it does not (yet) confirm they're attached to the right claim in the sentence.
- English filings only; scanned/image PDFs need OCR, which isn't wired in.
- Local narration on CPU is slow (a 1.5B model takes ~2–3 minutes for a long answer). That's the privacy trade; Groq mode is instant.
|