Some Changes to Documentation

LICENSE

@@ -0,0 +1,50 @@
+MIT License
+
+Copyright (c) 2026 Spandan Mukherjee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+----------------------------------------------------------------------------
+Third-party data and model notices
+----------------------------------------------------------------------------
+
+This repository may include or reference the following third-party assets.
+The MIT license above applies only to BankShield/DocSentry source code; the
+following assets are governed by their own terms:
+
+- AgamiAI Indian Bank Statements (Hugging Face) - Apache License 2.0
+  https://huggingface.co/datasets/AgamiAI/Indian-Bank-Statements
+
+- IDRBT Cheque Image Dataset - Research-use license; citation required.
+  https://www.idrbt.ac.in/idrbt-cheque-image-dataset/
+  When using IDRBT data, cite the IDRBT paper as required by their terms.
+
+- CASIA v2 image tampering dataset (referenced for optional CNN training) -
+  research use; see dataset page for terms.
+
+- MobileNetV2 architecture / ImageNet pretrained weights (TensorFlow/Keras) -
+  Apache License 2.0.
+
+- ReportLab, OpenCV, PyMuPDF, Pillow, scikit-learn, Streamlit and other
+  dependencies are governed by their respective open-source licenses (see
+  each project's repo).
+
+Users redistributing this software or any derivative must preserve these
+notices and comply with the upstream licenses of the data and models they
+choose to bundle.

README.md

@@ -1,103 +1,269 @@
-# DocSentry - Document Forensics for Bank Underwriting
+# DocSentry
 
-[![Built with Streamlit](https://img.shields.io/badge/Built%20with-Streamlit-FF4B4B.svg)](https://streamlit.io)
-[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+Document forensics and KYC compliance pipeline for bank underwriting workflows. Detects tampering and forgery in land records, legal documents, financial statements, and cheques. Validates KYC fields against RBI rules. Produces explainable risk scores and regulator-ready audit reports.
 
-**Real-time anomaly detection for land records, legal documents, and financial statements during loan underwriting.**
+100% open-source. No paid APIs. No LLM calls. CPU-only by default.
 
-> Detects tampering, edits, and forgery attempts in seconds. Generates explainable risk scores and underwriter-ready audit reports. 100% open-source, no paid APIs, no LLM calls. Runs on a laptop.
+---
 
-## :rocket: Live demo
+## Repository layout
 
-**Try it now:** [docsentry-yourname.streamlit.app](https://docsentry-yourname.streamlit.app) *(replace with your actual URL after deploying)*
+```
+Doc-Sentry/
+├── app.py                       Streamlit web UI (4 tabs)
+├── forensics.py                 Core detection engine
+├── audit_report.py              Bank-letterhead PDF report builder
+├── compliance.py                KYC validators, PII redaction, RBI report builder
+├── docsentry_master.ipynb       Single source-of-truth Jupyter notebook
+│
+├── requirements.txt             Python dependencies
+├── packages.txt                 System packages (Tesseract) for Streamlit Cloud
+├── .streamlit/config.toml       Streamlit theme + server config
+│
+├── sample_data/                 26 demo files for the live app
+│   ├── originals/               12 genuine documents
+│   ├── tampered/                12 tampered documents
+│   └── pdfs/                    2 PDFs (1 genuine, 1 tampered)
+│
+├── models/                      Trained model artefacts
+│   └── forgery_rf.joblib        Random Forest classifier
+│
+├── README.md  DEPLOY.md  RUN_APP.md  DATASETS.md  PUSH.md  LICENSE
+└── data/                        (gitignored) full training data + downloaded datasets
+```
+
+---
+
+## Module reference
 
-Pick a tampered land record from the dropdown and watch the forensic verdict appear in real time.
+### `forensics.py` — detection engine
 
-## :movie_camera: 3-minute demo flow
+The core analytical module. Stateless functions; all logic is independently testable.
 
-1. **Single-document analysis** — drop a PNG/PDF, see risk band (LOW/MEDIUM/HIGH/CRITICAL), heatmap overlays, evidence bullets
-2. **Cross-document consistency** — upload 2-4 documents for the same applicant; system flags mismatches in name/DOB/address
-3. **Batch audit** — point at a folder of documents; get a sortable risk table and CSV export
-4. **Audit PDF** — one-click download of a bank-letterhead PDF report ready for the underwriting file
+| Function | Returns | Description |
+|---|---|---|
+| `analyse_document(path)` | dict | End-to-end pipeline. Auto-detects type (image vs PDF), runs all relevant detectors, blends Random Forest + CNN predictions when available. Primary entry point. |
+| `score_image(path)` | (float, dict, list) | Composite forensic score for an image. Returns total, sub-scores by detector, and EXIF flags. |
+| `error_level_analysis(path, quality=90)` | (PIL.Image, float) | ELA visualisation + scalar suspicion score. Re-saves at given JPEG quality; tampered regions diverge from the rest of the image. |
+| `copy_move_detect(path)` | (np.ndarray, int, list) | Detects regions duplicated within the same image using ORB keypoint matching. Returns annotated visualisation, match count, and raw matches. |
+| `noise_inconsistency(path, block=32)` | (np.ndarray, float) | Per-block Laplacian variance. Returns a heatmap of outlier blocks and a normalised ratio. Useful for splicing detection. |
+| `exif_sanity(path)` | list of str | EXIF metadata audit. Flags missing EXIF, photo-editor signatures (Photoshop/GIMP/Snapseed), and timestamp inconsistencies. |
+| `pdf_structural_audit(path)` | dict | Counts `%%EOF` markers (incremental edits), compares producer vs creator, flags consumer-tool fingerprints (iLovePDF, Smallpdf, etc.). |
+| `pdf_font_audit(path)` | dict | Lists embedded fonts and flags unusually high font counts (a signal of inserted text). |
+| `ocr_text(path)` | str | Tesseract OCR with auto-fallback. Returns empty string if Tesseract isn't installed. |
+| `text_rule_checks(text)` | dict | Validates date monotonicity, amount sanity, IFSC format, account number patterns. |
+| `extract_features(path)` | dict | Feature vector for the Random Forest: 11 features (ELA, copy-move count, noise ratio, EXIF flag, 4 GLCM texture features, 3 colour histogram entropies). |
+| `predict_with_model(path)` | dict or None | Loads `models/forgery_rf.joblib` and returns tamper probability + verdict. None if model isn't present. |
+| `predict_with_cnn(path)` | dict or None | Lazy-loads `models/forgery_cnn.keras` (TensorFlow). None if model isn't present, so the app starts fast without TF. |
+| `extract_identity_fields(path)` | (dict, str) | Pulls name, DOB, address, account number, IFSC, and amounts from any document. |
+| `cross_doc_consistency(paths)` | dict | Compares identity fields across 2+ documents using `difflib.SequenceMatcher`. Returns per-field match status and an aggregate consistency risk. |
+| `generate_insights(score, sub, flags)` | dict | Converts numeric sub-scores into underwriter-readable bullets, risk band, and recommended action. |
+| `band(score)` | str | Maps a float to LOW / MEDIUM / HIGH / CRITICAL. Boundaries at 0.25, 0.50, 0.75. |
 
-## :package: What's included
+Constants of interest: `WEIGHTS`, `INSIGHT_RULES`, `ACTIONS`, `MODEL_PATH`, `CNN_MODEL_PATH`, `TESSERACT_OK`.
 
-| Component | What it does |
+### `app.py` — Streamlit UI
+
+Four-tab web app. Imports `forensics`, `compliance`, and `audit_report`.
+
+| Tab | Function |
 |---|---|
-| `forensics.py` | Core detection engine: ELA, copy-move, noise inconsistency, EXIF audit, PDF structure analysis, OCR + text rules, Random Forest ML model loader, cross-document consistency check |
-| `app.py` | Streamlit web app with 3 tabs and a sample picker for instant cloud demos |
-| `audit_report.py` | ReportLab-based PDF report generator with bank letterhead, risk verdict, evidence, embedded heatmaps |
-| `anomaly_detection_banking.ipynb` | Jupyter notebook walking through every detector + how to train your own classifier |
-| `sample_data/` | 26 demo documents (12 genuine + 12 tampered + 2 PDFs) ready to play with |
+| Single-document analysis | Drag-drop or pick a sample; shows risk band, sub-score breakdown, evidence list, ELA / copy-move / noise visualisations, PDF audit details, ML/CNN predictions, downloadable JSON + PDF reports. |
+| Cross-document check | Upload 2–4 documents for one applicant; the system extracts identity fields and shows a coloured comparison table with similarity scores. |
+| Batch audit | Point at a folder; scans every supported file and produces a sortable risk table + CSV. |
+| Compliance & Audit Pack | Three sub-tabs: KYC field validation (manual or doc-extracted), PII auto-redaction (PDF + text), RBI-style compliance report generation. |
+
+The sample picker auto-populates from `sample_data/`; useful for the deployed demo where users can't browse the local filesystem.
 
-## :brain: How it works
+### `audit_report.py` — bank-letterhead PDF
+
+Single public function: `build_pdf_report(report, source_path)` → `bytes`.
+
+Generates a multi-page PDF with header letterhead, metadata table, coloured risk-verdict box, sub-score breakdown table (with ASCII bar chart), evidence list, embedded heatmaps for image documents, structural audit details for PDFs, ML model verdict block, and footer disclaimer. Uses ReportLab Platypus.
+
+### `compliance.py` — KYC + regulatory
+
+| Function | Description |
+|---|---|
+| `validate_ifsc(code)` | Format check (`^[A-Z]{4}0[A-Z0-9]{6}$`) + lookup against an embedded RBI bank-code list (~36 major Indian banks). Returns bank name and branch code on success. |
+| `validate_pan(code)` | Format check (`^[A-Z]{5}\d{4}[A-Z]$`) + entity-type character validation (P=Individual, F=Firm, C=Company, etc.). |
+| `validate_aadhaar(num)` | 12-digit format + UIDAI Verhoeff checksum verification. Aadhaar numbers cannot start with 0 or 1 per UIDAI spec. |
+| `redact_text(text)` | Masks IFSC, PAN, Aadhaar, and account numbers in arbitrary text. |
+| `redact_pdf(input_path, output_path)` | Renders each PDF page, locates PII bounding boxes via `page.search_for`, overlays opaque black rectangles. |
+| `extract_pii_fields(path)` | Pulls all PII candidates from any document (PDF or image via OCR). |
+| `build_compliance_report(forensic_report, source_path, kyc_results)` | Generates a 5-section regulator-ready PDF: document ID + SHA-256, KYC verification table, fraud-screening verdict, recommended RBI risk treatment, auditor sign-off block. References specific RBI Master Directions. |
+
+### `docsentry_master.ipynb`
+
+Single source of truth. Sections:
+
+1. Environment auto-detection (Colab vs local)
+2. Datasets (synthetic generator + Kaggle CASIA v2 hook + manual download references)
+3. Image forensics
+4. PDF forensics
+5. OCR + text rules
+6. Random Forest training + saving
+7. (Optional) CNN training on Colab GPU
+8. End-to-end pipeline
+9. Cross-document consistency
+10. Dashboard + batch audit
+11. PDF report generator
+12. Export cell — writes `forensics.py`, `app.py`, `audit_report.py` to disk for the Streamlit demo
+13. Launch instructions
+
+Edit the notebook, re-run section 12, and the `.py` files used by Streamlit regenerate automatically.
+
+---
+
+## Pipeline architecture
 
 ```
-  Document ─┬─► Image forensics  (ELA, copy-move, noise, EXIF)
-            ��─► PDF structure    (EOF markers, fonts, producer/creator)
-            ├─► OCR + text rules (date monotonicity, math, IFSC validation)
-            ├─► Trained RF model (learned forgery signal blend)
-            └─► Anomaly Scorer ─► Risk band + Insights + Audit JSON
+                          ┌──────────────────────────┐
+                          │  Document (PNG/PDF/JPG)  │
+                          └────────────┬─────────────┘
+                                       │
+              ┌──────────────────┬─────┴─────┬──────────────────┐
+              ▼                  ▼           ▼                  ▼
+        ┌─────────┐         ┌─────────┐ ┌─────────┐         ┌─────────┐
+        │  ELA    │         │ Copy-   │ │ Noise   │         │  EXIF   │
+        │ analysis│         │  move   │ │ heatmap │         │  audit  │
+        └────┬────┘         └────┬────┘ └────┬────┘         └────┬────┘
+             └───────┬───────────┴───────────┴──────────┬────────┘
+                     │  (images)                        │
+                     │                                  ▼
+                     │                       ┌─────────────────────┐
+                     │                       │  OCR + text rules   │
+                     │                       │ dates · IFSC · math │
+                     │                       └──────────┬──────────┘
+                     ▼                                  │
+        ┌────────────────────────────┐                  │
+        │  Feature vector (11-dim)   │                  │
+        └──────────────┬─────────────┘                  │
+                       ▼                                │
+        ┌────────────────────────────┐                  │
+        │ Random Forest classifier   │                  │
+        │ (forgery_rf.joblib)        │                  │
+        └──────────────┬─────────────┘                  │
+                       │                                │
+                       ▼                                │
+        ┌────────────────────────────┐                  │
+        │ (Optional) CNN inference   │                  │
+        │ MobileNetV2 fine-tuned     │                  │
+        │ (forgery_cnn.keras)        │                  │
+        └──────────────┬─────────────┘                  │
+                       │                                │
+                       └──────────────┬─────────────────┘
+                                      ▼
+                      ┌──────────────────────────────┐
+                      │  Weighted ensemble scorer    │
+                      │  (rule + RF + CNN blend)     │
+                      └──────────────┬───────────────┘
+                                     ▼
+                      ┌──────────────────────────────┐
+                      │  Risk band + Evidence list   │
+                      │  Recommended action          │
+                      │  Audit JSON + PDF report     │
+                      └──────────────────────────────┘
 ```
 
-Each detector outputs a sub-score in [0, 1]. A weighted blend produces the final risk score, mapped to one of four bands. The trained Random Forest (when available) is blended 50/50 with the rule-based score. Every flag is explainable — no black-box "trust me, it's fraud."
+Each detector outputs a sub-score in `[0, 1]`. The default weight vector (in `forensics.WEIGHTS`) is `{ela: 0.20, copy_move: 0.25, noise: 0.15, exif: 0.10, pdf_struct: 0.15, text_rules: 0.10, math: 0.05}`. The Random Forest probability, when available, is blended 50/50 with the rule-based score. The CNN probability, when available, is blended at a weight between 0.4 and 0.7 depending on the CNN's reported validation AUC.
+
+---
+
+## Detection coverage
 
-## :white_check_mark: Detection capabilities
+**Image tampering**
+- Copy-move forgery — ORB keypoint matching with distance filter
+- Image splicing — block-wise noise inconsistency via Laplacian variance
+- Text edits / amount tampering — Error Level Analysis
+- Photoshop / GIMP / Snapseed edits — EXIF Software-tag string match
+- Timestamp inconsistencies — DateTime vs DateTimeOriginal comparison
 
-- **Copy-move forgery** (duplicate region within image) — ORB keypoint matching
-- **Image splicing** (region pasted from another source) — block-wise noise inconsistency
-- **Text edits / amount tampering** — Error Level Analysis (ELA)
-- **Photoshop / GIMP edits** — EXIF software-tag check
-- **PDF incremental edits** — multi-`%%EOF` marker detection
-- **PDF consumer-tool fingerprints** — iLovePDF, Smallpdf, PDFescape, Sejda producer flags
-- **Inconsistent fonts** — embedded-font count anomaly
-- **Date sequence violations** — monotonic check on extracted dates
-- **Round-number anomalies** — suspicious mega-amounts
-- **Missing IFSC with present account number** — bank-document validity rule
-- **Cross-document identity mismatches** — name/DOB/address fuzzy match across 2+ docs
+**PDF tampering**
+- Incremental edits — multi-`%%EOF` marker counting
+- Consumer-tool fingerprints — iLovePDF, Smallpdf, PDFescape, Sejda, Foxit Phantom strings in producer/creator
+- Producer/Creator mismatch — flags re-processed PDFs
+- Inserted text — embedded font count anomalies
 
-## :wrench: Run locally
+**Text-level**
+- Date sequence violations — monotonic check on extracted dates
+- Round-number anomalies — counts mega-amounts that are multiples of ₹1 lakh
+- Missing IFSC with account number present — invalid bank document
+
+**Cross-document**
+- Name / DOB / address fuzzy match across multiple documents
+- Per-field similarity scoring with green/yellow/red status
+
+**KYC validation**
+- IFSC: format + RBI bank-code list (36 banks)
+- PAN: format + entity-type character (10 types per income-tax dept spec)
+- Aadhaar: 12-digit format + UIDAI Verhoeff checksum
+
+**PII redaction**
+- Aadhaar, PAN, IFSC, account-number masking
+- PDF redaction with black rectangle overlays via `page.search_for` bounding boxes
+
+---
+
+## Running locally
 
 ```bash
-git clone https://github.com/YourName/docsentry.git
-cd docsentry
+git clone https://github.com/SpandanM110/Doc-Sentry.git
+cd Doc-Sentry
 pip install -r requirements.txt
 streamlit run app.py
 ```
 
-Opens at http://localhost:8501. Install Tesseract OCR separately for full text-rule support (https://github.com/UB-Mannheim/tesseract/wiki on Windows).
+Browser opens at `http://localhost:8501`.
+
+For full OCR text-rule support, install Tesseract OCR:
+- Windows: https://github.com/UB-Mannheim/tesseract/wiki
+- macOS: `brew install tesseract`
+- Linux: `sudo apt-get install tesseract-ocr`
+
+The app auto-detects Tesseract on standard Windows install paths; no environment variable required.
+
+See `RUN_APP.md` for a more detailed walkthrough.
+
+---
+
+## Deployment
+
+Push to GitHub, connect at https://share.streamlit.io, point at `app.py`, deploy. The `packages.txt` ensures Tesseract is installed on the Streamlit Cloud VM; `requirements.txt` covers Python dependencies.
+
+See `DEPLOY.md` for step-by-step instructions including troubleshooting.
+
+---
 
-## :cloud: Deploy your own (free)
+## Training your own model
 
-Push to a public GitHub repo, then deploy on [Streamlit Community Cloud](https://share.streamlit.io). Detailed steps in [DEPLOY.md](DEPLOY.md). Takes ~10 minutes end to end.
+Drop labelled data into `data/images/originals/` and `data/images/tampered/`, open `docsentry_master.ipynb`, run section 6. A Random Forest auto-trains on whatever you put there and saves to `models/forgery_rf.joblib`. The Streamlit app picks it up automatically on next restart — no code changes.
 
-## :books: Documentation
+For a CNN upgrade, set `TRAIN_CNN = True` in section 7 and run on a Colab T4 GPU (free tier). Saves `models/forgery_cnn.keras` + `models/forgery_cnn.meta.json`. The app loads this lazily on first request.
 
-- [DEPLOY.md](DEPLOY.md) — Streamlit Cloud deployment guide
-- [RUN_APP.md](RUN_APP.md) — Local run guide + demo flow
-- [DATASETS.md](DATASETS.md) — Public datasets (CASIA v2, MICC-F220, CoMoFoD, etc.) with download instructions
-- [COLAB_QUICKSTART.md](COLAB_QUICKSTART.md) — Google Colab usage
+See `DATASETS.md` for public datasets you can use.
 
-## :test_tube: Train your own model
+---
 
-Open `anomaly_detection_banking.ipynb`, drop your data into `data/images/originals/` and `data/images/tampered/`, run Section 7.5. The Random Forest auto-trains on whatever you put there. The app picks up the new `models/forgery_rf.joblib` automatically — no code changes.
+## Dependencies
 
-For a CNN upgrade, flip `TRAIN_CNN = True` in Section 7.6 and run on Google Colab with a free T4 GPU.
+OpenCV (cv2), Pillow (PIL), scikit-image, scikit-learn, joblib, PyMuPDF (fitz), pdfplumber, pikepdf, pytesseract, python-dateutil, Streamlit, ReportLab, NumPy, pandas, matplotlib. Optional: TensorFlow (only required for the CNN path).
 
-## :handshake: Tech stack
+All pip-installable. No GPU required for the default pipeline.
 
-OpenCV - PIL/Pillow - scikit-image - scikit-learn - PyMuPDF - pdfplumber - pikepdf - pytesseract - Streamlit - ReportLab - matplotlib - PyTorch/TensorFlow (optional)
+---
 
-Everything pip-installable. No GPU required for the default pipeline.
+## License
 
-## :scroll: License
+MIT — see `LICENSE`. The MIT license covers the source code in this repository. Third-party datasets and pretrained models bundled or referenced (CASIA v2, IDRBT cheque dataset, AgamiAI Indian Bank Statements, MobileNetV2 ImageNet weights, etc.) are governed by their own terms; those notices are reproduced in `LICENSE` below the MIT block.
 
-MIT. Use it, fork it, ship it. If you build something cool, send a screenshot.
+---
 
-## :pray: Acknowledgements
+## Acknowledgements
 
-- CASIA v2 image tampering dataset (Chinese Academy of Sciences)
-- MICC-F220 copy-move benchmark (University of Florence)
-- CoMoFoD dataset (University of Zagreb)
-- Tobacco-3482 document corpus (University of Maryland)
+- **AgamiAI Indian Bank Statements** (Hugging Face) — Apache 2.0
+- **IDRBT Cheque Image Dataset** — Institute for Development and Research in Banking Technology, India
+- **CASIA v2** image tampering dataset — Chinese Academy of Sciences
+- **MICC-F220** copy-move benchmark — University of Florence
+- **CoMoFoD** dataset — University of Zagreb
+- **Tobacco-3482** document corpus — University of Maryland