DocSentry - bank document forensics with 4 tabs

.gitattributes

@@ -0,0 +1,6 @@
+*.png filter=lfs diff=lfs merge=lfs -text
+*.jpg filter=lfs diff=lfs merge=lfs -text
+*.jpeg filter=lfs diff=lfs merge=lfs -text
+*.pdf filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.keras filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,36 @@
+# venv / Python
+.venv/
+__pycache__/
+*.pyc
+*.pyo
+.pytest_cache/
+
+# Notebook execution outputs
+.ipynb_checkpoints/
+
+# Logs & big audit dumps
+audit_log.csv
+audit_log.json
+reports/
+
+# Heavy data folders - keep sample_data/ for the live demo
+data/
+cheque data/
+
+# Trained large models (RF .joblib is OK to commit, CNN .keras is too big)
+models/forgery_cnn.keras
+
+# Outputs / temp
+*.tmp
+*.bak
+
+# OS
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+
+# Archive (the old notebooks we moved out)
+archive/

.streamlit/config.toml

@@ -0,0 +1,13 @@
+[theme]
+primaryColor = "#1e3a8a"
+backgroundColor = "#ffffff"
+secondaryBackgroundColor = "#f1f5f9"
+textColor = "#0f172a"
+font = "sans serif"
+
+[server]
+maxUploadSize = 50
+enableXsrfProtection = true
+
+[browser]
+gatherUsageStats = false

DATASETS.md

@@ -0,0 +1,195 @@
+# Datasets for Document Anomaly / Forgery Detection
+
+All datasets below are **free**. The only requirement is a free Kaggle account for the Kaggle-hosted ones. The notebook auto-picks up whatever you drop into the right folder — you do not need every dataset to get started.
+
+## Folder layout the notebook expects
+
+```
+data/
+├── images/
+│   ├── originals/        <-- genuine scans (.png/.jpg)
+│   └── tampered/         <-- forged scans (.png/.jpg)
+├── pdfs/
+│   ├── originals/        <-- genuine legal PDFs
+│   └── tampered/         <-- forged legal PDFs
+└── statements/           <-- bank statements, ITRs, receipts (any format)
+```
+
+Run `validate_data_layout()` in the notebook to confirm everything is in place.
+
+---
+
+## 1. Image tampering datasets
+
+### CASIA v2 — Gold-standard image tampering benchmark
+The most-used dataset for splicing/copy-move detection research. ~12k images (7k genuine + 5k tampered).
+
+- **Source:** https://www.kaggle.com/datasets/divg07/casia-20-image-tampering-detection-dataset
+- **Size:** ~2 GB
+- **Download:**
+  ```bash
+  kaggle datasets download -d divg07/casia-20-image-tampering-detection-dataset \
+      -p data/images --unzip
+  ```
+- **After download:** rename `Au/` → `originals/` and `Tp/` → `tampered/`
+
+### MICC-F220 — Classic copy-move benchmark
+220 images, perfect for testing copy-move detection.
+
+- **Source:** http://lci.micc.unifi.it/labd/2015/01/copy-move-forgery-detection-and-localization/
+- **Size:** ~50 MB
+- **Download:** manual (form on the page)
+
+### CoMoFoD — Copy-move with ground-truth masks
+260 image sets with masks. Ideal for training a CNN with pixel-level supervision.
+
+- **Source:** https://www.vcl.fer.hr/comofod/
+- **Size:** ~1 GB
+- **Download:** manual
+
+### Coverage — Genuine + tampered pairs
+100 pairs with similar-but-genuine objects (toughest case).
+
+- **Source:** https://github.com/wenbihan/coverage
+- **Size:** ~600 MB
+- **Download:** `git clone https://github.com/wenbihan/coverage.git`
+
+### Columbia Uncompressed Image Splicing
+180 spliced + 180 authentic images, lossless.
+
+- **Source:** https://www.ee.columbia.edu/ln/dvmm/downloads/AuthSplicedDataSet/AuthSplicedDataSet.htm
+- **Size:** ~1 GB
+- **Download:** manual (registration required, free)
+
+---
+
+## 2. Document / Legal PDF datasets
+
+### Tobacco-3482 — Real scanned legal docs
+3,482 real-world scanned legal documents (clean baseline of "genuine" docs).
+
+- **Source:** https://www.kaggle.com/datasets/patrickaudriaz/tobacco3482jpg
+- **Size:** ~200 MB
+- **Download:**
+  ```bash
+  kaggle datasets download -d patrickaudriaz/tobacco3482jpg \
+      -p data/pdfs/originals --unzip
+  ```
+
+### ICDAR Find-It — Document forgery challenge
+Official competition dataset for forged scientific documents.
+
+- **Source:** https://findit.univ-lr.fr/
+- **Size:** ~500 MB
+- **Download:** manual (registration required, free)
+
+### DocVQA / RVL-CDIP — Real bank/govt docs
+Massive dataset of real-world business documents.
+
+- **Source:** https://www.docvqa.org/datasets and https://www.cs.cmu.edu/~aharley/rvl-cdip/
+- **Size:** ~3 GB / 37 GB
+- **Use case:** populate `originals/` with realistic genuine documents
+
+### FUNSD — Form understanding
+199 fully-annotated forms (good for layout-anomaly training).
+
+- **Source:** https://guillaumejaume.github.io/FUNSD/
+- **Size:** ~50 MB
+
+---
+
+## 3. Financial statements / receipts / cheques
+
+### Receipts Fraud Detection
+500+ tampered and genuine receipt images.
+
+- **Source:** https://www.kaggle.com/datasets/trainingdatapro/receipts-fraud-detection-dataset
+- **Size:** ~100 MB
+- **Download:**
+  ```bash
+  kaggle datasets download -d trainingdatapro/receipts-fraud-detection-dataset \
+      -p data/statements --unzip
+  ```
+
+### Bank statements dataset
+Realistic bank statement PDFs and images.
+
+- **Source:** https://www.kaggle.com/datasets/dedeikhsandwisaputra/bank-statements-dataset
+- **Size:** ~80 MB
+- **Download:**
+  ```bash
+  kaggle datasets download -d dedeikhsandwisaputra/bank-statements-dataset \
+      -p data/statements --unzip
+  ```
+
+### IDRBT / Indian bank cheques
+Cheque images (Indian banking context).
+
+- **Source:** https://www.kaggle.com/datasets/arsh1207/bank-cheque-image-dataset
+- **Size:** ~50 MB
+- **Download:**
+  ```bash
+  kaggle datasets download -d arsh1207/bank-cheque-image-dataset \
+      -p data/statements --unzip
+  ```
+
+### SROIE — Scanned receipts
+Receipt OCR + key-information extraction challenge.
+
+- **Source:** https://rrc.cvc.uab.es/?ch=13
+- **Size:** ~150 MB
+
+---
+
+## 4. Land records (India-specific)
+
+There is no large public dataset for Indian land records — you have two practical options:
+
+1. **Synthesise.** The notebook already includes a `make_demo_pair()` function that generates realistic land-record images and tampered copies. You can extend this to produce hundreds of synthetic examples in minutes.
+2. **Use government open data.** Some state portals publish anonymised RoR (Record of Rights) samples — e.g.:
+   - Bhulekh portals (state-wise): https://bhulekh.gov.in/ (varies by state)
+   - DigiLocker sample certificates: https://www.digilocker.gov.in/
+3. **Use Tobacco-3482 or DocVQA as proxy** for general scanned-document forensics — the same forensic signals (ELA, copy-move, font mix) transfer directly.
+
+---
+
+## 5. Kaggle CLI setup (one-time, free)
+
+```bash
+pip install kaggle
+```
+
+1. Sign up at https://www.kaggle.com
+2. Open https://www.kaggle.com/settings → **Create New API Token**
+3. A file `kaggle.json` will download. Place it at:
+   - Windows: `C:\Users\<you>\.kaggle\kaggle.json`
+   - Linux/Mac: `~/.kaggle/kaggle.json`
+4. On Linux/Mac: `chmod 600 ~/.kaggle/kaggle.json`
+
+After that, all the `kaggle datasets download …` commands above will just work.
+
+---
+
+## 6. Minimum data needed to train
+
+The Random Forest in Section 7.5 of the notebook will give meaningful results with:
+
+- **~50 genuine + 50 tampered images** — workable baseline
+- **~200 + 200** — good results, ROC-AUC typically 0.85+
+- **~1000 + 1000** (e.g. full CASIA v2) — production-grade results
+
+For the optional CNN in Section 7.6, target at least 200 images per class.
+
+---
+
+## 7. Quick-start recipe (fastest path to working demo)
+
+1. `pip install kaggle` and set up the API token (Section 5 above)
+2. Download CASIA v2:
+   ```bash
+   kaggle datasets download -d divg07/casia-20-image-tampering-detection-dataset \
+       -p data/images --unzip
+   ```
+3. Rename the extracted `Au` → `originals` and `Tp` → `tampered`
+4. Open `anomaly_detection_banking.ipynb` and run all cells
+5. Section 7.5 will train automatically on the data you just placed

DEPLOY.md

@@ -0,0 +1,133 @@
+# Deploying DocSentry to Streamlit Community Cloud
+
+Free, public URL like `https://docsentry-yourname.streamlit.app`. Judges click a link, see your app. ~10 minutes total.
+
+## What you need
+- A free **GitHub** account (https://github.com)
+- A free **Streamlit Community Cloud** account (https://share.streamlit.io)
+- That's it — no credit card, no Docker, no AWS
+
+## Files already prepared for deployment
+
+| File | Purpose |
+|---|---|
+| `requirements.txt` | Python packages |
+| `packages.txt` | System packages (Tesseract OCR) — apt-installed on the cloud VM |
+| `.streamlit/config.toml` | Theme: blue corporate colors |
+| `.gitignore` | Excludes `.venv/`, `data/` (use `sample_data/` instead), checkpoints |
+| `sample_data/` | 26 curated demo files (~21 MB) — small enough for GitHub |
+| `app.py` | Auto-detects sample_data on startup; judges can pick a sample without uploading |
+
+## Step 1 — Create a GitHub repository (3 minutes)
+
+1. Go to https://github.com/new
+2. Repository name: `docsentry` (or whatever you want)
+3. Set to **Public** (required for free Streamlit Cloud tier)
+4. Do NOT initialize with README — we have our own
+5. Click **Create repository**
+6. Copy the URL shown — looks like `https://github.com/YourName/docsentry.git`
+
+## Step 2 — Push the project to GitHub
+
+Open PowerShell in your project folder:
+
+```powershell
+cd "C:\Users\HP\Desktop\Anomaly Based project"
+
+# Install git for Windows if you don't have it: https://git-scm.com/download/win
+git init
+git add .
+git commit -m "Initial commit: DocSentry document forensics demo"
+git branch -M main
+git remote add origin https://github.com/YourName/docsentry.git
+git push -u origin main
+```
+
+If git asks for credentials: use your GitHub username + a **personal access token** (not your password). Make a token at https://github.com/settings/tokens — give it `repo` scope.
+
+After this, refresh your GitHub page. You should see all the files: `app.py`, `forensics.py`, `audit_report.py`, `requirements.txt`, `packages.txt`, `sample_data/`, etc.
+
+## Step 3 — Deploy on Streamlit Community Cloud (5 minutes)
+
+1. Go to https://share.streamlit.io
+2. Sign in with your GitHub account (one click — it's the same login)
+3. Click **Create app** (top right)
+4. Pick **Deploy from GitHub**
+5. Fill in:
+   - **Repository**: `YourName/docsentry`
+   - **Branch**: `main`
+   - **Main file path**: `app.py`
+   - **App URL** (custom): `docsentry-yourname` (or whatever's free)
+6. Click **Deploy**
+
+Streamlit Cloud now:
+1. Reads `packages.txt` → installs Tesseract OCR on the VM
+2. Reads `requirements.txt` → pip-installs every dependency
+3. Loads `app.py` → starts the Streamlit server
+
+This takes 3-5 minutes the first time. Watch the build log on the right side of the screen.
+
+When you see **"Your app is live"**, click **Open app** — that's your public URL.
+
+## Step 4 — Test the live deployment
+
+On the live URL:
+1. **Tab 1**: pick a sample from the dropdown (e.g. `tampered/land_005_copy_move.png`) → should show **HIGH** band + heatmaps
+2. **Tab 2**: upload two different files from `sample_data/originals/` → consistency check
+3. **Tab 3**: change folder to `sample_data` → batch audit
+
+If anything errors, check the logs via the **Manage app** menu on the live page.
+
+## Step 5 — Share the link
+
+You now have a public URL. Drop it in your pitch deck, on your GitHub README, on Devpost. Judges click → 30-second cold start → working demo, no install required.
+
+## Updating the live app
+
+Push to GitHub → Streamlit Cloud auto-deploys in ~1 minute. No redeploy button needed.
+
+```powershell
+git add .
+git commit -m "Updated thresholds for higher precision"
+git push
+```
+
+The live URL stays the same.
+
+## Resource limits on the free tier
+
+- **1 GB RAM** — enough for all forensic operations on documents up to ~2000x2000 pixels
+- **CPU only** — no GPU. The Random Forest path runs fine; the CNN path (Section 7.6 in the notebook) won't fit here
+- **App sleeps after ~7 days** of zero traffic — first visitor after that waits ~30 seconds for a cold start
+- **Public access** — anyone with the URL can use it. Fine for hackathons. If you need it private, upgrade to Streamlit Teams ($25/mo) or self-host
+
+## Common issues
+
+**"ModuleNotFoundError: forensics"**
+Make sure `forensics.py` and `audit_report.py` are at the **repo root** (not in a subfolder). Streamlit Cloud runs `app.py` from the repo root and Python only finds modules in the same folder.
+
+**"tesseract: not found" on the live app**
+The `packages.txt` file might not have been committed. Run `git status` and `git add packages.txt`, then `git push`. Verify on GitHub that `packages.txt` exists in the root.
+
+**Sample data dropdown is empty on live app**
+The `.gitignore` excludes `data/` but not `sample_data/`. Check on GitHub that the `sample_data/` folder is visible. If it's missing, you might have a leftover `data/` exclusion. Remove `sample_data/` from `.gitignore` if so.
+
+**Build times out**
+Free tier build limit is ~15 minutes. If you exceed, trim `requirements.txt` (e.g. remove `seaborn`, `imagehash`, `exifread` if unused). They're nice-to-have but not core.
+
+**`models/forgery_rf.joblib` missing**
+Train it locally first (run Section 7.5 in the notebook) then commit + push. The app gracefully falls back to rule-based-only scoring if the model is missing.
+
+**Memory error on large uploads**
+Free tier has 1 GB RAM. Tell judges to use PNG/JPG under 5 MB or downscale large scans first. Or upgrade to Teams ($25/mo, 8 GB RAM) for the hackathon.
+
+## Alternative: Hugging Face Spaces (also free)
+
+If Streamlit Cloud is slow on your day of demo, the same code deploys to Hugging Face Spaces with no changes:
+
+1. Create a Space at https://huggingface.co/new-space
+2. Pick **Streamlit** as the SDK
+3. Push to its git repo using your HF write token
+4. Done. Public URL like `huggingface.co/spaces/yourname/docsentry`
+
+HF Spaces gives you 16 GB RAM free vs Streamlit's 1 GB, but the URL is uglier. Use whichever wins on the day.

PUSH.md

@@ -0,0 +1,90 @@
+# Push to GitHub - One-time setup
+
+Your repo: https://github.com/SpandanM110/Doc-Sentry.git
+
+The `.gitignore` is configured to exclude the 826 MB `cheque data/` folder, the
+~250 MB `data/` folder, the `.venv`, and other heavy stuff. **Only ~400 KB of
+code, docs, and 26 sample files will be pushed.**
+
+## Step 1 - Open PowerShell in your project folder
+
+```powershell
+cd "C:\Users\HP\Desktop\Anomaly Based project"
+```
+
+## Step 2 - Initialize git + commit
+
+Run these one by one (or paste all at once):
+
+```powershell
+git init -b main
+git config user.email "spandanmukherjeegithub@gmail.com"
+git config user.name "Spandan"
+
+git add .
+git status      # confirm only ~30 files staged (not the big data/ folders)
+git commit -m "Initial commit: DocSentry - bank document forensics with 4 tabs"
+```
+
+## Step 3 - Connect to GitHub + push
+
+```powershell
+git remote add origin https://github.com/SpandanM110/Doc-Sentry.git
+git push -u origin main
+```
+
+When prompted for credentials:
+- **Username:** your GitHub username (`SpandanM110`)
+- **Password:** a **Personal Access Token** (NOT your GitHub password)
+
+If you don't have a PAT yet:
+1. Go to https://github.com/settings/tokens
+2. Click **Generate new token (classic)**
+3. Give it a name (e.g. "DocSentry repo")
+4. Tick the **repo** scope
+5. Click Generate, then **copy the token** (it's only shown once)
+6. Paste it as the password when git prompts
+
+## Step 4 - Verify
+
+Open https://github.com/SpandanM110/Doc-Sentry in your browser.
+You should see all your files: `app.py`, `forensics.py`, `compliance.py`,
+`docsentry_master.ipynb`, the markdown docs, and the `sample_data/` folder.
+
+## Future pushes (after first time)
+
+```powershell
+git add .
+git commit -m "describe what you changed"
+git push
+```
+
+The push uses your cached credentials automatically after the first time.
+
+## If something goes wrong
+
+**"Repository already exists" or other init errors**
+
+```powershell
+# delete the .git folder and start over
+rmdir /s /q .git
+git init -b main
+```
+
+**Files too large to push**
+
+If you ever accidentally add a large file (> 100 MB):
+
+```powershell
+git rm --cached path/to/big/file
+echo path/to/big/file >> .gitignore
+git add .gitignore
+git commit -m "exclude large file"
+```
+
+**Want to add the cheque data later?**
+
+It's 826 MB - GitHub will reject it. Two options:
+1. Use Git LFS (`git lfs install` + `git lfs track "*.tif"`)
+2. Document the dataset source in DATASETS.md instead and have users
+   download it themselves (recommended)

README.md

@@ -0,0 +1,103 @@
+# DocSentry - Document Forensics for Bank Underwriting
+
+[![Built with Streamlit](https://img.shields.io/badge/Built%20with-Streamlit-FF4B4B.svg)](https://streamlit.io)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Real-time anomaly detection for land records, legal documents, and financial statements during loan underwriting.**
+
+> Detects tampering, edits, and forgery attempts in seconds. Generates explainable risk scores and underwriter-ready audit reports. 100% open-source, no paid APIs, no LLM calls. Runs on a laptop.
+
+## :rocket: Live demo
+
+**Try it now:** [docsentry-yourname.streamlit.app](https://docsentry-yourname.streamlit.app) *(replace with your actual URL after deploying)*
+
+Pick a tampered land record from the dropdown and watch the forensic verdict appear in real time.
+
+## :movie_camera: 3-minute demo flow
+
+1. **Single-document analysis** — drop a PNG/PDF, see risk band (LOW/MEDIUM/HIGH/CRITICAL), heatmap overlays, evidence bullets
+2. **Cross-document consistency** — upload 2-4 documents for the same applicant; system flags mismatches in name/DOB/address
+3. **Batch audit** — point at a folder of documents; get a sortable risk table and CSV export
+4. **Audit PDF** — one-click download of a bank-letterhead PDF report ready for the underwriting file
+
+## :package: What's included
+
+| Component | What it does |
+|---|---|
+| `forensics.py` | Core detection engine: ELA, copy-move, noise inconsistency, EXIF audit, PDF structure analysis, OCR + text rules, Random Forest ML model loader, cross-document consistency check |
+| `app.py` | Streamlit web app with 3 tabs and a sample picker for instant cloud demos |
+| `audit_report.py` | ReportLab-based PDF report generator with bank letterhead, risk verdict, evidence, embedded heatmaps |
+| `anomaly_detection_banking.ipynb` | Jupyter notebook walking through every detector + how to train your own classifier |
+| `sample_data/` | 26 demo documents (12 genuine + 12 tampered + 2 PDFs) ready to play with |
+
+## :brain: How it works
+
+```
+  Document ─┬─► Image forensics  (ELA, copy-move, noise, EXIF)
+            ├─► PDF structure    (EOF markers, fonts, producer/creator)
+            ├─► OCR + text rules (date monotonicity, math, IFSC validation)
+            ├─► Trained RF model (learned forgery signal blend)
+            └─► Anomaly Scorer ─► Risk band + Insights + Audit JSON
+```
+
+Each detector outputs a sub-score in [0, 1]. A weighted blend produces the final risk score, mapped to one of four bands. The trained Random Forest (when available) is blended 50/50 with the rule-based score. Every flag is explainable — no black-box "trust me, it's fraud."
+
+## :white_check_mark: Detection capabilities
+
+- **Copy-move forgery** (duplicate region within image) — ORB keypoint matching
+- **Image splicing** (region pasted from another source) — block-wise noise inconsistency
+- **Text edits / amount tampering** — Error Level Analysis (ELA)
+- **Photoshop / GIMP edits** — EXIF software-tag check
+- **PDF incremental edits** — multi-`%%EOF` marker detection
+- **PDF consumer-tool fingerprints** — iLovePDF, Smallpdf, PDFescape, Sejda producer flags
+- **Inconsistent fonts** — embedded-font count anomaly
+- **Date sequence violations** — monotonic check on extracted dates
+- **Round-number anomalies** — suspicious mega-amounts
+- **Missing IFSC with present account number** — bank-document validity rule
+- **Cross-document identity mismatches** — name/DOB/address fuzzy match across 2+ docs
+
+## :wrench: Run locally
+
+```bash
+git clone https://github.com/YourName/docsentry.git
+cd docsentry
+pip install -r requirements.txt
+streamlit run app.py
+```
+
+Opens at http://localhost:8501. Install Tesseract OCR separately for full text-rule support (https://github.com/UB-Mannheim/tesseract/wiki on Windows).
+
+## :cloud: Deploy your own (free)
+
+Push to a public GitHub repo, then deploy on [Streamlit Community Cloud](https://share.streamlit.io). Detailed steps in [DEPLOY.md](DEPLOY.md). Takes ~10 minutes end to end.
+
+## :books: Documentation
+
+- [DEPLOY.md](DEPLOY.md) — Streamlit Cloud deployment guide
+- [RUN_APP.md](RUN_APP.md) — Local run guide + demo flow
+- [DATASETS.md](DATASETS.md) — Public datasets (CASIA v2, MICC-F220, CoMoFoD, etc.) with download instructions
+- [COLAB_QUICKSTART.md](COLAB_QUICKSTART.md) — Google Colab usage
+
+## :test_tube: Train your own model
+
+Open `anomaly_detection_banking.ipynb`, drop your data into `data/images/originals/` and `data/images/tampered/`, run Section 7.5. The Random Forest auto-trains on whatever you put there. The app picks up the new `models/forgery_rf.joblib` automatically — no code changes.
+
+For a CNN upgrade, flip `TRAIN_CNN = True` in Section 7.6 and run on Google Colab with a free T4 GPU.
+
+## :handshake: Tech stack
+
+OpenCV - PIL/Pillow - scikit-image - scikit-learn - PyMuPDF - pdfplumber - pikepdf - pytesseract - Streamlit - ReportLab - matplotlib - PyTorch/TensorFlow (optional)
+
+Everything pip-installable. No GPU required for the default pipeline.
+
+## :scroll: License
+
+MIT. Use it, fork it, ship it. If you build something cool, send a screenshot.
+
+## :pray: Acknowledgements
+
+- CASIA v2 image tampering dataset (Chinese Academy of Sciences)
+- MICC-F220 copy-move benchmark (University of Florence)
+- CoMoFoD dataset (University of Zagreb)
+- Tobacco-3482 document corpus (University of Maryland)

RUN_APP.md

@@ -0,0 +1,117 @@
+# Running the DocSentry Demo App
+
+## Quick start (3 commands)
+
+```powershell
+# From: C:\Users\HP\Desktop\Anomaly Based project\
+
+pip install -r requirements.txt
+streamlit run app.py
+```
+
+That's it. Streamlit will open the app at http://localhost:8501 in your browser.
+
+## Optional: install Tesseract OCR (for full text-rule checks)
+
+1. Download from https://github.com/UB-Mannheim/tesseract/wiki
+2. Run the `.exe` installer
+3. Check "Add Tesseract to system PATH" during install
+4. Restart your terminal, then `streamlit run app.py` again
+
+The app works without Tesseract too — only the text/OCR-based checks are skipped.
+
+## What's in the app
+
+**Tab 1 — Single-document analysis** *(your primary demo)*
+- Drag-drop a PNG / JPG / PDF
+- See risk band (LOW / MEDIUM / HIGH / CRITICAL) in big colored text
+- Sub-score breakdown bar chart
+- ELA heatmap, copy-move match visualization, noise inconsistency heatmap (image files)
+- Producer / creator / fonts table (PDFs)
+- Trained ML model verdict (if `models/forgery_rf.joblib` exists)
+- Download audit JSON or formatted PDF report
+
+**Tab 2 — Cross-document consistency check** *(the novel angle)*
+- Upload 2–4 documents for the same applicant
+- App extracts name, DOB, address, account, IFSC from each
+- Field-by-field comparison table with green/yellow/red status
+- Mismatch detector with similarity scores
+- Download consistency report JSON
+
+**Tab 3 — Batch audit**
+- Point at a folder, scan every file in it
+- Get risk band per file as a sortable table
+- Download CSV for the underwriting team
+
+## Demo flow (for the pitch)
+
+1. **Open Tab 1**, drop in a clean `land_000.png` from `data/images/originals/` → "LOW" green band, no evidence
+2. **Drop in a `land_005_copy_move.png`** from `data/images/tampered/` → "HIGH" orange band, copy-move evidence
+3. **Click through the heatmap tabs** → judges see real visualizations, not just numbers
+4. **Click "Download audit PDF"** → a bank-letterhead report renders
+5. **Switch to Tab 2**, upload 2 different `agreement_*.png` files → "HIGH" mismatch because names differ
+6. **Switch to Tab 3**, point at `data/` → batch-process 250+ files in a few seconds
+
+Total demo time: 3 minutes. Hands the judges something to download.
+
+## Project file layout
+
+```
+C:\Users\HP\Desktop\Anomaly Based project\
+├── app.py                              <-- Streamlit UI (this app)
+├── forensics.py                        <-- Core detection module
+├── audit_report.py                     <-- PDF report generator
+├── requirements.txt                    <-- pip dependencies
+├── RUN_APP.md                          <-- this file
+├── DATASETS.md                         <-- where to get more training data
+├── COLAB_QUICKSTART.md                 <-- Colab usage guide
+├── anomaly_detection_banking.ipynb     <-- local Jupyter notebook
+├── anomaly_detection_banking_COLAB.ipynb <-- Colab notebook
+├── data/
+│   ├── images/originals/        130 genuine docs
+│   ├── images/tampered/         130 tampered docs
+│   ├── pdfs/originals/          30 PDFs
+│   ├── pdfs/tampered/           30 tampered PDFs
+│   └── statements/              60 statements
+└── models/
+    └── forgery_rf.joblib        (created after running Section 7.5 in the notebook)
+```
+
+## Common issues
+
+**"ModuleNotFoundError: No module named 'forensics'"**
+You're not in the project folder. `cd "C:\Users\HP\Desktop\Anomaly Based project"` first, then `streamlit run app.py`.
+
+**"streamlit: command not found"**
+Streamlit didn't install. Re-run `pip install -r requirements.txt`. On Windows, you may need `python -m streamlit run app.py` instead.
+
+**The "Download audit PDF" button shows a warning**
+Make sure `reportlab` installed cleanly. Re-run `pip install reportlab`.
+
+**Cross-doc tab says "ocr_skipped" for every field**
+You don't have Tesseract installed. The forensic checks still work; only the cross-doc field extraction needs OCR. Install Tesseract (see above) to unlock that tab.
+
+**The trained ML model section doesn't appear**
+You haven't run Section 7.5 in the notebook yet. Open `anomaly_detection_banking.ipynb`, run all cells through Section 7.5; that creates `models/forgery_rf.joblib`. The Streamlit app picks it up automatically.
+
+## Architecture sketch
+
+```
+┌──────────────────────────────────────────────┐
+│              app.py  (Streamlit)             │
+│   Tab1: Single doc   Tab2: Cross-doc         │
+│   Tab3: Batch audit  Downloads: JSON + PDF   │
+└──────────────────┬───────────────────────────┘
+                   │ imports
+        ┌──────────┴──────────┐
+        ▼                     ▼
+   forensics.py        audit_report.py
+   - ELA / copy-move    - ReportLab PDF
+   - noise / EXIF       - Heatmap embeds
+   - PDF audit          - Bank letterhead
+   - OCR + text rules   - Evidence list
+   - RF model load
+   - Cross-doc check
+```
+
+All three files run on plain Python 3.10+, CPU-only, no paid APIs.

app.py

@@ -0,0 +1,522 @@
+"""
+app.py - Streamlit demo for real-time document anomaly detection
+
+Run with:
+    streamlit run app.py
+
+Tabs:
+    1. Single-document analysis  - drag-drop one file, see forensic verdict
+    2. Cross-document check       - upload 2+ docs, check identity consistency
+    3. Batch audit                - point at a folder, get an audit CSV
+"""
+
+import io
+import json
+import tempfile
+from pathlib import Path
+
+import streamlit as st
+import numpy as np
+import pandas as pd
+import cv2
+from PIL import Image
+import matplotlib.pyplot as plt
+
+import forensics
+import compliance
+from audit_report import build_pdf_report   # Sprint 3 module
+
+# -------------------------------------------------------------
+# Page config + global CSS
+# -------------------------------------------------------------
+st.set_page_config(page_title="BankShield - Document Forensics",
+                   page_icon=":lock:", layout="wide")
+
+st.markdown("""
+<style>
+.big-risk {font-size: 48px; font-weight: 800; padding: 14px 28px;
+           border-radius: 12px; color: white; text-align: center;
+           letter-spacing: 1px;}
+.low      {background: #16a34a;}
+.medium   {background: #ca8a04;}
+.high     {background: #ea580c;}
+.critical {background: #dc2626;}
+.metric-card {background: #f8fafc; padding: 14px; border-radius: 8px;
+              border-left: 4px solid #2563eb;}
+</style>
+""", unsafe_allow_html=True)
+
+# -------------------------------------------------------------
+# Header
+# -------------------------------------------------------------
+st.title(":shield: BankShield - Document Forensics")
+st.caption("Real-time anomaly detection for underwriting. "
+           "Land records | Legal documents | Financial statements.")
+
+if not forensics.TESSERACT_OK:
+    st.warning("Tesseract OCR is not installed - text-rule checks will be skipped. "
+               "Install from https://github.com/UB-Mannheim/tesseract/wiki for full functionality.")
+
+# -------------------------------------------------------------
+# Helpers
+# -------------------------------------------------------------
+def risk_badge(band_str):
+    klass = band_str.lower()
+    st.markdown(f"<div class='big-risk {klass}'>{band_str}</div>",
+                unsafe_allow_html=True)
+
+
+def save_uploaded(uploaded_file):
+    """Persist an uploaded file to a temp path; return Path."""
+    suffix = Path(uploaded_file.name).suffix
+    tmp = tempfile.NamedTemporaryFile(delete=False, suffix=suffix)
+    tmp.write(uploaded_file.getbuffer())
+    tmp.close()
+    return Path(tmp.name)
+
+
+def sub_score_chart(sub_scores):
+    fig, ax = plt.subplots(figsize=(7, 3.2))
+    keys = list(sub_scores.keys())
+    vals = list(sub_scores.values())
+    colours = ["#16a34a" if v < 0.4 else "#ea580c" if v < 0.7 else "#dc2626"
+               for v in vals]
+    ax.barh(keys, vals, color=colours)
+    ax.set_xlim(0, 1)
+    ax.set_xlabel("score (0 = clean, 1 = suspicious)")
+    ax.set_title("Sub-score breakdown")
+    ax.invert_yaxis()
+    plt.tight_layout()
+    return fig
+
+
+# -------------------------------------------------------------
+# TABS
+# -------------------------------------------------------------
+tab1, tab2, tab3, tab4 = st.tabs([
+    ":mag: Single-document analysis",
+    ":busts_in_silhouette: Cross-document check",
+    ":file_folder: Batch audit",
+    ":scales: Compliance & Audit Pack",
+])
+
+# =============================================================
+# TAB 1 - Single document
+# =============================================================
+with tab1:
+    # Sample picker: lets cloud demos work without an upload
+    sample_dir = Path("sample_data")
+    sample_paths = []
+    if sample_dir.exists():
+        for sub in ("originals", "tampered", "pdfs"):
+            sample_paths.extend(sorted((sample_dir / sub).glob("*")))
+    sample_options = ["(upload your own)"] + [str(p.relative_to(sample_dir)) for p in sample_paths]
+
+    pick = st.selectbox("Try a sample document, or upload your own:", sample_options, key="sample_pick")
+
+    path = None
+    if pick != "(upload your own)":
+        path = sample_dir / pick
+        st.caption(f"Loaded sample: `{pick}`")
+    else:
+        uploaded = st.file_uploader(
+            "Upload a document (PNG / JPG / PDF)",
+            type=["png", "jpg", "jpeg", "pdf", "tif", "tiff"],
+            key="single",
+        )
+        if uploaded:
+            path = save_uploaded(uploaded)
+
+    if path is not None:
+        with st.spinner("Analyzing forensic signals..."):
+            report = forensics.analyse_document(path)
+
+        # --- top row: risk badge + action ---
+        c1, c2 = st.columns([1, 2])
+        with c1:
+            risk_badge(report["risk_band"])
+            st.metric("Risk score", f"{report['risk_score']:.3f}")
+        with c2:
+            st.markdown("**Recommended action**")
+            st.info(report["recommended_action"])
+            st.markdown("**Evidence**")
+            for e in report["evidence"]:
+                st.markdown(f"- {e}")
+
+        st.divider()
+
+        # --- detail row: image preview + sub-scores ---
+        left, right = st.columns([1, 1])
+        with left:
+            st.markdown("#### Document preview")
+            if report["type"] == "image":
+                st.image(str(path), use_container_width=True)
+            elif report["type"] == "pdf":
+                import fitz
+                with fitz.open(path) as d:
+                    pix = d[0].get_pixmap(dpi=110)
+                    img = Image.frombytes("RGB", (pix.width, pix.height), pix.samples)
+                    st.image(img, use_container_width=True)
+
+        with right:
+            st.markdown("#### Sub-score breakdown")
+            sub = report.get("sub_scores")
+            if sub:
+                st.pyplot(sub_score_chart(sub))
+
+        # --- forensic visualizations (images only) ---
+        if report["type"] == "image":
+            st.divider()
+            st.markdown("#### Forensic visualizations")
+            tabs_viz = st.tabs(["Error Level Analysis", "Copy-move matches", "Noise heatmap"])
+            with tabs_viz[0]:
+                ela_img, ela_s = forensics.error_level_analysis(path)
+                st.image(ela_img, caption=f"ELA score: {ela_s:.2f}",
+                         use_container_width=True)
+                st.caption("Bright regions = compression-inconsistent areas (likely edited).")
+            with tabs_viz[1]:
+                viz, n, _ = forensics.copy_move_detect(path)
+                st.image(cv2.cvtColor(viz, cv2.COLOR_BGR2RGB),
+                         caption=f"Copy-move matches: {n}",
+                         use_container_width=True)
+                st.caption("Red lines connect duplicated regions within the image.")
+            with tabs_viz[2]:
+                heat, ratio = forensics.noise_inconsistency(path)
+                fig, ax = plt.subplots(figsize=(6, 4))
+                ax.imshow(heat, cmap="hot")
+                ax.set_title(f"Noise outlier ratio: {ratio:.2%}")
+                ax.axis("off")
+                st.pyplot(fig)
+                st.caption("Hotspots = local noise inconsistencies (splicing signal).")
+
+        # --- PDF-specific audit details ---
+        if report["type"] == "pdf":
+            st.divider()
+            st.markdown("#### PDF structural audit")
+            audit = report.get("pdf_audit", {})
+            fonts = report.get("font_audit", {})
+            cc1, cc2 = st.columns(2)
+            with cc1:
+                st.metric("EOF markers", audit.get("eof_markers", "-"))
+                st.metric("Pages", audit.get("pages", "-"))
+                st.markdown("**Metadata flags:**")
+                for f in audit.get("flags", []):
+                    st.markdown(f"- {f}")
+            with cc2:
+                meta = audit.get("metadata", {}) or {}
+                st.markdown("**Producer:** " + str(meta.get("producer", "-")))
+                st.markdown("**Creator:** "  + str(meta.get("creator", "-")))
+                st.markdown("**Fonts used:** " + ", ".join(fonts.get("fonts", []) or ["-"]))
+                st.markdown("**Font flags:**")
+                for f in fonts.get("flags", []):
+                    st.markdown(f"- {f}")
+
+        # --- ML predictions (RF + CNN side-by-side if available) ---
+        has_rf  = "ml_prediction"  in report
+        has_cnn = "cnn_prediction" in report
+        if has_rf or has_cnn:
+            st.divider()
+            st.markdown("#### Trained model verdicts")
+            cols = st.columns(2 if (has_rf and has_cnn) else 1)
+            ci = 0
+            if has_rf:
+                ml = report["ml_prediction"]
+                with cols[ci]:
+                    st.markdown("**Random Forest** (forensic features)")
+                    cc1, cc2 = st.columns(2)
+                    cc1.metric("Tamper probability", f"{ml['tamper_probability']:.1%}")
+                    cc2.metric("Verdict", ml["verdict"])
+                ci += 1
+            if has_cnn:
+                cnn = report["cnn_prediction"]
+                with cols[ci]:
+                    st.markdown("**CNN** (MobileNetV2 on CASIA v2)")
+                    cc1, cc2 = st.columns(2)
+                    cc1.metric("Tamper probability", f"{cnn['tamper_probability']:.1%}")
+                    cc2.metric("Verdict", cnn["verdict"])
+                    if cnn.get("val_auc"):
+                        st.caption(f"Model val ROC-AUC: {cnn['val_auc']:.3f}")
+
+        # --- downloads ---
+        st.divider()
+        dl1, dl2 = st.columns(2)
+        with dl1:
+            st.download_button(
+                "Download audit JSON",
+                data=json.dumps(report, indent=2, default=str),
+                file_name=f"audit_{path.stem}.json",
+                mime="application/json",
+            )
+        with dl2:
+            try:
+                pdf_bytes = build_pdf_report(report, path)
+                st.download_button(
+                    "Download audit PDF report",
+                    data=pdf_bytes,
+                    file_name=f"audit_report_{path.stem}.pdf",
+                    mime="application/pdf",
+                )
+            except Exception as e:
+                st.warning(f"PDF report generation skipped: {e}")
+
+# =============================================================
+# TAB 2 - Cross-document consistency
+# =============================================================
+with tab2:
+    st.markdown("Upload 2 or more documents for the **same applicant** "
+                "(e.g. land record + bank statement + ID). The system will "
+                "extract identity fields and flag any mismatches.")
+
+    uploads = st.file_uploader(
+        "Upload 2-4 documents",
+        type=["png", "jpg", "jpeg", "pdf"],
+        accept_multiple_files=True,
+        key="multi",
+    )
+
+    if uploads and len(uploads) >= 2:
+        paths = [save_uploaded(u) for u in uploads]
+        with st.spinner("Extracting identity fields from each document..."):
+            result = forensics.cross_doc_consistency(paths)
+
+        # --- header ---
+        c1, c2 = st.columns([1, 2])
+        with c1:
+            risk_badge(result["consistency_band"])
+            st.metric("Consistency risk", f"{result['consistency_risk_score']:.3f}")
+        with c2:
+            st.metric("Mismatches", result["mismatches"])
+            st.metric("Likely mismatches", result["likely_mismatches"])
+
+        st.divider()
+        st.markdown("#### Field-by-field comparison")
+
+        # Build a comparison table
+        field_rows = []
+        files = [Path(p).name for p in paths]
+        for field, res in result["field_results"].items():
+            row = {"Field": field, "Status": res["status"],
+                   "Similarity": res.get("similarity")}
+            for fn, val in zip(files, res["values"]):
+                row[fn] = val or "(not found)"
+            field_rows.append(row)
+        df = pd.DataFrame(field_rows)
+
+        def colour_status(val):
+            if val == "match":          return "background-color: #dcfce7"
+            if val == "likely_match":   return "background-color: #fef3c7"
+            if val == "mismatch":       return "background-color: #fecaca"
+            return ""
+        st.dataframe(df.style.applymap(colour_status, subset=["Status"]),
+                     use_container_width=True)
+
+        st.divider()
+        st.markdown("#### Per-document extracts")
+        for doc in result["documents"]:
+            with st.expander(Path(doc["file"]).name):
+                st.json(doc["fields"])
+
+        st.download_button(
+            "Download consistency report JSON",
+            data=json.dumps(result, indent=2, default=str),
+            file_name="cross_doc_consistency.json",
+            mime="application/json",
+        )
+    elif uploads:
+        st.info("Upload at least 2 documents to run the cross-check.")
+
+# =============================================================
+# TAB 3 - Batch audit
+# =============================================================
+with tab3:
+    st.markdown("Point at a folder on your machine to run a batch audit. "
+                "Produces a CSV with risk band per file.")
+    default = Path.cwd() / ("sample_data" if not (Path.cwd() / "data").exists() else "data")
+    folder = st.text_input("Folder path", value=str(default))
+    if st.button("Run batch audit"):
+        root = Path(folder)
+        if not root.exists():
+            st.error(f"Folder not found: {root}")
+        else:
+            results = []
+            files = [p for p in root.rglob("*")
+                     if p.suffix.lower() in {".png", ".jpg", ".jpeg", ".pdf", ".tif"}]
+            if not files:
+                st.warning("No supported files found in folder.")
+            else:
+                progress = st.progress(0.0)
+                for i, p in enumerate(files):
+                    try:
+                        r = forensics.analyse_document(p)
+                        results.append({
+                            "file": str(p.relative_to(root)),
+                            "type": r.get("type"),
+                            "risk_score": r.get("risk_score"),
+                            "risk_band": r.get("risk_band"),
+                            "action": r.get("recommended_action"),
+                        })
+                    except Exception as e:
+                        results.append({"file": str(p), "error": str(e)})
+                    progress.progress((i + 1) / len(files))
+                df = pd.DataFrame(results)
+                st.success(f"Analysed {len(files)} files.")
+                st.dataframe(df, use_container_width=True)
+                csv = df.to_csv(index=False).encode("utf-8")
+                st.download_button("Download audit CSV", data=csv,
+                                   file_name="audit_log.csv", mime="text/csv")
+
+# =============================================================
+# TAB 4 - Compliance & Audit Pack (KYC + PII redaction + RBI report)
+# =============================================================
+with tab4:
+    st.markdown("**Three regulatory tools in one tab** - KYC field validation, "
+                "PII auto-redaction, and RBI-style compliance reports.")
+    sub_a, sub_b, sub_c = st.tabs([
+        ":id: KYC Field Validation",
+        ":lock: PII Auto-Redaction",
+        ":scroll: RBI Compliance Report",
+    ])
+
+    # -------- 4A: KYC validators (manual input) --------
+    with sub_a:
+        st.markdown("#### Validate KYC fields against RBI rules")
+        st.caption("IFSC: format + RBI bank-code list  |  PAN: format + entity-type "
+                   "char  |  Aadhaar: 12-digit + UIDAI Verhoeff checksum.")
+
+        c1, c2, c3 = st.columns(3)
+        ifsc_in = c1.text_input("IFSC code", value="SBIN0001234")
+        pan_in  = c2.text_input("PAN", value="ABCPQ1234F")
+        aad_in  = c3.text_input("Aadhaar number (12 digits)", value="234567890124")
+
+        if st.button("Validate all", key="kyc_validate"):
+            r_ifsc = compliance.validate_ifsc(ifsc_in)
+            r_pan  = compliance.validate_pan(pan_in)
+            r_aad  = compliance.validate_aadhaar(aad_in)
+            for label, r in [("IFSC", r_ifsc), ("PAN", r_pan), ("Aadhaar", r_aad)]:
+                if r["ok"]:
+                    st.success("**" + label + "**: VALID. " + " ".join(r["flags"]))
+                    if label == "IFSC":
+                        bn = r.get("bank_name", "-")
+                        bc = r.get("branch_code", "-")
+                        st.caption("Bank: " + bn + ", branch code: " + bc)
+                    if label == "PAN":
+                        et = r.get("entity_type", "-")
+                        st.caption("Entity type: " + et)
+                    if label == "Aadhaar":
+                        mk = r.get("masked", "-")
+                        st.caption("Masked: " + mk)
+                else:
+                    st.error("**" + label + "**: INVALID. " + " | ".join(r["flags"]))
+
+        st.divider()
+        st.markdown("#### Or: extract & validate from a document")
+        kyc_file = st.file_uploader("Upload doc to scan for KYC fields",
+                                     type=["pdf", "png", "jpg"], key="kyc_doc")
+        if kyc_file:
+            kyc_path = save_uploaded(kyc_file)
+            with st.spinner("Extracting KYC fields..."):
+                fields, _ = compliance.extract_pii_fields(kyc_path)
+            n_ifsc = len(fields["ifsc"])
+            n_pan = len(fields["pan"])
+            n_aad = len(fields["aadhaar"])
+            n_acc = len(fields["accounts"])
+            st.markdown("**Found in document:** " + str(n_ifsc) + " IFSC, " +
+                        str(n_pan) + " PAN, " + str(n_aad) + " Aadhaar candidates, " +
+                        str(n_acc) + " account numbers")
+            # Validate unique IFSCs (first 5)
+            uniq_ifsc = list(set(fields["ifsc"]))[:5]
+            if uniq_ifsc:
+                st.markdown("**IFSC validation (first 5 unique):**")
+                rows = [compliance.validate_ifsc(c) for c in uniq_ifsc]
+                st.dataframe(pd.DataFrame(rows), use_container_width=True)
+            if fields["pan"]:
+                st.markdown("**PAN validation:**")
+                rows = [compliance.validate_pan(c) for c in fields["pan"][:5]]
+                st.dataframe(pd.DataFrame(rows), use_container_width=True)
+
+    # -------- 4B: PII redaction --------
+    with sub_b:
+        st.markdown("#### Auto-redact PII for safe sharing")
+        st.caption("Masks IFSC, PAN, Aadhaar, and account numbers. Use before "
+                   "forwarding to external vendors / for DPDP Act compliance.")
+        rd_file = st.file_uploader("Upload document to redact",
+                                    type=["pdf", "png", "jpg"], key="rd")
+        if rd_file:
+            src_path = save_uploaded(rd_file)
+            if str(src_path).lower().endswith(".pdf"):
+                out_path = Path(tempfile.gettempdir()) / f"redacted_{src_path.stem}.pdf"
+                with st.spinner("Redacting PDF..."):
+                    found = compliance.redact_pdf(str(src_path), str(out_path))
+                total = sum(len(v) for v in found.values())
+                st.success("Redacted " + str(total) + " PII items.")
+                summary = {k: len(v) for k, v in found.items()}
+                st.json(summary)
+                with open(out_path, "rb") as f:
+                    st.download_button("Download redacted PDF", f.read(),
+                                        file_name=out_path.name, mime="application/pdf")
+            else:
+                # image - just OCR + redact text
+                fields, text = compliance.extract_pii_fields(src_path)
+                red_text, _ = compliance.redact_text(text)
+                st.markdown("**Original (OCR):**")
+                st.code(text[:600], language=None)
+                st.markdown("**Redacted:**")
+                st.code(red_text[:600], language=None)
+                st.download_button("Download redacted text", red_text,
+                                    file_name=f"redacted_{src_path.stem}.txt")
+
+    # -------- 4C: RBI compliance report --------
+    with sub_c:
+        st.markdown("#### Generate an RBI Master-Direction-style audit PDF")
+        st.caption("Runs full forensic analysis + KYC verification + RBI risk-treatment "
+                   "recommendation, then produces a regulator-ready PDF.")
+        cr_file = st.file_uploader("Upload document for compliance audit",
+                                    type=["pdf", "png", "jpg"], key="cr")
+        if cr_file:
+            src_path = save_uploaded(cr_file)
+            with st.spinner("Running forensic analysis..."):
+                f_report = forensics.analyse_document(src_path)
+            with st.spinner("Validating KYC fields..."):
+                fields, _ = compliance.extract_pii_fields(src_path)
+                kyc_results = {}
+                if fields["ifsc"]:
+                    kyc_results["ifsc"] = compliance.validate_ifsc(fields["ifsc"][0])
+                if fields["pan"]:
+                    kyc_results["pan"] = compliance.validate_pan(fields["pan"][0])
+                if fields["aadhaar"]:
+                    kyc_results["aadhaar"] = compliance.validate_aadhaar(fields["aadhaar"][0])
+
+            # Summary cards
+            cc1, cc2, cc3 = st.columns(3)
+            cc1.metric("Forensic risk", f_report.get("risk_band", "-"))
+            cc2.metric("KYC fields found",
+                        sum(len(fields[k]) for k in ("ifsc", "pan", "aadhaar")))
+            cc3.metric("KYC checks passed",
+                        sum(1 for r in kyc_results.values() if r.get("ok")))
+
+            # KYC table
+            if kyc_results:
+                rows = [{"Field": k.upper(), "Value": r.get("code", "-"),
+                         "Status": "PASS" if r.get("ok") else "FAIL",
+                         "Notes": "; ".join(r.get("flags", []))[:60]}
+                        for k, r in kyc_results.items()]
+                st.dataframe(pd.DataFrame(rows), use_container_width=True)
+            else:
+                st.info("No KYC fields found in this document to validate.")
+
+            # Generate the report
+            with st.spinner("Building RBI compliance PDF..."):
+                pdf_bytes = compliance.build_compliance_report(
+                    f_report, src_path, kyc_results)
+            st.success(f"Generated compliance audit ({len(pdf_bytes)//1024} KB)")
+            st.download_button("Download RBI Compliance Report (PDF)", pdf_bytes,
+                                file_name=f"compliance_{src_path.stem}.pdf",
+                                mime="application/pdf")
+
+
+# -------------------------------------------------------------
+# Footer
+# -------------------------------------------------------------
+st.divider()
+st.caption("BankShield prototype - rule-based + trainable RF model - "
+           "100% open source, runs locally.")

audit_report.py

@@ -0,0 +1,262 @@
+"""
+audit_report.py - generate a bank-letterhead-styled PDF audit report.
+
+Used by app.py to produce the 'Download audit PDF' button.
+Returns bytes so it can be streamed directly to the user without disk writes.
+"""
+
+import io
+from datetime import datetime
+from pathlib import Path
+
+from reportlab.lib.pagesizes import A4
+from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
+from reportlab.lib.units import cm
+from reportlab.lib import colors
+from reportlab.platypus import (SimpleDocTemplate, Paragraph, Spacer, Table,
+                                 TableStyle, Image as RLImage, PageBreak)
+from reportlab.lib.enums import TA_CENTER, TA_LEFT
+import cv2
+import numpy as np
+from PIL import Image as PILImage
+
+import forensics
+
+
+BAND_COLOURS = {
+    "LOW":      colors.HexColor("#16a34a"),
+    "MEDIUM":   colors.HexColor("#ca8a04"),
+    "HIGH":     colors.HexColor("#ea580c"),
+    "CRITICAL": colors.HexColor("#dc2626"),
+}
+
+
+def _styles():
+    s = getSampleStyleSheet()
+    s.add(ParagraphStyle(name="LetterheadBank", parent=s["Title"],
+                          fontSize=22, textColor=colors.HexColor("#1e3a8a"),
+                          spaceAfter=4))
+    s.add(ParagraphStyle(name="LetterheadSub", parent=s["Normal"],
+                          fontSize=10, textColor=colors.grey,
+                          spaceAfter=12, alignment=TA_CENTER))
+    s.add(ParagraphStyle(name="SectionH", parent=s["Heading2"],
+                          fontSize=13, textColor=colors.HexColor("#1e3a8a"),
+                          spaceAfter=6, spaceBefore=12))
+    s.add(ParagraphStyle(name="Evidence", parent=s["Normal"],
+                          fontSize=10, leftIndent=12, bulletIndent=2,
+                          spaceAfter=2))
+    s.add(ParagraphStyle(name="Mono", parent=s["Normal"],
+                          fontName="Courier", fontSize=8,
+                          textColor=colors.dimgray))
+    return s
+
+
+def _pil_to_flowable(pil_img, max_width=15 * cm, max_height=8 * cm):
+    """Convert a PIL image to a sized RL Image flowable."""
+    buf = io.BytesIO()
+    pil_img.convert("RGB").save(buf, "PNG")
+    buf.seek(0)
+    img = RLImage(buf)
+    # scale preserving aspect ratio
+    iw, ih = pil_img.size
+    ratio = min(max_width / iw, max_height / ih)
+    img.drawWidth  = iw * ratio
+    img.drawHeight = ih * ratio
+    return img
+
+
+def _make_heatmap_image(heat, cmap="hot"):
+    import matplotlib.pyplot as plt
+    fig, ax = plt.subplots(figsize=(5, 3))
+    ax.imshow(heat, cmap=cmap)
+    ax.axis("off")
+    buf = io.BytesIO()
+    fig.savefig(buf, format="png", dpi=110, bbox_inches="tight")
+    plt.close(fig)
+    buf.seek(0)
+    return PILImage.open(buf)
+
+
+def build_pdf_report(report, source_path):
+    """
+    report:        the dict returned by forensics.analyse_document(...)
+    source_path:   Path to the original document being analysed
+    Returns:       bytes of the rendered PDF
+    """
+    source_path = Path(source_path)
+    s = _styles()
+    buf = io.BytesIO()
+    doc = SimpleDocTemplate(buf, pagesize=A4,
+                            leftMargin=2 * cm, rightMargin=2 * cm,
+                            topMargin=1.5 * cm, bottomMargin=1.5 * cm)
+    story = []
+
+    # ---- Letterhead ----
+    story.append(Paragraph("DOCSENTRY - DOCUMENT FORENSICS REPORT",
+                           s["LetterheadBank"]))
+    story.append(Paragraph("Confidential - For Underwriting Use Only",
+                           s["LetterheadSub"]))
+
+    # ---- Document metadata table ----
+    meta_data = [
+        ["Field",          "Value"],
+        ["Document",       source_path.name],
+        ["Type",           report.get("type", "-")],
+        ["Analysed at",    report.get("analysed_at", "-")[:19].replace("T", " ")],
+        ["SHA-256",        report.get("sha256", "-")[:32] + "..."],
+    ]
+    t = Table(meta_data, colWidths=[4 * cm, 13 * cm])
+    t.setStyle(TableStyle([
+        ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#1e3a8a")),
+        ("TEXTCOLOR",  (0, 0), (-1, 0), colors.white),
+        ("GRID",       (0, 0), (-1, -1), 0.4, colors.grey),
+        ("FONTNAME",   (0, 0), (-1, 0), "Helvetica-Bold"),
+        ("FONTSIZE",   (0, 0), (-1, -1), 9),
+        ("ROWBACKGROUNDS", (0, 1), (-1, -1),
+            [colors.HexColor("#f8fafc"), colors.white]),
+    ]))
+    story.append(t)
+    story.append(Spacer(1, 0.6 * cm))
+
+    # ---- Risk verdict ----
+    band_str = report.get("risk_band", "UNKNOWN")
+    band_colour = BAND_COLOURS.get(band_str, colors.grey)
+    verdict_table = Table([
+        [Paragraph(f"<para alignment='center'><font size=22 color='white'>"
+                    f"<b>{band_str}</b></font></para>", s["Normal"]),
+         Paragraph(f"<b>Risk score:</b> {report.get('risk_score', '-')}<br/>"
+                    f"<b>Action:</b> {report.get('recommended_action', '-')}",
+                    s["Normal"])],
+    ], colWidths=[5 * cm, 12 * cm])
+    verdict_table.setStyle(TableStyle([
+        ("BACKGROUND", (0, 0), (0, 0), band_colour),
+        ("BACKGROUND", (1, 0), (1, 0), colors.HexColor("#f1f5f9")),
+        ("VALIGN",     (0, 0), (-1, -1), "MIDDLE"),
+        ("BOX",        (0, 0), (-1, -1), 0.4, colors.grey),
+        ("LEFTPADDING",  (0, 0), (-1, -1), 12),
+        ("RIGHTPADDING", (0, 0), (-1, -1), 12),
+        ("TOPPADDING",   (0, 0), (-1, -1), 12),
+        ("BOTTOMPADDING",(0, 0), (-1, -1), 12),
+    ]))
+    story.append(verdict_table)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ---- Sub-score breakdown ----
+    story.append(Paragraph("Sub-score breakdown", s["SectionH"]))
+    sub_rows = [["Detector", "Score (0=clean, 1=suspicious)"]]
+    for k, v in (report.get("sub_scores") or {}).items():
+        bar = "#" * int(v * 30)
+        sub_rows.append([k, f"{v:.2f}  {bar}"])
+    if len(sub_rows) > 1:
+        sub_t = Table(sub_rows, colWidths=[5 * cm, 12 * cm])
+        sub_t.setStyle(TableStyle([
+            ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#1e3a8a")),
+            ("TEXTCOLOR",  (0, 0), (-1, 0), colors.white),
+            ("GRID",       (0, 0), (-1, -1), 0.4, colors.grey),
+            ("FONTNAME",   (1, 1), (1, -1), "Courier"),
+            ("FONTSIZE",   (0, 0), (-1, -1), 9),
+        ]))
+        story.append(sub_t)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ---- Evidence ----
+    story.append(Paragraph("Forensic evidence", s["SectionH"]))
+    for ev in report.get("evidence", []):
+        story.append(Paragraph(f"&bull; {ev}", s["Evidence"]))
+    story.append(Spacer(1, 0.3 * cm))
+
+    # ---- Image-specific: heatmaps ----
+    if report.get("type") == "image":
+        try:
+            story.append(PageBreak())
+            story.append(Paragraph("Forensic visualizations", s["SectionH"]))
+
+            # ELA
+            ela_img, ela_s = forensics.error_level_analysis(source_path)
+            story.append(Paragraph(f"<b>Error Level Analysis</b> (score: {ela_s:.2f})",
+                                    s["Normal"]))
+            story.append(_pil_to_flowable(ela_img))
+            story.append(Spacer(1, 0.3 * cm))
+
+            # Copy-move
+            viz, n_cm, _ = forensics.copy_move_detect(source_path)
+            if viz is not None:
+                story.append(Paragraph(f"<b>Copy-move matches:</b> {n_cm}",
+                                        s["Normal"]))
+                viz_rgb = cv2.cvtColor(viz, cv2.COLOR_BGR2RGB)
+                story.append(_pil_to_flowable(PILImage.fromarray(viz_rgb)))
+                story.append(Spacer(1, 0.3 * cm))
+
+            # Noise heatmap
+            heat, ratio = forensics.noise_inconsistency(source_path)
+            story.append(Paragraph(f"<b>Noise outlier ratio:</b> {ratio:.2%}",
+                                    s["Normal"]))
+            story.append(_pil_to_flowable(_make_heatmap_image(heat)))
+        except Exception as e:
+            story.append(Paragraph(f"Could not render heatmaps: {e}", s["Normal"]))
+
+    # ---- PDF-specific audit details ----
+    if report.get("type") == "pdf":
+        audit = report.get("pdf_audit", {})
+        fonts = report.get("font_audit", {})
+        story.append(Paragraph("PDF structural audit", s["SectionH"]))
+        meta = audit.get("metadata", {}) or {}
+        pdf_rows = [
+            ["Pages",       str(audit.get("pages", "-"))],
+            ["EOF markers", str(audit.get("eof_markers", "-"))],
+            ["Producer",    str(meta.get("producer", "-"))],
+            ["Creator",     str(meta.get("creator", "-"))],
+            ["Fonts used",  ", ".join(fonts.get("fonts", []) or ["-"])],
+        ]
+        pdf_t = Table(pdf_rows, colWidths=[5 * cm, 12 * cm])
+        pdf_t.setStyle(TableStyle([
+            ("GRID",     (0, 0), (-1, -1), 0.4, colors.grey),
+            ("FONTSIZE", (0, 0), (-1, -1), 9),
+            ("BACKGROUND", (0, 0), (0, -1), colors.HexColor("#f1f5f9")),
+        ]))
+        story.append(pdf_t)
+        story.append(Spacer(1, 0.3 * cm))
+        for f in audit.get("flags", []) + fonts.get("flags", []):
+            if f not in ("clean", "ok"):
+                story.append(Paragraph(f"&bull; {f}", s["Evidence"]))
+
+    # ---- ML model verdict (if present) ----
+    if "ml_prediction" in report:
+        ml = report["ml_prediction"]
+        story.append(Paragraph("Trained ML model verdict", s["SectionH"]))
+        ml_t = Table([
+            ["Tamper probability", f"{ml['tamper_probability']:.1%}"],
+            ["Verdict",            ml["verdict"]],
+        ], colWidths=[5 * cm, 12 * cm])
+        ml_t.setStyle(TableStyle([
+            ("GRID", (0, 0), (-1, -1), 0.4, colors.grey),
+            ("FONTSIZE", (0, 0), (-1, -1), 9),
+            ("BACKGROUND", (0, 0), (0, -1), colors.HexColor("#f1f5f9")),
+        ]))
+        story.append(ml_t)
+
+    # ---- Footer ----
+    story.append(Spacer(1, 0.6 * cm))
+    story.append(Paragraph(
+        "<i>This report was generated automatically by DocSentry. "
+        "Findings are based on forensic-signal heuristics and an explainable "
+        "Random Forest classifier. Manual verification is required for any "
+        "file in HIGH or CRITICAL bands.</i>", s["Mono"]))
+
+    doc.build(story)
+    buf.seek(0)
+    return buf.read()
+
+
+if __name__ == "__main__":
+    # CLI smoke test
+    import sys
+    if len(sys.argv) < 2:
+        print("Usage: python audit_report.py <file>")
+        sys.exit(1)
+    src = Path(sys.argv[1])
+    report = forensics.analyse_document(src)
+    pdf_bytes = build_pdf_report(report, src)
+    out = src.with_suffix(".audit.pdf")
+    out.write_bytes(pdf_bytes)
+    print(f"Wrote {out}  ({len(pdf_bytes)} bytes)")

compliance.py

@@ -0,0 +1,408 @@
+"""
+compliance.py - Compliance & Audit Pack for DocSentry
+
+Three capabilities:
+  1. KYC validators - IFSC, PAN, Aadhaar format + checksum verification
+  2. PII redaction - mask Aadhaar/PAN/account/IFSC in extracted text + new PDF
+  3. Compliance report - RBI Master Direction style audit PDF
+
+Imported by app.py Tab 4. None of this needs a network call; the validators
+use RBI-published format rules.
+"""
+
+import io
+import re
+from pathlib import Path
+from datetime import datetime
+
+import fitz                                        # PyMuPDF for PDF rendering
+from reportlab.lib.pagesizes import A4
+from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
+from reportlab.lib.units import cm
+from reportlab.lib import colors
+from reportlab.platypus import (SimpleDocTemplate, Paragraph, Spacer, Table,
+                                 TableStyle)
+
+# ============================================================
+# 1. KYC validators
+# ============================================================
+
+# RBI publishes the IFSC structure: 4-char bank code + '0' + 6-char branch code.
+# Known major bank prefixes (subset of RBI's master list - ~150 banks).
+RBI_BANK_CODES = {
+    "SBIN": "State Bank of India",        "HDFC": "HDFC Bank",
+    "ICIC": "ICICI Bank",                  "UTIB": "Axis Bank",
+    "PUNB": "Punjab National Bank",        "BARB": "Bank of Baroda",
+    "CNRB": "Canara Bank",                 "UBIN": "Union Bank of India",
+    "IOBA": "Indian Overseas Bank",        "IBKL": "IDBI Bank",
+    "KKBK": "Kotak Mahindra Bank",         "YESB": "Yes Bank",
+    "RATN": "RBL Bank",                    "INDB": "IndusInd Bank",
+    "IDIB": "Indian Bank",                 "MAHB": "Bank of Maharashtra",
+    "UCBA": "UCO Bank",                    "BKID": "Bank of India",
+    "ANDB": "Andhra Bank",                 "ALLA": "Allahabad Bank",
+    "CBIN": "Central Bank of India",       "ORBC": "Oriental Bank of Commerce",
+    "SYNB": "Syndicate Bank",              "VIJB": "Vijaya Bank",
+    "PSIB": "Punjab & Sind Bank",          "FDRL": "Federal Bank",
+    "KARB": "Karnataka Bank",              "SIBL": "South Indian Bank",
+    "TMBL": "Tamilnad Mercantile Bank",    "CIUB": "City Union Bank",
+    "BDBL": "Bandhan Bank",                "AUBL": "AU Small Finance Bank",
+    "IDFB": "IDFC First Bank",             "ESFB": "Equitas Small Finance Bank",
+    "DCBL": "DCB Bank",                    "PROG": "Progressive National Bank",
+}
+
+IFSC_RE = re.compile(r"^[A-Z]{4}0[A-Z0-9]{6}$")
+
+
+def validate_ifsc(code):
+    """Returns dict: ok, bank_name, flags."""
+    code = (code or "").upper().strip()
+    flags = []
+    if not code:
+        return {"ok": False, "code": "", "flags": ["empty"]}
+    if not IFSC_RE.match(code):
+        flags.append("invalid format (need AAAA0NNNNNN)")
+        return {"ok": False, "code": code, "flags": flags}
+    bank = code[:4]
+    if bank not in RBI_BANK_CODES:
+        flags.append(f"bank code '{bank}' not in known RBI master list")
+        bank_name = "unknown"
+    else:
+        bank_name = RBI_BANK_CODES[bank]
+    return {"ok": not flags, "code": code, "bank_code": bank,
+            "bank_name": bank_name, "branch_code": code[5:], "flags": flags or ["valid"]}
+
+
+# --- PAN: AAAAA9999A; 4th char is entity type, 5th is surname initial ---
+PAN_RE = re.compile(r"^[A-Z]{5}[0-9]{4}[A-Z]$")
+PAN_ENTITY_TYPES = {
+    "P": "Individual",  "F": "Firm",        "C": "Company",
+    "A": "AOP",         "T": "Trust",       "B": "BOI",
+    "L": "LLP",         "J": "Artificial Juridical Person",
+    "G": "Government",  "H": "HUF",
+}
+
+
+def validate_pan(code):
+    code = (code or "").upper().strip()
+    flags = []
+    if not code:
+        return {"ok": False, "code": "", "flags": ["empty"]}
+    if not PAN_RE.match(code):
+        flags.append("invalid format (need AAAAA9999A)")
+        return {"ok": False, "code": code, "flags": flags}
+    entity = PAN_ENTITY_TYPES.get(code[3], "unknown entity type")
+    if entity == "unknown entity type":
+        flags.append(f"unrecognised 4th character '{code[3]}'")
+    return {"ok": not flags, "code": code, "entity_type": entity,
+            "flags": flags or ["valid"]}
+
+
+# --- Aadhaar: 12 digits, last digit is Verhoeff check ---
+# Verhoeff algorithm tables (UIDAI standard)
+_VERHOEFF_D = [
+    [0,1,2,3,4,5,6,7,8,9], [1,2,3,4,0,6,7,8,9,5],
+    [2,3,4,0,1,7,8,9,5,6], [3,4,0,1,2,8,9,5,6,7],
+    [4,0,1,2,3,9,5,6,7,8], [5,9,8,7,6,0,4,3,2,1],
+    [6,5,9,8,7,1,0,4,3,2], [7,6,5,9,8,2,1,0,4,3],
+    [8,7,6,5,9,3,2,1,0,4], [9,8,7,6,5,4,3,2,1,0],
+]
+_VERHOEFF_P = [
+    [0,1,2,3,4,5,6,7,8,9], [1,5,7,6,2,8,3,0,9,4],
+    [5,8,0,3,7,9,6,1,4,2], [8,9,1,6,0,4,3,5,2,7],
+    [9,4,5,3,1,2,6,8,7,0], [4,2,8,6,5,7,3,9,0,1],
+    [2,7,9,3,8,0,6,4,1,5], [7,0,4,6,9,1,3,2,5,8],
+]
+
+
+def _verhoeff_check(digits):
+    c = 0
+    for i, d in enumerate(reversed(digits)):
+        c = _VERHOEFF_D[c][_VERHOEFF_P[i % 8][d]]
+    return c == 0
+
+
+AADHAAR_RE = re.compile(r"^\d{12}$")
+
+
+def validate_aadhaar(num):
+    num = re.sub(r"\s+", "", str(num or ""))
+    flags = []
+    if not num:
+        return {"ok": False, "code": "", "flags": ["empty"]}
+    if not AADHAAR_RE.match(num):
+        flags.append("invalid format (need 12 digits)")
+        return {"ok": False, "code": num, "flags": flags}
+    if num.startswith(("0", "1")):
+        flags.append("Aadhaar numbers cannot start with 0 or 1")
+    if not _verhoeff_check([int(d) for d in num]):
+        flags.append("Verhoeff checksum failed")
+    return {"ok": not flags, "code": num,
+            "masked": "XXXX-XXXX-" + num[-4:],
+            "flags": flags or ["valid (Verhoeff checksum passed)"]}
+
+
+# ============================================================
+# 2. PII redaction
+# ============================================================
+
+# Reuse the field regexes from forensics
+ACC_RE   = re.compile(r"\b\d{9,18}\b")
+IFSC_ALL = re.compile(r"\b[A-Z]{4}0[A-Z0-9]{6}\b")
+PAN_ALL  = re.compile(r"\b[A-Z]{5}\d{4}[A-Z]\b")
+AAD_ALL  = re.compile(r"\b\d{4}\s?\d{4}\s?\d{4}\b")
+
+
+def redact_text(text):
+    """Mask PII fields in arbitrary text. Returns redacted text + list of found PII."""
+    found = {"ifsc": [], "pan": [], "aadhaar": [], "account": []}
+
+    def _mask(s, last_n=4):
+        return "X" * (len(s) - last_n) + s[-last_n:]
+
+    for m in IFSC_ALL.findall(text):
+        found["ifsc"].append(m)
+        text = text.replace(m, _mask(m))
+    for m in PAN_ALL.findall(text):
+        found["pan"].append(m)
+        text = text.replace(m, _mask(m))
+    for m in AAD_ALL.findall(text):
+        found["aadhaar"].append(m)
+        text = text.replace(m, "XXXX-XXXX-" + m[-4:])
+    for m in ACC_RE.findall(text):
+        if len(m) >= 10 and m not in found["aadhaar"]:
+            found["account"].append(m)
+            text = text.replace(m, _mask(m))
+    return text, found
+
+
+def redact_pdf(input_path, output_path):
+    """Create a redacted copy of a PDF with PII boxes filled in black."""
+    src = fitz.open(input_path)
+    out = fitz.open()
+    found_all = {"ifsc": [], "pan": [], "aadhaar": [], "account": []}
+
+    for page_num, page in enumerate(src):
+        text_dict = page.get_text("dict")
+        new_page = out.new_page(width=page.rect.width, height=page.rect.height)
+        new_page.show_pdf_page(new_page.rect, src, page_num)
+
+        # Search each PII pattern on the page and overlay black rectangles
+        for pattern, key in [(IFSC_ALL, "ifsc"), (PAN_ALL, "pan"),
+                              (AAD_ALL, "aadhaar"), (ACC_RE, "account")]:
+            text = page.get_text()
+            for m in set(pattern.findall(text)):
+                if key == "account" and (len(m) < 10 or m in found_all["aadhaar"]):
+                    continue
+                found_all[key].append(m)
+                # Find bounding boxes for this text on the page
+                rects = page.search_for(m)
+                for r in rects:
+                    new_page.draw_rect(r, color=(0, 0, 0), fill=(0, 0, 0),
+                                        fill_opacity=1, overlay=True)
+
+    out.save(output_path)
+    out.close()
+    src.close()
+    return found_all
+
+
+def extract_pii_fields(path):
+    """Convenience: pull all PII fields from a document (PDF or image)."""
+    if str(path).lower().endswith(".pdf"):
+        with fitz.open(path) as d:
+            text = "\n".join(p.get_text() for p in d)
+    else:
+        try:
+            import pytesseract
+            from PIL import Image
+            text = pytesseract.image_to_string(Image.open(path))
+        except Exception:
+            text = ""
+    fields = {
+        "ifsc":     IFSC_ALL.findall(text),
+        "pan":      PAN_ALL.findall(text),
+        "aadhaar":  AAD_ALL.findall(text),
+        "accounts": [m for m in ACC_RE.findall(text) if len(m) >= 10],
+    }
+    fields["aadhaar"] = list(set(fields["aadhaar"]))
+    return fields, text
+
+
+# ============================================================
+# 3. RBI-style compliance report PDF
+# ============================================================
+
+def build_compliance_report(forensic_report, source_path, kyc_results=None):
+    """
+    Produces an RBI Master-Direction-style audit PDF with mandatory fields:
+      - Customer identification
+      - KYC verification status (per RBI KYC Master Direction 2016)
+      - Fraud-screening status
+      - Recommended risk weighting
+      - Auditor sign-off block
+    """
+    source_path = Path(source_path)
+    s = getSampleStyleSheet()
+    s.add(ParagraphStyle("RBITitle", parent=s["Title"], fontSize=18,
+                         textColor=colors.HexColor("#0c4a6e")))
+    s.add(ParagraphStyle("RBISub", parent=s["Normal"], fontSize=10,
+                         textColor=colors.dimgray, alignment=1))
+    s.add(ParagraphStyle("RBISection", parent=s["Heading2"], fontSize=12,
+                         textColor=colors.HexColor("#0c4a6e"), spaceAfter=4))
+    s.add(ParagraphStyle("Mono", parent=s["Normal"], fontName="Courier",
+                         fontSize=8, textColor=colors.dimgray))
+
+    buf = io.BytesIO()
+    doc = SimpleDocTemplate(buf, pagesize=A4,
+                            leftMargin=2 * cm, rightMargin=2 * cm,
+                            topMargin=1.5 * cm, bottomMargin=1.5 * cm)
+    story = []
+
+    # ----- Header -----
+    story.append(Paragraph("DOCSENTRY - REGULATORY COMPLIANCE AUDIT", s["RBITitle"]))
+    story.append(Paragraph("Pursuant to RBI Master Direction on KYC (2016, as amended) "
+                            "and RBI Master Circular on Frauds (DBS.CO.CFMC.BC.No.1/23.04.001/2023-24)",
+                            s["RBISub"]))
+    story.append(Spacer(1, 0.5 * cm))
+
+    # ----- Section 1: Document identification -----
+    story.append(Paragraph("1. Document Identification", s["RBISection"]))
+    ident = [
+        ["Field", "Value"],
+        ["Document filename", source_path.name],
+        ["Document type", forensic_report.get("type", "-")],
+        ["SHA-256 (full)", forensic_report.get("sha256", "-")],
+        ["Audit timestamp (UTC)", forensic_report.get("analysed_at", "-")],
+        ["Audit ID", "DS-" + forensic_report.get("sha256", "0" * 16)[:16].upper()],
+    ]
+    t = Table(ident, colWidths=[5 * cm, 12 * cm])
+    t.setStyle(TableStyle([
+        ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#0c4a6e")),
+        ("TEXTCOLOR",  (0, 0), (-1, 0), colors.white),
+        ("GRID",       (0, 0), (-1, -1), 0.4, colors.grey),
+        ("FONTSIZE",   (0, 0), (-1, -1), 8),
+        ("FONTNAME",   (0, 0), (-1, 0), "Helvetica-Bold"),
+    ]))
+    story.append(t)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ----- Section 2: KYC verification status -----
+    story.append(Paragraph("2. KYC Field Verification", s["RBISection"]))
+    if not kyc_results:
+        story.append(Paragraph("<i>No KYC fields supplied for verification.</i>", s["Normal"]))
+    else:
+        kyc_rows = [["Field", "Value", "Status", "Notes"]]
+        for kind, result in kyc_results.items():
+            status = "PASS" if result.get("ok") else "FAIL"
+            value = result.get("code", "-")
+            if kind == "aadhaar" and result.get("masked"):
+                value = result["masked"]
+            notes = "; ".join(result.get("flags", []))
+            kyc_rows.append([kind.upper(), value, status, notes])
+        kt = Table(kyc_rows, colWidths=[2.5 * cm, 4 * cm, 1.5 * cm, 9 * cm])
+        kt.setStyle(TableStyle([
+            ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#0c4a6e")),
+            ("TEXTCOLOR",  (0, 0), (-1, 0), colors.white),
+            ("GRID",       (0, 0), (-1, -1), 0.4, colors.grey),
+            ("FONTSIZE",   (0, 0), (-1, -1), 8),
+        ]))
+        story.append(kt)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ----- Section 3: Fraud-screening status -----
+    story.append(Paragraph("3. Fraud / Tamper Screening", s["RBISection"]))
+    band = forensic_report.get("risk_band", "UNKNOWN")
+    score = forensic_report.get("risk_score", "-")
+    band_colour = {"LOW": colors.HexColor("#16a34a"),
+                   "MEDIUM": colors.HexColor("#ca8a04"),
+                   "HIGH": colors.HexColor("#ea580c"),
+                   "CRITICAL": colors.HexColor("#dc2626")}.get(band, colors.grey)
+    fraud_box = Table([
+        [Paragraph(f"<para alignment='center'><font size=18 color='white'><b>{band}</b></font></para>",
+                    s["Normal"]),
+         Paragraph(f"<b>Composite risk score:</b> {score}<br/>"
+                    f"<b>Detection ensemble:</b> Rule-based + Random Forest"
+                    f"{' + CNN (MobileNetV2)' if 'cnn_prediction' in forensic_report else ''}<br/>"
+                    f"<b>Recommended action:</b> {forensic_report.get('recommended_action', '-')}",
+                    s["Normal"])]
+    ], colWidths=[4 * cm, 13 * cm])
+    fraud_box.setStyle(TableStyle([
+        ("BACKGROUND", (0, 0), (0, 0), band_colour),
+        ("BACKGROUND", (1, 0), (1, 0), colors.HexColor("#f0f9ff")),
+        ("VALIGN",     (0, 0), (-1, -1), "MIDDLE"),
+        ("TOPPADDING", (0, 0), (-1, -1), 10),
+        ("BOTTOMPADDING", (0, 0), (-1, -1), 10),
+        ("LEFTPADDING", (0, 0), (-1, -1), 12),
+        ("RIGHTPADDING", (0, 0), (-1, -1), 12),
+    ]))
+    story.append(fraud_box)
+    story.append(Spacer(1, 0.3 * cm))
+    story.append(Paragraph("Detected anomalies:", s["Normal"]))
+    for ev in forensic_report.get("evidence", []):
+        story.append(Paragraph("&bull; " + ev, s["Normal"]))
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ----- Section 4: Recommended risk weighting -----
+    story.append(Paragraph("4. Recommended Risk Treatment (per RBI guidelines)",
+                            s["RBISection"]))
+    rw_map = {"LOW":      ("Standard onboarding", "Risk weight 75% (retail)"),
+              "MEDIUM":   ("Request additional verification", "Risk weight 100%"),
+              "HIGH":     ("Mandatory manual review by fraud-risk team",
+                            "Risk weight 150% pending verification"),
+              "CRITICAL": ("Decline / escalate to compliance",
+                            "Reject application; report under Suspicious Transaction Report (STR) workflow")}
+    treatment, risk_weight = rw_map.get(band, ("-", "-"))
+    rw_table = Table([
+        ["Recommended treatment", treatment],
+        ["Suggested risk weight", risk_weight],
+        ["Audit retention period", "10 years (per RBI Master Direction on KYC, Para 67)"],
+    ], colWidths=[5 * cm, 12 * cm])
+    rw_table.setStyle(TableStyle([
+        ("GRID",     (0, 0), (-1, -1), 0.4, colors.grey),
+        ("BACKGROUND", (0, 0), (0, -1), colors.HexColor("#f0f9ff")),
+        ("FONTSIZE", (0, 0), (-1, -1), 9),
+    ]))
+    story.append(rw_table)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ----- Section 5: Auditor sign-off -----
+    story.append(Paragraph("5. Auditor Sign-Off", s["RBISection"]))
+    signoff = Table([
+        ["Generated by", "DocSentry Automated Audit Engine v1.0"],
+        ["Reviewer name", "_______________________________"],
+        ["Designation",   "_______________________________"],
+        ["Date",          "_______________________________"],
+        ["Signature",     "_______________________________"],
+    ], colWidths=[5 * cm, 12 * cm])
+    signoff.setStyle(TableStyle([
+        ("GRID",     (0, 0), (-1, -1), 0.4, colors.grey),
+        ("FONTSIZE", (0, 0), (-1, -1), 9),
+        ("BACKGROUND", (0, 0), (0, -1), colors.HexColor("#f0f9ff")),
+    ]))
+    story.append(signoff)
+    story.append(Spacer(1, 0.4 * cm))
+
+    # ----- Disclaimer -----
+    story.append(Paragraph(
+        "<i>This document is an automated compliance report. The recommendations "
+        "herein are advisory and do not substitute statutory verification or "
+        "manual review by an authorised compliance officer. All audit records "
+        "must be retained per RBI Master Direction requirements.</i>", s["Mono"]))
+
+    doc.build(story)
+    buf.seek(0)
+    return buf.read()
+
+
+if __name__ == "__main__":
+    # Quick smoke tests
+    print("IFSC SBIN0123456 :", validate_ifsc("SBIN0123456"))
+    print("IFSC FAKE0000000 :", validate_ifsc("FAKE0000000"))
+    print("PAN ABCDE1234F   :", validate_pan("ABCDE1234F"))
+    print("PAN bad          :", validate_pan("ABCD12345"))
+    print("Aadhaar valid    :", validate_aadhaar("234567890124"))    # not real
+    print("Aadhaar starts 0 :", validate_aadhaar("012345678901"))
+    text = "Name: RAMESH. Account: 78439336112. IFSC: SBIN0001234. PAN: ABCDE1234F"
+    redacted, found = redact_text(text)
+    print("Redacted:", redacted)
+    print("Found:", found)

docsentry_master.ipynb

@@ -0,0 +1,1505 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# DocSentry - Master Notebook\n",
+    "\n",
+    "**Single source of truth.** Everything lives here: detectors, training,\n",
+    "evaluation, cross-doc check, PDF report generator, AND a cell that exports\n",
+    "the supporting `.py` files for the Streamlit demo.\n",
+    "\n",
+    "**Use case:** real-time document anomaly detection for bank underwriting.\n",
+    "Land records, legal documents, financial statements.\n",
+    "\n",
+    "**Pipeline:**\n",
+    "```\n",
+    "  Document -> Image forensics  (ELA, copy-move, noise, EXIF)\n",
+    "           -> PDF structure    (EOF count, fonts, producer)\n",
+    "           -> OCR + text rules (date monotonicity, math, IFSC)\n",
+    "           -> Random Forest    (forensic feature blend)\n",
+    "           -> CNN              (MobileNetV2 on CASIA v2)\n",
+    "           -> Risk band + Insights + Audit JSON + PDF report\n",
+    "```\n",
+    "\n",
+    "**100% open-source, no paid APIs, no LLM calls.** Runs on a laptop CPU.\n",
+    "GPU only required for the optional CNN training section.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 0. Environment auto-detection\n",
+    "\n",
+    "Detects whether you are on Colab or local; auto-installs deps if Colab.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys, os, platform\n",
+    "IS_COLAB = 'google.colab' in sys.modules\n",
+    "IS_WINDOWS = platform.system() == 'Windows'\n",
+    "print('Colab:', IS_COLAB, '  Windows:', IS_WINDOWS)\n",
+    "\n",
+    "# One-shot install (skip if you already pip-installed locally)\n",
+    "if IS_COLAB:\n",
+    "    !apt-get -qq install -y tesseract-ocr\n",
+    "    %pip install --quiet \\\n",
+    "        numpy pandas matplotlib seaborn scikit-image scikit-learn joblib \\\n",
+    "        opencv-python-headless pillow pytesseract pdfplumber pymupdf pikepdf \\\n",
+    "        imagehash exifread python-dateutil kaggle reportlab\n",
+    "print('Setup complete.')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import io, json, re, math, hashlib, shutil, tempfile, warnings\n",
+    "from pathlib import Path\n",
+    "from datetime import datetime\n",
+    "from collections import Counter\n",
+    "\n",
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "import matplotlib.pyplot as plt\n",
+    "from PIL import Image, ImageChops, ImageEnhance, ImageDraw, ImageFont, ImageFilter\n",
+    "import cv2\n",
+    "import fitz                            # PyMuPDF\n",
+    "import pdfplumber\n",
+    "import pikepdf\n",
+    "import pytesseract\n",
+    "\n",
+    "warnings.filterwarnings('ignore')\n",
+    "plt.rcParams['figure.figsize'] = (10, 6)\n",
+    "\n",
+    "# Auto-detect Tesseract on Windows / Mac / Linux\n",
+    "TESSERACT_OK = False\n",
+    "for c in [shutil.which('tesseract'),\n",
+    "          r'C:\\\\Program Files\\\\Tesseract-OCR\\\\tesseract.exe',\n",
+    "          r'C:\\\\Program Files (x86)\\\\Tesseract-OCR\\\\tesseract.exe',\n",
+    "          os.path.expanduser(r'~\\\\AppData\\\\Local\\\\Programs\\\\Tesseract-OCR\\\\tesseract.exe')]:\n",
+    "    if c and os.path.isfile(c):\n",
+    "        pytesseract.pytesseract.tesseract_cmd = c\n",
+    "        TESSERACT_OK = True\n",
+    "        print('Tesseract:', c)\n",
+    "        break\n",
+    "if not TESSERACT_OK:\n",
+    "    print('Tesseract not found. OCR-based checks will be skipped.')\n",
+    "    print('Windows install: https://github.com/UB-Mannheim/tesseract/wiki')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Datasets\n",
+    "\n",
+    "Folder layout the notebook expects:\n",
+    "```\n",
+    "data/\n",
+    "  images/originals/    <-- genuine scans\n",
+    "  images/tampered/     <-- forged scans\n",
+    "  pdfs/originals/\n",
+    "  pdfs/tampered/\n",
+    "  statements/\n",
+    "```\n",
+    "\n",
+    "Three ways to populate `data/`:\n",
+    "1. **Synthetic generator** (next cell) - 130 docs each, no downloads, runs in ~3 min\n",
+    "2. **Kaggle CASIA v2** - the 12k-image industry benchmark (see cell 1.3)\n",
+    "3. **Manual datasets** - MICC-F220, CoMoFoD, ICDAR Find-It, Tobacco-3482 (see DATASETS.md)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "DATA = Path('data')\n",
+    "for sub in ['images/originals', 'images/tampered',\n",
+    "            'pdfs/originals', 'pdfs/tampered', 'statements']:\n",
+    "    (DATA / sub).mkdir(parents=True, exist_ok=True)\n",
+    "print('Folders ready under', DATA.resolve())\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.1 Synthetic banking-document generator\n",
+    "\n",
+    "Produces realistic land records, loan agreements, and bank statements.\n",
+    "Tampering variants: copy-move, text-edit, splice, compression-after-edit.\n",
+    "Resumable - skips existing files.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import random\n",
+    "random.seed(42); np.random.seed(42)\n",
+    "\n",
+    "FIRST = ['RAMESH','SURESH','AMIT','PRIYA','ANITA','VIKAS','POOJA','RAHUL',\n",
+    "         'DEEPAK','SUNITA','ARJUN','MEENA','KIRAN','NEHA','SANJAY','GEETA']\n",
+    "LAST  = ['KUMAR','SHARMA','VERMA','SINGH','GUPTA','PATEL','REDDY','RAO',\n",
+    "         'NAIR','JOSHI','MEHTA','AGGARWAL','BANERJEE','MISHRA']\n",
+    "VILLAGES = ['NARAYANPUR','RAMGARH','BHIWANI','KISHANGARH','SITAPUR','JAGADHRI']\n",
+    "BANKS = ['State Bank of India','HDFC Bank','ICICI Bank','Axis Bank',\n",
+    "         'Punjab National Bank','Bank of Baroda','Canara Bank']\n",
+    "IFSC_PFX = ['SBIN','HDFC','ICIC','UTIB','PUNB','BARB','CNRB']\n",
+    "\n",
+    "def rand_name():   return f'{random.choice(FIRST)} {random.choice(LAST)}'\n",
+    "def rand_date():   return f'{random.randint(1,28):02d}-{random.randint(1,12):02d}-{random.randint(2018,2024)}'\n",
+    "def rand_amount(low=100000, high=10000000): return (random.randint(low,high)//1000)*1000\n",
+    "def rand_account():return ''.join(str(random.randint(0,9)) for _ in range(random.randint(11,14)))\n",
+    "def rand_ifsc():   return f\"{random.choice(IFSC_PFX)}0{''.join(random.choice('0123456789ABCDEF') for _ in range(6))}\"\n",
+    "def fmt_inr(a):\n",
+    "    s = str(a)[::-1]; parts=[s[:3]]; s=s[3:]\n",
+    "    while s: parts.append(s[:2]); s=s[2:]\n",
+    "    return 'Rs ' + ','.join(parts)[::-1]\n",
+    "\n",
+    "def get_fonts():\n",
+    "    for p in ['/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf',\n",
+    "              'DejaVuSans.ttf', 'arial.ttf']:\n",
+    "        try:\n",
+    "            return (ImageFont.truetype(p,22), ImageFont.truetype(p,16),\n",
+    "                    ImageFont.truetype(p,14))\n",
+    "        except OSError: continue\n",
+    "    f = ImageFont.load_default(); return f,f,f\n",
+    "BIG, MID, SMALL = get_fonts()\n",
+    "print('Helpers ready.')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def make_land_record():\n",
+    "    img = Image.new('RGB', (900,600), 'white'); d = ImageDraw.Draw(img)\n",
+    "    d.rectangle([20,20,880,580], outline='black', width=2)\n",
+    "    fields = dict(survey=f'{random.randint(50,300)}/{random.randint(1,9)}',\n",
+    "                  owner=rand_name(), area=f'{random.uniform(0.1,5):.2f} hectares',\n",
+    "                  village=random.choice(VILLAGES), date=rand_date(),\n",
+    "                  amount=fmt_inr(rand_amount()))\n",
+    "    d.text((40,40),'GOVERNMENT OF INDIA - LAND RECORD',font=BIG,fill='black')\n",
+    "    for i,(k,v) in enumerate([('Survey No', fields['survey']),\n",
+    "                              ('Owner', fields['owner']),\n",
+    "                              ('Area', fields['area']),\n",
+    "                              ('Village', fields['village']),\n",
+    "                              ('Date', fields['date']),\n",
+    "                              ('Stamp value', fields['amount'])]):\n",
+    "        d.text((40, 90+i*30), f'{k:11s}: {v}', font=MID, fill='black')\n",
+    "    sx,sy = random.randint(520,600), random.randint(370,410)\n",
+    "    d.rectangle([sx,sy,sx+250,sy+140], outline='black', width=1)\n",
+    "    d.text((sx+30,sy+30),'OFFICIAL SEAL',font=MID,fill='black')\n",
+    "    d.text((sx+30,sy+60),'Tehsildar / Patwari',font=MID,fill='black')\n",
+    "    fields['seal_box']=(sx,sy,sx+250,sy+140)\n",
+    "    fields['amount_pos']=(170,256,380,285)\n",
+    "    return img, fields\n",
+    "\n",
+    "def make_loan_agreement():\n",
+    "    img = Image.new('RGB',(900,700),'white'); d = ImageDraw.Draw(img)\n",
+    "    d.rectangle([20,20,880,680], outline='black', width=2)\n",
+    "    fields = dict(borrower=rand_name(), principal=fmt_inr(rand_amount(500000,8000000)),\n",
+    "                  tenure=f'{random.choice([36,60,84,120,180])} months',\n",
+    "                  rate=f'{random.uniform(6.5,12.5):.2f}% p.a.',\n",
+    "                  date=rand_date(), bank=random.choice(BANKS),\n",
+    "                  account=rand_account(), ifsc=rand_ifsc())\n",
+    "    d.text((40,40),'LOAN AGREEMENT',font=BIG,fill='black')\n",
+    "    for i,(k,v) in enumerate([('Borrower',fields['borrower']),\n",
+    "                              ('Principal',fields['principal']),\n",
+    "                              ('Tenure',fields['tenure']),\n",
+    "                              ('Rate',fields['rate']),\n",
+    "                              ('Date',fields['date']),\n",
+    "                              ('Bank',fields['bank']),\n",
+    "                              ('A/c No',fields['account']),\n",
+    "                              ('IFSC',fields['ifsc'])]):\n",
+    "        d.text((40,110+i*35), f'{k:11s}: {v}', font=MID, fill='black')\n",
+    "    sx,sy=560,520\n",
+    "    d.rectangle([sx,sy,sx+260,sy+120], outline='black', width=1)\n",
+    "    d.text((sx+20,sy+20),'AUTHORISED SIGNATORY',font=SMALL,fill='black')\n",
+    "    fields['sig_box']=(sx,sy,sx+260,sy+120)\n",
+    "    fields['principal_pos']=(170,141,380,170)\n",
+    "    return img, fields\n",
+    "\n",
+    "def make_bank_statement():\n",
+    "    img = Image.new('RGB',(900,800),'white'); d = ImageDraw.Draw(img)\n",
+    "    d.rectangle([20,20,880,780], outline='black', width=2)\n",
+    "    d.text((40,40), random.choice(BANKS).upper(), font=BIG, fill='black')\n",
+    "    d.text((40,80),  f'Account Holder : {rand_name()}', font=MID, fill='black')\n",
+    "    d.text((40,110), f'Account No     : {rand_account()}', font=MID, fill='black')\n",
+    "    d.text((40,140), f'IFSC           : {rand_ifsc()}', font=MID, fill='black')\n",
+    "    d.line([(40,215),(860,215)], fill='black', width=1)\n",
+    "    d.text((50,220),'Date',font=MID,fill='black')\n",
+    "    d.text((180,220),'Narration',font=MID,fill='black')\n",
+    "    d.text((500,220),'Debit',font=MID,fill='black')\n",
+    "    d.text((620,220),'Credit',font=MID,fill='black')\n",
+    "    d.line([(40,250),(860,250)], fill='black', width=1)\n",
+    "    bal=random.randint(20000,200000); y=260\n",
+    "    for _ in range(random.randint(8,14)):\n",
+    "        date=f'{random.randint(1,28):02d}-04-2024'\n",
+    "        narr=random.choice(['UPI Transfer','ATM Withdrawal','Salary Credit','Cheque Deposit','EMI Debit'])\n",
+    "        is_cr=random.random()>0.55; amt=random.randint(1000,50000)\n",
+    "        bal = bal+amt if is_cr else bal-amt\n",
+    "        d.text((50,y),date,font=SMALL,fill='black')\n",
+    "        d.text((180,y),narr,font=SMALL,fill='black')\n",
+    "        d.text((500,y),'' if is_cr else f'{amt:,}',font=SMALL,fill='black')\n",
+    "        d.text((620,y),f'{amt:,}' if is_cr else '',font=SMALL,fill='black')\n",
+    "        y += 28\n",
+    "    return img, {}\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def tamper_copy_move(img, fields, kind):\n",
+    "    arr = np.array(img)\n",
+    "    box = fields.get('seal_box') or fields.get('sig_box') or (600,400,850,540)\n",
+    "    x0,y0,x1,y1 = box\n",
+    "    patch = arr[y0:y1, x0:x1].copy()\n",
+    "    tx = max(40, x0 - 300); ty = y0\n",
+    "    if tx + (x1-x0) < arr.shape[1] and ty + (y1-y0) < arr.shape[0]:\n",
+    "        arr[ty:ty+(y1-y0), tx:tx+(x1-x0)] = patch\n",
+    "    return Image.fromarray(arr)\n",
+    "\n",
+    "def tamper_text_edit(img, fields, kind):\n",
+    "    pos = fields.get('amount_pos') or fields.get('principal_pos') or (600,80,800,110)\n",
+    "    d = ImageDraw.Draw(img)\n",
+    "    d.rectangle(pos, fill='white')\n",
+    "    d.text((pos[0]+5, pos[1]+5), fmt_inr(rand_amount(1000000,50000000)),\n",
+    "            font=MID, fill='black')\n",
+    "    return img\n",
+    "\n",
+    "def tamper_splice(img, fields, kind, donor):\n",
+    "    arr=np.array(img); donor_arr=np.array(donor.resize(img.size))\n",
+    "    h,w=arr.shape[:2]; bh,bw=80,200\n",
+    "    x=random.randint(40,w-bw-40); y=random.randint(300,h-bh-40)\n",
+    "    arr[y:y+bh, x:x+bw] = donor_arr[y:y+bh, x:x+bw]\n",
+    "    return Image.fromarray(arr)\n",
+    "\n",
+    "def tamper_compression(img):\n",
+    "    buf=io.BytesIO(); img.save(buf,'JPEG',quality=random.randint(35,60)); buf.seek(0)\n",
+    "    return Image.open(buf).convert('RGB')\n",
+    "\n",
+    "def add_scan_noise(img):\n",
+    "    arr=np.array(img).astype(np.float32)\n",
+    "    arr=np.clip(arr+np.random.normal(0,3,arr.shape),0,255).astype(np.uint8)\n",
+    "    return Image.fromarray(arr)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "GENERATORS = {'land': make_land_record, 'agreement': make_loan_agreement,\n",
+    "              'statement': make_bank_statement}\n",
+    "COUNTS = {'land': 30, 'agreement': 20, 'statement': 15}    # adjust as needed\n",
+    "\n",
+    "def generate_dataset(counts=COUNTS):\n",
+    "    print('Generating genuine documents...')\n",
+    "    genuine = []\n",
+    "    for kind, n in counts.items():\n",
+    "        for i in range(n):\n",
+    "            p = DATA/'images/originals'/f'{kind}_{i:03d}.png'\n",
+    "            if p.exists():\n",
+    "                try:    img=Image.open(p).convert('RGB')\n",
+    "                except: p.unlink(missing_ok=True); img,fields=GENERATORS[kind]()\n",
+    "                else:   _,fields=GENERATORS[kind]()\n",
+    "            else:\n",
+    "                img,fields = GENERATORS[kind]()\n",
+    "                img = add_scan_noise(img); img.save(p)\n",
+    "            genuine.append((kind, img.copy(), fields))\n",
+    "    print(f'  {sum(counts.values())} originals on disk.')\n",
+    "    print('Generating tampered documents...')\n",
+    "    tampers = ['copy_move','text_edit','splice','compression_after_edit']\n",
+    "    nt = 0\n",
+    "    for kind, n in counts.items():\n",
+    "        for i in range(n):\n",
+    "            if list((DATA/'images/tampered').glob(f'{kind}_{i:03d}_*.png')):\n",
+    "                continue\n",
+    "            img,fields = GENERATORS[kind]()\n",
+    "            t = random.choice(tampers)\n",
+    "            if   t=='copy_move':              out = tamper_copy_move(img, fields, kind)\n",
+    "            elif t=='text_edit':              out = tamper_text_edit(img, fields, kind)\n",
+    "            elif t=='splice':                 out = tamper_splice(img, fields, kind, random.choice(genuine)[1])\n",
+    "            else:                              out = tamper_compression(tamper_text_edit(img, fields, kind))\n",
+    "            out = add_scan_noise(out)\n",
+    "            out.save(DATA/'images/tampered'/f'{kind}_{i:03d}_{t}.png')\n",
+    "            nt += 1\n",
+    "    print(f'  {nt} new tampered images written.')\n",
+    "\n",
+    "# Run generator (idempotent - skip cells you already ran)\n",
+    "generate_dataset()\n",
+    "\n",
+    "# Show one genuine + one tampered\n",
+    "samples = sorted((DATA/'images/originals').glob('land_*.png'))[:1]\n",
+    "tsamples = sorted((DATA/'images/tampered').glob('land_*.png'))[:1]\n",
+    "if samples and tsamples:\n",
+    "    fig, ax = plt.subplots(1,2, figsize=(14,5))\n",
+    "    ax[0].imshow(Image.open(samples[0])); ax[0].set_title('Genuine'); ax[0].axis('off')\n",
+    "    ax[1].imshow(Image.open(tsamples[0])); ax[1].set_title(f'Tampered ({tsamples[0].name.split(\"_\",2)[2]})'); ax[1].axis('off')\n",
+    "    plt.show()\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.2 Demo PDFs (for the PDF forensic detectors)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def make_demo_pdfs(n=15):\n",
+    "    for i in range(n):\n",
+    "        op = DATA/'pdfs/originals'/f'agreement_{i:03d}.pdf'\n",
+    "        tp = DATA/'pdfs/tampered'/f'agreement_{i:03d}_tampered.pdf'\n",
+    "        if op.exists() and tp.exists(): continue\n",
+    "        doc = fitz.open(); page = doc.new_page()\n",
+    "        text = (f'LOAN AGREEMENT\\n\\nBorrower : {rand_name()}\\n'\n",
+    "                f'Principal: {fmt_inr(rand_amount())}\\nTenure   : 60 months\\n'\n",
+    "                f'Rate     : 8.5% p.a.\\nDate     : {rand_date()}')\n",
+    "        page.insert_text((72,72), text, fontsize=14)\n",
+    "        doc.set_metadata({'producer':'PyMuPDF','creator':'PyMuPDF'})\n",
+    "        doc.save(op); doc.close()\n",
+    "        doc = fitz.open(op); page = doc[0]\n",
+    "        page.draw_rect(fitz.Rect(150,102,360,122), color=(1,1,1), fill=(1,1,1))\n",
+    "        page.insert_text((150,118), fmt_inr(rand_amount(10000000,90000000)),\n",
+    "                         fontsize=14, fontname='helv')\n",
+    "        doc.set_metadata({'producer':random.choice(['iLovePDF','Smallpdf','PDFescape','Sejda']),\n",
+    "                         'creator':'PyMuPDF'})\n",
+    "        doc.save(tp, deflate=True); doc.close()\n",
+    "make_demo_pdfs()\n",
+    "print('PDFs ready.')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.3 (Optional) Kaggle CASIA v2 download\n",
+    "\n",
+    "Adds 12,000 real tampered/genuine images on top of your synthetic ones.\n",
+    "Requires `kaggle.json` (https://www.kaggle.com/settings -> Create New API Token).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "USE_CASIA = False    # flip True after placing kaggle.json\n",
+    "\n",
+    "if USE_CASIA:\n",
+    "    if IS_COLAB:\n",
+    "        from google.colab import files\n",
+    "        if not os.path.exists('/root/.kaggle/kaggle.json'):\n",
+    "            up = files.upload()      # browse-and-select\n",
+    "            os.makedirs('/root/.kaggle', exist_ok=True)\n",
+    "            for n in up:\n",
+    "                if n.endswith('.json'):\n",
+    "                    shutil.copy(n, '/root/.kaggle/kaggle.json')\n",
+    "                    os.chmod('/root/.kaggle/kaggle.json', 0o600)\n",
+    "                    break\n",
+    "    !kaggle datasets download -d divg07/casia-20-image-tampering-detection-dataset \\\n",
+    "        -p data/images --unzip --force\n",
+    "    # rename Au/Tp -> originals/tampered\n",
+    "    for src, dst in [('Au','originals'),('Tp','tampered')]:\n",
+    "        for cand in [f'data/images/{src}', f'data/images/CASIA2/{src}']:\n",
+    "            if os.path.isdir(cand) and not os.path.isdir(f'data/images/{dst}'):\n",
+    "                shutil.move(cand, f'data/images/{dst}'); break\n",
+    "    print('CASIA v2 ready.')\n",
+    "else:\n",
+    "    print('USE_CASIA = False - skipping. Synthetic data only.')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Image forensics detectors\n",
+    "\n",
+    "Four classical techniques, none requiring training:\n",
+    "- **Error Level Analysis** - re-save at known JPEG quality; tampered regions diverge\n",
+    "- **Copy-move** - ORB keypoint matching finds duplicated regions\n",
+    "- **Noise inconsistency** - per-block Laplacian variance for splice detection\n",
+    "- **EXIF sanity** - missing metadata, photo-editor fingerprints, time mismatches\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def error_level_analysis(path, quality=90, scale=15):\n",
+    "    orig = Image.open(path).convert('RGB')\n",
+    "    buf = io.BytesIO(); orig.save(buf,'JPEG',quality=quality); buf.seek(0)\n",
+    "    resaved = Image.open(buf)\n",
+    "    diff = ImageChops.difference(orig, resaved)\n",
+    "    max_diff = max([e[1] for e in diff.getextrema()]) or 1\n",
+    "    ela = ImageEnhance.Brightness(diff).enhance(scale * 255 / max_diff)\n",
+    "    return ela, float(np.array(diff).mean())\n",
+    "\n",
+    "def copy_move_detect(path, min_dist=40, max_matches=80):\n",
+    "    img = cv2.imread(str(path))\n",
+    "    if img is None: return None, 0, []\n",
+    "    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
+    "    orb = cv2.ORB_create(nfeatures=2000)\n",
+    "    kp, des = orb.detectAndCompute(gray, None)\n",
+    "    if des is None or len(kp)<10: return img, 0, []\n",
+    "    matches = cv2.BFMatcher(cv2.NORM_HAMMING).knnMatch(des, des, k=10)\n",
+    "    good = []\n",
+    "    for ml in matches:\n",
+    "        for m in ml[1:]:\n",
+    "            p1,p2 = kp[m.queryIdx].pt, kp[m.trainIdx].pt\n",
+    "            d = math.hypot(p1[0]-p2[0], p1[1]-p2[1])\n",
+    "            if d > min_dist and m.distance < 40: good.append((p1,p2,d))\n",
+    "    good = good[:max_matches]\n",
+    "    out = img.copy()\n",
+    "    for p1,p2,_ in good:\n",
+    "        cv2.line(out, tuple(map(int,p1)), tuple(map(int,p2)), (0,0,255), 1)\n",
+    "        cv2.circle(out, tuple(map(int,p1)), 3, (0,255,0), -1)\n",
+    "        cv2.circle(out, tuple(map(int,p2)), 3, (0,255,0), -1)\n",
+    "    return out, len(good), good\n",
+    "\n",
+    "def noise_inconsistency(path, block=32):\n",
+    "    img = cv2.imread(str(path), cv2.IMREAD_GRAYSCALE)\n",
+    "    if img is None: return np.zeros((1,1)), 0.0\n",
+    "    H,W = img.shape; Hc,Wc = (H//block)*block, (W//block)*block\n",
+    "    if Hc==0 or Wc==0: return np.zeros((1,1)), 0.0\n",
+    "    img = img[:Hc,:Wc]\n",
+    "    lap = cv2.Laplacian(img, cv2.CV_64F)\n",
+    "    blocks = lap.reshape(Hc//block,block,Wc//block,block).transpose(0,2,1,3).reshape(-1,block*block)\n",
+    "    var = blocks.var(axis=1)\n",
+    "    z = (var - var.mean()) / (var.std() + 1e-9)\n",
+    "    return np.abs(z).reshape(Hc//block, Wc//block), float((np.abs(z)>2.5).sum()/max(1,len(z)))\n",
+    "\n",
+    "def exif_sanity(path):\n",
+    "    try: exif = Image.open(path).getexif()\n",
+    "    except Exception: return ['cannot read image']\n",
+    "    if not exif: return ['no EXIF metadata (re-saved or stripped)']\n",
+    "    tags = {Image.ExifTags.TAGS.get(k,k):v for k,v in exif.items()}\n",
+    "    flags = []; sw = str(tags.get('Software','')).lower()\n",
+    "    for bad in ['photoshop','gimp','paint','snapseed','picsart']:\n",
+    "        if bad in sw: flags.append('edited with '+bad)\n",
+    "    if 'DateTimeOriginal' in tags and 'DateTime' in tags:\n",
+    "        if tags['DateTimeOriginal'] != tags['DateTime']:\n",
+    "            flags.append('modified-time differs from original-time')\n",
+    "    return flags or ['exif clean']\n",
+    "\n",
+    "print('Image forensics ready.')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.1 Visual smoke test\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "originals = sorted((DATA/'images/originals').glob('land_*.png'))[:1]\n",
+    "tampered  = sorted((DATA/'images/tampered').glob('land_*.png'))[:1]\n",
+    "\n",
+    "for label, path in [('Genuine', originals[0]), ('Tampered', tampered[0])]:\n",
+    "    fig, ax = plt.subplots(1, 3, figsize=(16, 4))\n",
+    "    ax[0].imshow(Image.open(path)); ax[0].set_title(f'{label} - source'); ax[0].axis('off')\n",
+    "    ela, s = error_level_analysis(path)\n",
+    "    ax[1].imshow(ela); ax[1].set_title(f'ELA (score={s:.2f})'); ax[1].axis('off')\n",
+    "    viz, n, _ = copy_move_detect(path)\n",
+    "    ax[2].imshow(cv2.cvtColor(viz, cv2.COLOR_BGR2RGB))\n",
+    "    ax[2].set_title(f'Copy-move ({n} matches)'); ax[2].axis('off')\n",
+    "    plt.show()\n",
+    "    print(f'{label}: EXIF -> {exif_sanity(path)}')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. PDF forensics detectors\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def pdf_structural_audit(path):\n",
+    "    raw = Path(path).read_bytes(); eofs = raw.count(b'%%EOF')\n",
+    "    with fitz.open(path) as d:\n",
+    "        info = d.metadata or {}; n_pages = d.page_count\n",
+    "    flags = []\n",
+    "    if eofs > 1: flags.append(f'{eofs} EOF markers (incremental updates)')\n",
+    "    prod = (info.get('producer') or '').lower()\n",
+    "    crt  = (info.get('creator')  or '').lower()\n",
+    "    if prod and crt and prod != crt:\n",
+    "        flags.append(f'producer/creator differ: {prod} vs {crt}')\n",
+    "    for t in ['ilovepdf','smallpdf','pdfescape','sejda','foxit phantom']:\n",
+    "        if t in prod or t in crt: flags.append('edited via consumer tool: '+t)\n",
+    "    return {'pages':n_pages, 'eof_markers':eofs, 'metadata':info,\n",
+    "            'flags': flags or ['clean']}\n",
+    "\n",
+    "def pdf_font_audit(path):\n",
+    "    fonts = []\n",
+    "    with fitz.open(path) as d:\n",
+    "        for page in d: fonts.append({f[3] for f in page.get_fonts()})\n",
+    "    allf = set().union(*fonts) if fonts else set()\n",
+    "    return {'fonts': sorted(allf),\n",
+    "            'flags': ['unusually high font count: '+str(len(allf))] if len(allf)>4 else ['ok']}\n",
+    "\n",
+    "import pprint\n",
+    "for label, p in [('Genuine', DATA/'pdfs/originals/agreement_000.pdf'),\n",
+    "                 ('Tampered', DATA/'pdfs/tampered/agreement_000_tampered.pdf')]:\n",
+    "    if p.exists():\n",
+    "        print(f'\\n=== {label} ==='); pprint.pp(pdf_structural_audit(p))\n",
+    "        print('Fonts:', pdf_font_audit(p))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. OCR + text-level rules\n",
+    "\n",
+    "Date monotonicity, amount sanity, IFSC format, account-without-IFSC,\n",
+    "round-number anomalies. Skipped gracefully if Tesseract isn't installed.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "AMT_RE  = re.compile(r'(?<![A-Za-z])[-]?\\d{1,3}(?:,\\d{2,3})*(?:\\.\\d{1,2})?')\n",
+    "DATE_RE = re.compile(r'(\\d{1,2}[-/]\\d{1,2}[-/]\\d{2,4})')\n",
+    "IFSC_RE = re.compile(r'\\b[A-Z]{4}0[A-Z0-9]{6}\\b')\n",
+    "ACC_RE  = re.compile(r'\\b\\d{9,18}\\b')\n",
+    "\n",
+    "def ocr_text(path):\n",
+    "    if not TESSERACT_OK: return ''\n",
+    "    try: return pytesseract.image_to_string(Image.open(path))\n",
+    "    except Exception: return ''\n",
+    "\n",
+    "def parse_amounts(text):\n",
+    "    out=[]\n",
+    "    for m in AMT_RE.findall(text):\n",
+    "        try: out.append(float(m.replace(',', '')))\n",
+    "        except ValueError: pass\n",
+    "    return out\n",
+    "\n",
+    "def text_rule_checks(text):\n",
+    "    if not text:\n",
+    "        return {'n_dates':0,'n_amounts':0,'n_ifsc':0,'n_accounts':0,'flags':['ocr_skipped']}\n",
+    "    flags = []\n",
+    "    dates = DATE_RE.findall(text); ifsc = IFSC_RE.findall(text)\n",
+    "    accs  = ACC_RE.findall(text);  amts = parse_amounts(text)\n",
+    "    if dates:\n",
+    "        try:\n",
+    "            from dateutil import parser as dp\n",
+    "            ds = [dp.parse(d, dayfirst=True) for d in dates]\n",
+    "            if any(ds[i] > ds[i+1] for i in range(len(ds)-1)):\n",
+    "                flags.append('dates not monotonic')\n",
+    "        except Exception: flags.append('unparseable dates')\n",
+    "    if amts:\n",
+    "        big = [a for a in amts if a >= 100000 and a % 100000 == 0]\n",
+    "        if len(big) > 3: flags.append(f'{len(big)} suspiciously round large amounts')\n",
+    "    if accs and not ifsc: flags.append('account number present but no IFSC')\n",
+    "    return {'n_dates':len(dates), 'n_amounts':len(amts),\n",
+    "            'n_ifsc':len(ifsc), 'n_accounts':len(accs),\n",
+    "            'flags': flags or ['ok']}\n",
+    "\n",
+    "print('OCR + text rules ready.')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Anomaly scoring + risk band + insights\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "WEIGHTS = {'ela':0.20, 'copy_move':0.25, 'noise':0.15, 'exif':0.10,\n",
+    "           'pdf_struct':0.15, 'text_rules':0.10, 'math':0.05}\n",
+    "\n",
+    "def band(s):\n",
+    "    return ('LOW' if s<0.25 else 'MEDIUM' if s<0.50 else\n",
+    "            'HIGH' if s<0.75 else 'CRITICAL')\n",
+    "\n",
+    "INSIGHT_RULES = [\n",
+    "    ('copy_move',0.4,'Possible copy-paste forgery: repeated visual region. Inspect seal/signature area.'),\n",
+    "    ('ela',      0.4,'Compression artefacts inconsistent with a single-source scan. Likely re-saved after edits.'),\n",
+    "    ('noise',    0.4,'Localised noise inconsistency - common in image splicing.'),\n",
+    "    ('exif',     0.4,'Image metadata indicates edits in a photo-editor or stripped EXIF.'),\n",
+    "    ('pdf_struct',0.4,'PDF structural anomalies (incremental edits or consumer-tool fingerprint).'),\n",
+    "]\n",
+    "ACTIONS = {'LOW':'Proceed with standard underwriting.',\n",
+    "           'MEDIUM':'Request additional verification documents.',\n",
+    "           'HIGH':'Escalate to fraud-risk team; manual review mandatory.',\n",
+    "           'CRITICAL':'Block file; trigger investigation workflow.'}\n",
+    "\n",
+    "def score_image(path):\n",
+    "    _, ela_s = error_level_analysis(path)\n",
+    "    _, ncm,_ = copy_move_detect(path)\n",
+    "    _, nr   = noise_inconsistency(path)\n",
+    "    ef = exif_sanity(path)\n",
+    "    sub = {'ela':min(ela_s/25.0,1.0),\n",
+    "           'copy_move':min(ncm/50.0,1.0),\n",
+    "           'noise':min(nr*4,1.0),\n",
+    "           'exif':0.0 if ef==['exif clean'] else 0.6}\n",
+    "    return sum(WEIGHTS[k]*v for k,v in sub.items()), sub, ef\n",
+    "\n",
+    "def generate_insights(score, sub, extra=None):\n",
+    "    bullets = [m for k,t,m in INSIGHT_RULES if sub.get(k,0) >= t]\n",
+    "    if extra:\n",
+    "        bullets += ['Flag: '+str(f) for f in extra if f not in ('exif clean','ok','clean')]\n",
+    "    if not bullets: bullets = ['No anomaly indicators above threshold.']\n",
+    "    return {'risk_score':round(score,3), 'risk_band':band(score),\n",
+    "            'recommended_action':ACTIONS[band(score)], 'evidence':bullets}\n",
+    "\n",
+    "for label, path in [('Genuine', sorted((DATA/'images/originals').glob('land_*.png'))[0]),\n",
+    "                    ('Tampered', sorted((DATA/'images/tampered').glob('land_*.png'))[0])]:\n",
+    "    s, sub, ef = score_image(path)\n",
+    "    print(f'{label:9s} score={s:.3f} band={band(s)} sub={sub} exif={ef}')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Random Forest training (auto-trains on data/images/)\n",
+    "\n",
+    "Extracts forensic features per image; trains a Random Forest;\n",
+    "saves to `models/forgery_rf.joblib`. Loaded automatically by the pipeline.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage.feature import graycomatrix, graycoprops\n",
+    "from sklearn.ensemble import RandomForestClassifier\n",
+    "from sklearn.model_selection import train_test_split\n",
+    "from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score\n",
+    "import joblib\n",
+    "\n",
+    "def extract_features(path):\n",
+    "    f = {}\n",
+    "    _, f['ela_mean'] = error_level_analysis(path)\n",
+    "    _, f['copy_move_matches'], _ = copy_move_detect(path)\n",
+    "    _, f['noise_outlier_ratio']  = noise_inconsistency(path)\n",
+    "    f['exif_clean'] = int(exif_sanity(path) == ['exif clean'])\n",
+    "    g = cv2.imread(str(path), cv2.IMREAD_GRAYSCALE)\n",
+    "    gs = cv2.resize(g, (256,256))\n",
+    "    glcm = graycomatrix(gs, [1], [0], 256, symmetric=True, normed=True)\n",
+    "    f['glcm_contrast']    = float(graycoprops(glcm,'contrast')[0,0])\n",
+    "    f['glcm_homogeneity'] = float(graycoprops(glcm,'homogeneity')[0,0])\n",
+    "    f['glcm_energy']      = float(graycoprops(glcm,'energy')[0,0])\n",
+    "    f['glcm_correlation'] = float(graycoprops(glcm,'correlation')[0,0])\n",
+    "    c = cv2.imread(str(path))\n",
+    "    if c is not None:\n",
+    "        for i,ch in enumerate(['b','g','r']):\n",
+    "            h = cv2.calcHist([c],[i],None,[32],[0,256]).flatten()\n",
+    "            h = h/(h.sum()+1e-9)\n",
+    "            f['hist_'+ch+'_entropy'] = float(-(h*np.log2(h+1e-9)).sum())\n",
+    "    return f\n",
+    "\n",
+    "def build_training_table(root=DATA/'images'):\n",
+    "    rows = []\n",
+    "    for label, sub in [(0,'originals'), (1,'tampered')]:\n",
+    "        for p in (root/sub).rglob('*'):\n",
+    "            if p.suffix.lower() in {'.png','.jpg','.jpeg'}:\n",
+    "                try:\n",
+    "                    f = extract_features(p)\n",
+    "                    f['label']=label; rows.append(f)\n",
+    "                except Exception as e:\n",
+    "                    print('skip', p.name, '->', e)\n",
+    "    return pd.DataFrame(rows)\n",
+    "\n",
+    "train_df = build_training_table()\n",
+    "print(f'Training rows: {len(train_df)}  Classes: {train_df[\"label\"].value_counts().to_dict() if len(train_df) else \"none\"}')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MODEL_PATH = Path('models/forgery_rf.joblib')\n",
+    "MODEL_PATH.parent.mkdir(exist_ok=True)\n",
+    "\n",
+    "if len(train_df) < 10:\n",
+    "    print('Not enough real data - skipping training. Run section 1.1 first.')\n",
+    "else:\n",
+    "    FEATURES = [c for c in train_df.columns if c != 'label']\n",
+    "    X = train_df[FEATURES]; y = train_df['label']\n",
+    "    Xtr,Xte,ytr,yte = train_test_split(X,y, test_size=0.25, random_state=42, stratify=y)\n",
+    "    clf = RandomForestClassifier(n_estimators=300, max_depth=10,\n",
+    "                                  class_weight='balanced', random_state=42, n_jobs=-1)\n",
+    "    clf.fit(Xtr, ytr)\n",
+    "    pred = clf.predict(Xte); prob = clf.predict_proba(Xte)[:,1]\n",
+    "    print(classification_report(yte, pred, target_names=['genuine','tampered']))\n",
+    "    print('Confusion:'); print(confusion_matrix(yte, pred))\n",
+    "    try: print('ROC-AUC:', round(roc_auc_score(yte, prob), 3))\n",
+    "    except Exception: pass\n",
+    "    joblib.dump({'model':clf, 'features':FEATURES}, MODEL_PATH)\n",
+    "    # Feature importance plot\n",
+    "    imp = pd.Series(clf.feature_importances_, index=FEATURES).sort_values()\n",
+    "    plt.figure(figsize=(8,5))\n",
+    "    imp.plot.barh(color='steelblue'); plt.title('Forensic feature importance')\n",
+    "    plt.tight_layout(); plt.show()\n",
+    "    print(f'Model saved: {MODEL_PATH.resolve()}')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def predict_with_model(path, model_path=MODEL_PATH):\n",
+    "    if not Path(model_path).exists(): return None\n",
+    "    b = joblib.load(model_path)\n",
+    "    f = extract_features(path)\n",
+    "    p = b['model'].predict_proba(pd.DataFrame([f])[b['features']])[0,1]\n",
+    "    return {'tamper_probability':round(float(p),3),\n",
+    "            'verdict':'TAMPERED' if p>=0.5 else 'GENUINE'}\n",
+    "\n",
+    "if MODEL_PATH.exists():\n",
+    "    print('Genuine :', predict_with_model(sorted((DATA/'images/originals').glob('land_*.png'))[0]))\n",
+    "    print('Tampered:', predict_with_model(sorted((DATA/'images/tampered').glob('land_*.png'))[0]))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. (Optional) CNN training on real CASIA v2\n",
+    "\n",
+    "Flip `TRAIN_CNN = True` once you have ~200+ real images per class.\n",
+    "Trains MobileNetV2 in two phases (head only, then unfreezed top layers).\n",
+    "Saves to `models/forgery_cnn.keras` + `forgery_cnn.meta.json`.\n",
+    "Runs in ~25 min on Colab T4 GPU. Skip if running on a CPU laptop.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "TRAIN_CNN = False\n",
+    "\n",
+    "if TRAIN_CNN:\n",
+    "    if IS_COLAB: %pip install --quiet tensorflow\n",
+    "    import tensorflow as tf\n",
+    "    from tensorflow.keras import layers, Model\n",
+    "    print('TF:', tf.__version__, '  GPU:', tf.config.list_physical_devices('GPU'))\n",
+    "    IMG, BATCH = 224, 16\n",
+    "    train_ds = tf.keras.utils.image_dataset_from_directory(\n",
+    "        'data/images', validation_split=0.2, subset='training', seed=42,\n",
+    "        image_size=(IMG,IMG), batch_size=BATCH, label_mode='binary')\n",
+    "    val_ds = tf.keras.utils.image_dataset_from_directory(\n",
+    "        'data/images', validation_split=0.2, subset='validation', seed=42,\n",
+    "        image_size=(IMG,IMG), batch_size=BATCH, label_mode='binary')\n",
+    "    CLASS_NAMES = train_ds.class_names\n",
+    "    augment = tf.keras.Sequential([\n",
+    "        layers.RandomFlip('horizontal'), layers.RandomRotation(0.04),\n",
+    "        layers.RandomBrightness(0.15), layers.RandomContrast(0.15)])\n",
+    "    train_ds = train_ds.map(lambda x,y: (augment(x,training=True), y)).prefetch(tf.data.AUTOTUNE)\n",
+    "    val_ds = val_ds.cache().prefetch(tf.data.AUTOTUNE)\n",
+    "    base = tf.keras.applications.MobileNetV2(input_shape=(IMG,IMG,3),\n",
+    "                                              include_top=False, weights='imagenet')\n",
+    "    base.trainable = False\n",
+    "    inp = layers.Input(shape=(IMG,IMG,3))\n",
+    "    x = tf.keras.applications.mobilenet_v2.preprocess_input(inp)\n",
+    "    x = base(x, training=False)\n",
+    "    x = layers.GlobalAveragePooling2D()(x)\n",
+    "    x = layers.Dropout(0.35)(x)\n",
+    "    x = layers.Dense(256, activation='relu')(x)\n",
+    "    x = layers.Dropout(0.25)(x)\n",
+    "    out = layers.Dense(1, activation='sigmoid')(x)\n",
+    "    cnn = Model(inp, out)\n",
+    "    cnn.compile(optimizer=tf.keras.optimizers.Adam(1e-3),\n",
+    "                 loss='binary_crossentropy',\n",
+    "                 metrics=['accuracy', tf.keras.metrics.AUC(name='auc')])\n",
+    "    print('Phase 1: head-only training...')\n",
+    "    cnn.fit(train_ds, validation_data=val_ds, epochs=5)\n",
+    "    print('Phase 2: fine-tuning top of backbone...')\n",
+    "    base.trainable = True\n",
+    "    for l in base.layers[:-40]: l.trainable = False\n",
+    "    cnn.compile(optimizer=tf.keras.optimizers.Adam(1e-5),\n",
+    "                 loss='binary_crossentropy',\n",
+    "                 metrics=['accuracy', tf.keras.metrics.AUC(name='auc')])\n",
+    "    cnn.fit(train_ds, validation_data=val_ds, epochs=5)\n",
+    "    loss, acc, auc = cnn.evaluate(val_ds, verbose=0)\n",
+    "    print(f'Val loss {loss:.3f}  acc {acc:.3f}  AUC {auc:.3f}')\n",
+    "    Path('models').mkdir(exist_ok=True)\n",
+    "    cnn.save('models/forgery_cnn.keras')\n",
+    "    json.dump({'class_names':CLASS_NAMES, 'image_size':IMG, 'val_auc':float(auc),\n",
+    "               'val_accuracy':float(acc)},\n",
+    "              open('models/forgery_cnn.meta.json','w'), indent=2)\n",
+    "    print('Saved: models/forgery_cnn.keras')\n",
+    "else:\n",
+    "    print('TRAIN_CNN = False - skipping. Flip True + Colab GPU runtime to train.')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "CNN_MODEL_PATH = Path('models/forgery_cnn.keras')\n",
+    "CNN_META_PATH  = Path('models/forgery_cnn.meta.json')\n",
+    "_CNN = {'model':None, 'meta':None, 'tried':False}\n",
+    "\n",
+    "def _load_cnn():\n",
+    "    if _CNN['tried']: return _CNN['model'], _CNN['meta']\n",
+    "    _CNN['tried'] = True\n",
+    "    if not CNN_MODEL_PATH.exists(): return None, None\n",
+    "    try:\n",
+    "        import tensorflow as tf\n",
+    "        _CNN['model'] = tf.keras.models.load_model(CNN_MODEL_PATH)\n",
+    "        _CNN['meta']  = (json.loads(CNN_META_PATH.read_text())\n",
+    "                        if CNN_META_PATH.exists() else {'image_size':224})\n",
+    "    except Exception as e: print('CNN load failed:', e)\n",
+    "    return _CNN['model'], _CNN['meta']\n",
+    "\n",
+    "def predict_with_cnn(path):\n",
+    "    m, meta = _load_cnn()\n",
+    "    if m is None: return None\n",
+    "    sz = meta.get('image_size', 224)\n",
+    "    arr = np.array(Image.open(path).convert('RGB').resize((sz,sz)))[None].astype(np.float32)\n",
+    "    p = float(m.predict(arr, verbose=0)[0,0])\n",
+    "    return {'tamper_probability':round(p,3),\n",
+    "            'verdict':'TAMPERED' if p>=0.5 else 'GENUINE',\n",
+    "            'model':'MobileNetV2 (CASIA v2 fine-tuned)',\n",
+    "            'val_auc': meta.get('val_auc')}\n",
+    "\n",
+    "print('CNN inference ready (uses model when present).')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 8. End-to-end pipeline\n",
+    "\n",
+    "Single `analyse_document(path)` call that:\n",
+    "- Detects type (image vs PDF)\n",
+    "- Runs all relevant detectors\n",
+    "- Blends RF + CNN predictions if their models exist\n",
+    "- Returns a complete audit dict\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def analyse_document(path):\n",
+    "    path = Path(path); ext = path.suffix.lower()\n",
+    "    r = {'file':str(path),\n",
+    "         'analysed_at':datetime.utcnow().isoformat()+'Z',\n",
+    "         'sha256':hashlib.sha256(path.read_bytes()).hexdigest()}\n",
+    "    if ext in ('.png','.jpg','.jpeg','.tif','.tiff','.bmp'):\n",
+    "        r['type'] = 'image'\n",
+    "        s, sub, ef = score_image(path)\n",
+    "        try:\n",
+    "            tr = text_rule_checks(ocr_text(path))\n",
+    "            sub['text_rules'] = 0.0 if tr['flags']==['ok'] else 0.5\n",
+    "            s = sum(WEIGHTS.get(k,0)*v for k,v in sub.items())\n",
+    "        except Exception as e: tr = {'error':str(e)}\n",
+    "        try:\n",
+    "            ml = predict_with_model(path)\n",
+    "            if ml is not None:\n",
+    "                s = 0.5*s + 0.5*ml['tamper_probability']\n",
+    "                r['ml_prediction'] = ml\n",
+    "        except Exception as e: r['ml_error'] = str(e)\n",
+    "        try:\n",
+    "            cnn = predict_with_cnn(path)\n",
+    "            if cnn is not None:\n",
+    "                w = max(0.4, min(0.7, (cnn.get('val_auc') or 0.85)))\n",
+    "                s = (1-w)*s + w*cnn['tamper_probability']\n",
+    "                r['cnn_prediction'] = cnn\n",
+    "        except Exception as e: r['cnn_error'] = str(e)\n",
+    "        r.update({'sub_scores':sub, 'exif_flags':ef, 'text_rules':tr,\n",
+    "                  **generate_insights(s, sub, ef+tr.get('flags',[]))})\n",
+    "    elif ext == '.pdf':\n",
+    "        r['type'] = 'pdf'\n",
+    "        audit = pdf_structural_audit(path); fonts = pdf_font_audit(path)\n",
+    "        sub = {'pdf_struct':0.8 if audit['flags']!=['clean'] else 0.1,\n",
+    "               'text_rules':0.6 if fonts['flags']!=['ok'] else 0.1}\n",
+    "        s = sum(WEIGHTS.get(k,0)*v for k,v in sub.items())\n",
+    "        r.update({'sub_scores':sub, 'pdf_audit':audit, 'font_audit':fonts,\n",
+    "                  **generate_insights(s, sub, audit['flags']+fonts['flags'])})\n",
+    "    else: r['type']='unsupported'; r['error']='extension '+ext\n",
+    "    return r\n",
+    "\n",
+    "# Demo on 4 files\n",
+    "for p in [sorted((DATA/'images/originals').glob('land_*.png'))[0],\n",
+    "          sorted((DATA/'images/tampered').glob('land_*.png'))[0],\n",
+    "          DATA/'pdfs/originals/agreement_000.pdf',\n",
+    "          DATA/'pdfs/tampered/agreement_000_tampered.pdf']:\n",
+    "    if p.exists():\n",
+    "        r = analyse_document(p)\n",
+    "        print(f'\\n--- {p.name} ---')\n",
+    "        print(f\"  band: {r['risk_band']}  score: {r['risk_score']}  action: {r['recommended_action']}\")\n",
+    "        for e in r['evidence']: print('  *', e)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 9. Cross-document consistency check\n",
+    "\n",
+    "Upload 2+ docs for the same applicant; system extracts identity fields\n",
+    "and flags mismatches in name, DOB, address, account, IFSC.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "NAME_RE = re.compile(r'(?:Name|Owner|Borrower|Holder|Account Holder)\\s*[:\\-]\\s*([A-Z][A-Z\\s.]{2,40})', re.I)\n",
+    "DOB_RE  = re.compile(r'(?:DOB|Date of Birth|Born)\\s*[:\\-]\\s*(\\d{1,2}[-/]\\d{1,2}[-/]\\d{2,4})', re.I)\n",
+    "ADDR_RE = re.compile(r'(?:Address|Village|Residence)\\s*[:\\-]\\s*([A-Z0-9][A-Z0-9\\s,.\\-/]{3,80})', re.I)\n",
+    "\n",
+    "def _norm(s): return re.sub(r'\\s+', ' ', (s or '').strip().upper())\n",
+    "\n",
+    "def extract_identity_fields(path):\n",
+    "    if str(path).lower().endswith('.pdf'):\n",
+    "        with fitz.open(path) as d: text = '\\n'.join(p.get_text() for p in d)\n",
+    "    else: text = ocr_text(path)\n",
+    "    f = {k:None for k in ('name','dob','address','account','ifsc')}; f['amounts']=[]\n",
+    "    if not text: return f, text\n",
+    "    for k, rx in [('name',NAME_RE),('dob',DOB_RE),('address',ADDR_RE)]:\n",
+    "        m = rx.search(text)\n",
+    "        if m: f[k] = _norm(m.group(1))\n",
+    "    accs = ACC_RE.findall(text); ifsc = IFSC_RE.findall(text)\n",
+    "    if accs: f['account'] = accs[0]\n",
+    "    if ifsc: f['ifsc'] = ifsc[0]\n",
+    "    f['amounts'] = parse_amounts(text)\n",
+    "    return f, text\n",
+    "\n",
+    "def cross_doc_consistency(paths):\n",
+    "    if len(paths) < 2: return {'error':'need >=2 documents'}\n",
+    "    extracts = [{'file':str(p), 'fields':extract_identity_fields(p)[0]} for p in paths]\n",
+    "    field_results = {}\n",
+    "    from difflib import SequenceMatcher\n",
+    "    for field in ['name','dob','address','account','ifsc']:\n",
+    "        vals = [e['fields'].get(field) for e in extracts]\n",
+    "        present = [v for v in vals if v]\n",
+    "        if len(present) < 2:\n",
+    "            field_results[field] = {'status':'insufficient_data','values':vals,'similarity':None}\n",
+    "            continue\n",
+    "        sims = [SequenceMatcher(None, a, b).ratio()\n",
+    "                for i,a in enumerate(present) for b in present[i+1:]]\n",
+    "        ms = min(sims)\n",
+    "        status = 'match' if ms>=0.95 else 'likely_match' if ms>=0.75 else 'mismatch'\n",
+    "        field_results[field] = {'status':status,'values':vals,'similarity':round(ms,3)}\n",
+    "    mm = sum(1 for r in field_results.values() if r['status']=='mismatch')\n",
+    "    lm = sum(1 for r in field_results.values() if r['status']=='likely_match')\n",
+    "    rs = min(1.0, mm*0.5 + lm*0.2)\n",
+    "    return {'documents':extracts, 'field_results':field_results,\n",
+    "            'mismatches':mm, 'likely_mismatches':lm,\n",
+    "            'consistency_risk_score':round(rs,3), 'consistency_band':band(rs)}\n",
+    "\n",
+    "files = [sorted((DATA/'images/originals').glob('land_*.png'))[0],\n",
+    "         sorted((DATA/'images/originals').glob('agreement_*.png'))[0]]\n",
+    "r = cross_doc_consistency(files)\n",
+    "print('Band:', r['consistency_band'], ' Mismatches:', r['mismatches'])\n",
+    "for f, v in r['field_results'].items():\n",
+    "    print(f'  {f:10s} {v[\"status\"]:20s} sim={v[\"similarity\"]}')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 10. Underwriter dashboard + batch audit\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def render_dashboard(report):\n",
+    "    fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
+    "    sub = report.get('sub_scores', {})\n",
+    "    if sub:\n",
+    "        keys = list(sub.keys()); vals = list(sub.values())\n",
+    "        bars = axes[0].barh(keys, vals)\n",
+    "        for b,v in zip(bars, vals):\n",
+    "            b.set_color('green' if v<0.4 else 'orange' if v<0.7 else 'red')\n",
+    "        axes[0].set_xlim(0,1); axes[0].set_title('Sub-scores')\n",
+    "    axes[1].axis('off')\n",
+    "    cmap = {'LOW':'green','MEDIUM':'gold','HIGH':'orange','CRITICAL':'red'}\n",
+    "    risk = report.get('risk_band','N/A')\n",
+    "    axes[1].text(0.05, 0.85, f'RISK: {risk}', fontsize=22,\n",
+    "                 color=cmap.get(risk,'black'), weight='bold')\n",
+    "    axes[1].text(0.05, 0.70, f'Score: {report.get(\"risk_score\",\"-\")}', fontsize=14)\n",
+    "    axes[1].text(0.05, 0.60, f'Action: {report.get(\"recommended_action\",\"-\")}', fontsize=11)\n",
+    "    y = 0.45\n",
+    "    for e in report.get('evidence', []):\n",
+    "        axes[1].text(0.05, y, '- '+e, fontsize=10, wrap=True); y -= 0.07\n",
+    "    plt.tight_layout(); plt.show()\n",
+    "\n",
+    "render_dashboard(analyse_document(sorted((DATA/'images/tampered').glob('land_*.png'))[0]))\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def batch_analyse(folder, out_csv='audit_log.csv'):\n",
+    "    folder = Path(folder); reports = []\n",
+    "    for p in folder.rglob('*'):\n",
+    "        if p.suffix.lower() in {'.png','.jpg','.jpeg','.pdf'}:\n",
+    "            try: reports.append(analyse_document(p))\n",
+    "            except Exception as e: reports.append({'file':str(p),'error':str(e)})\n",
+    "    df = pd.DataFrame([{'file':r.get('file'), 'type':r.get('type'),\n",
+    "                         'risk_score':r.get('risk_score'),\n",
+    "                         'risk_band':r.get('risk_band'),\n",
+    "                         'action':r.get('recommended_action')} for r in reports])\n",
+    "    df.to_csv(out_csv, index=False)\n",
+    "    return df\n",
+    "\n",
+    "audit = batch_analyse(DATA)\n",
+    "audit.head(10)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 11. PDF audit report generator\n",
+    "\n",
+    "Bank-letterhead PDF with risk verdict, evidence, embedded heatmaps.\n",
+    "Uses ReportLab.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install --quiet reportlab\n",
+    "from reportlab.lib.pagesizes import A4\n",
+    "from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle\n",
+    "from reportlab.lib.units import cm\n",
+    "from reportlab.lib import colors\n",
+    "from reportlab.platypus import (SimpleDocTemplate, Paragraph, Spacer, Table,\n",
+    "                                 TableStyle, Image as RLImage, PageBreak)\n",
+    "from reportlab.lib.enums import TA_CENTER\n",
+    "\n",
+    "BAND_C = {'LOW':colors.HexColor('#16a34a'),'MEDIUM':colors.HexColor('#ca8a04'),\n",
+    "          'HIGH':colors.HexColor('#ea580c'),'CRITICAL':colors.HexColor('#dc2626')}\n",
+    "\n",
+    "def build_pdf_report(report, source_path):\n",
+    "    source_path = Path(source_path); s = getSampleStyleSheet()\n",
+    "    s.add(ParagraphStyle('Title2', parent=s['Title'], fontSize=22,\n",
+    "                          textColor=colors.HexColor('#1e3a8a')))\n",
+    "    s.add(ParagraphStyle('Mono', parent=s['Normal'], fontName='Courier',\n",
+    "                          fontSize=8, textColor=colors.dimgray))\n",
+    "    buf = io.BytesIO()\n",
+    "    doc = SimpleDocTemplate(buf, pagesize=A4,\n",
+    "                             leftMargin=2*cm, rightMargin=2*cm,\n",
+    "                             topMargin=1.5*cm, bottomMargin=1.5*cm)\n",
+    "    story = [\n",
+    "        Paragraph('DOCSENTRY - DOCUMENT FORENSICS REPORT', s['Title2']),\n",
+    "        Paragraph('<para alignment=\"center\"><font color=\"grey\">Confidential - For Underwriting Use Only</font></para>', s['Normal']),\n",
+    "        Spacer(1, 0.4*cm),\n",
+    "    ]\n",
+    "    # metadata table\n",
+    "    meta = [['Field','Value'],\n",
+    "            ['Document', source_path.name],\n",
+    "            ['Type', report.get('type','-')],\n",
+    "            ['Analysed at', report.get('analysed_at','-')[:19].replace('T',' ')],\n",
+    "            ['SHA-256', report.get('sha256','-')[:32]+'...']]\n",
+    "    t = Table(meta, colWidths=[4*cm, 13*cm])\n",
+    "    t.setStyle(TableStyle([\n",
+    "        ('BACKGROUND',(0,0),(-1,0), colors.HexColor('#1e3a8a')),\n",
+    "        ('TEXTCOLOR',(0,0),(-1,0), colors.white),\n",
+    "        ('GRID',(0,0),(-1,-1), 0.4, colors.grey),\n",
+    "        ('FONTSIZE',(0,0),(-1,-1), 9)]))\n",
+    "    story += [t, Spacer(1, 0.4*cm)]\n",
+    "    # verdict box\n",
+    "    band_str = report.get('risk_band','UNKNOWN')\n",
+    "    bc = BAND_C.get(band_str, colors.grey)\n",
+    "    vt = Table([[Paragraph(f'<para alignment=\"center\"><font size=22 color=\"white\"><b>{band_str}</b></font></para>', s['Normal']),\n",
+    "                 Paragraph(f'<b>Risk score:</b> {report.get(\"risk_score\",\"-\")}<br/>'\n",
+    "                            f'<b>Action:</b> {report.get(\"recommended_action\",\"-\")}', s['Normal'])]],\n",
+    "                colWidths=[5*cm, 12*cm])\n",
+    "    vt.setStyle(TableStyle([\n",
+    "        ('BACKGROUND',(0,0),(0,0), bc),\n",
+    "        ('BACKGROUND',(1,0),(1,0), colors.HexColor('#f1f5f9')),\n",
+    "        ('VALIGN',(0,0),(-1,-1),'MIDDLE'),\n",
+    "        ('TOPPADDING',(0,0),(-1,-1), 12),\n",
+    "        ('BOTTOMPADDING',(0,0),(-1,-1), 12),\n",
+    "        ('LEFTPADDING',(0,0),(-1,-1), 12),\n",
+    "        ('RIGHTPADDING',(0,0),(-1,-1), 12)]))\n",
+    "    story.append(vt)\n",
+    "    story.append(Spacer(1, 0.4*cm))\n",
+    "    # evidence\n",
+    "    story.append(Paragraph('<b>Forensic evidence</b>', s['Heading3']))\n",
+    "    for e in report.get('evidence', []):\n",
+    "        story.append(Paragraph('&bull; '+e, s['Normal']))\n",
+    "    story.append(Spacer(1, 0.3*cm))\n",
+    "    story.append(Paragraph('<i>Generated by DocSentry. Heuristic + ML ensemble. '\n",
+    "                            'Manual review required for HIGH/CRITICAL.</i>', s['Mono']))\n",
+    "    doc.build(story)\n",
+    "    buf.seek(0); return buf.read()\n",
+    "\n",
+    "# Generate a sample report\n",
+    "sample = sorted((DATA/'images/tampered').glob('land_*.png'))[0]\n",
+    "r = analyse_document(sample)\n",
+    "pdf_bytes = build_pdf_report(r, sample)\n",
+    "Path('reports').mkdir(exist_ok=True)\n",
+    "(Path('reports')/f'audit_{sample.stem}.pdf').write_bytes(pdf_bytes)\n",
+    "print(f'Wrote: reports/audit_{sample.stem}.pdf  ({len(pdf_bytes)} bytes)')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 12. Export Streamlit demo files (forensics.py, app.py, audit_report.py)\n",
+    "\n",
+    "**This is the cell that links the notebook to the live web demo.**\n",
+    "Run it once. It writes 3 files at the repo root with the same logic\n",
+    "as above. After that:\n",
+    "```\n",
+    "streamlit run app.py\n",
+    "```\n",
+    "\n",
+    "Re-run this cell any time you change the detector logic and want the app\n",
+    "to pick up your changes. The cell is idempotent.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Section 12 - Generate forensics.py from in-notebook functions\n",
+    "import inspect\n",
+    "\n",
+    "FORENSICS_HEADER = \"\"\"# Auto-generated from docsentry_master.ipynb. Edit notebook, not this file.\n",
+    "import os, io, re, math, json, hashlib, shutil, warnings\n",
+    "from pathlib import Path\n",
+    "from datetime import datetime\n",
+    "import numpy as np, pandas as pd\n",
+    "from PIL import Image, ImageChops, ImageEnhance\n",
+    "import cv2, fitz, pytesseract, joblib\n",
+    "from skimage.feature import graycomatrix, graycoprops\n",
+    "from difflib import SequenceMatcher\n",
+    "warnings.filterwarnings('ignore')\n",
+    "\n",
+    "TESSERACT_OK = False\n",
+    "for _c in [shutil.which('tesseract'),\n",
+    "           r'C:\\\\Program Files\\\\Tesseract-OCR\\\\tesseract.exe',\n",
+    "           r'C:\\\\Program Files (x86)\\\\Tesseract-OCR\\\\tesseract.exe',\n",
+    "           os.path.expanduser(r'~\\\\AppData\\\\Local\\\\Programs\\\\Tesseract-OCR\\\\tesseract.exe')]:\n",
+    "    if _c and os.path.isfile(_c):\n",
+    "        pytesseract.pytesseract.tesseract_cmd = _c\n",
+    "        TESSERACT_OK = True; break\n",
+    "\n",
+    "AMT_RE  = re.compile(r'(?<![A-Za-z])[-]?\\\\d{1,3}(?:,\\\\d{2,3})*(?:\\\\.\\\\d{1,2})?')\n",
+    "DATE_RE = re.compile(r'(\\\\d{1,2}[-/]\\\\d{1,2}[-/]\\\\d{2,4})')\n",
+    "IFSC_RE = re.compile(r'\\\\b[A-Z]{4}0[A-Z0-9]{6}\\\\b')\n",
+    "ACC_RE  = re.compile(r'\\\\b\\\\d{9,18}\\\\b')\n",
+    "NAME_RE = re.compile(r'(?:Name|Owner|Borrower|Holder|Account Holder)\\\\s*[:\\\\-]\\\\s*([A-Z][A-Z\\\\s.]{2,40})', re.IGNORECASE)\n",
+    "DOB_RE  = re.compile(r'(?:DOB|Date of Birth|Born)\\\\s*[:\\\\-]\\\\s*(\\\\d{1,2}[-/]\\\\d{1,2}[-/]\\\\d{2,4})', re.IGNORECASE)\n",
+    "ADDR_RE = re.compile(r'(?:Address|Village|Residence)\\\\s*[:\\\\-]\\\\s*([A-Z0-9][A-Z0-9\\\\s,.\\\\-/]{3,80})', re.IGNORECASE)\n",
+    "\n",
+    "MODEL_PATH     = Path('models/forgery_rf.joblib')\n",
+    "CNN_MODEL_PATH = Path('models/forgery_cnn.keras')\n",
+    "CNN_META_PATH  = Path('models/forgery_cnn.meta.json')\n",
+    "_CNN = {'model': None, 'meta': None, 'tried': False}\n",
+    "\"\"\"\n",
+    "\n",
+    "FORENSICS_HEADER += f'WEIGHTS = {WEIGHTS!r}\\n'\n",
+    "FORENSICS_HEADER += f'INSIGHT_RULES = {INSIGHT_RULES!r}\\n'\n",
+    "FORENSICS_HEADER += f'ACTIONS = {ACTIONS!r}\\n\\n'\n",
+    "\n",
+    "FUNCS = [error_level_analysis, copy_move_detect, noise_inconsistency,\n",
+    "         exif_sanity, pdf_structural_audit, pdf_font_audit,\n",
+    "         ocr_text, parse_amounts, text_rule_checks,\n",
+    "         band, score_image, generate_insights,\n",
+    "         extract_features, predict_with_model,\n",
+    "         _load_cnn, predict_with_cnn,\n",
+    "         extract_identity_fields, cross_doc_consistency,\n",
+    "         analyse_document]\n",
+    "body = ''\n",
+    "for fn in FUNCS:\n",
+    "    body += inspect.getsource(fn) + '\\n'\n",
+    "\n",
+    "Path('forensics.py').write_text(FORENSICS_HEADER + body)\n",
+    "print('Wrote forensics.py')\n",
+    "import ast\n",
+    "ast.parse(Path('forensics.py').read_text())\n",
+    "print('forensics.py syntax OK')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "APP_PY = '''\"\"\"app.py - Streamlit demo (auto-generated from docsentry_master.ipynb)\"\"\"\n",
+    "import io, json, tempfile\n",
+    "from pathlib import Path\n",
+    "import streamlit as st\n",
+    "import numpy as np, pandas as pd, cv2\n",
+    "from PIL import Image\n",
+    "import matplotlib.pyplot as plt\n",
+    "import forensics\n",
+    "from audit_report import build_pdf_report\n",
+    "\n",
+    "st.set_page_config(page_title=\"DocSentry\", page_icon=\":lock:\", layout=\"wide\")\n",
+    "st.markdown(\"\"\"<style>.big-risk{font-size:48px;font-weight:800;padding:14px 28px;\n",
+    "  border-radius:12px;color:white;text-align:center}.low{background:#16a34a}\n",
+    "  .medium{background:#ca8a04}.high{background:#ea580c}.critical{background:#dc2626}\n",
+    "  </style>\"\"\", unsafe_allow_html=True)\n",
+    "st.title(\":shield: DocSentry - Document Forensics\")\n",
+    "st.caption(\"Real-time anomaly detection for underwriting.\")\n",
+    "if not forensics.TESSERACT_OK:\n",
+    "    st.warning(\"Tesseract not installed - text-rule checks skipped.\")\n",
+    "\n",
+    "def risk_badge(b): st.markdown(f\"<div class=\\'big-risk {b.lower()}\\'>{b}</div>\", unsafe_allow_html=True)\n",
+    "def save(u):\n",
+    "    t = tempfile.NamedTemporaryFile(delete=False, suffix=Path(u.name).suffix)\n",
+    "    t.write(u.getbuffer()); t.close(); return Path(t.name)\n",
+    "\n",
+    "tab1, tab2, tab3 = st.tabs([\":mag: Single doc\", \":busts_in_silhouette: Cross-doc\", \":file_folder: Batch\"])\n",
+    "\n",
+    "with tab1:\n",
+    "    sd = Path(\"sample_data\")\n",
+    "    samples = [p for sub in (\"originals\",\"tampered\",\"pdfs\") for p in sorted((sd/sub).glob(\"*\")) if sd.exists()]\n",
+    "    opts = [\"(upload)\"] + [str(p.relative_to(sd)) for p in samples]\n",
+    "    pick = st.selectbox(\"Try a sample, or upload:\", opts)\n",
+    "    path = None\n",
+    "    if pick != \"(upload)\": path = sd / pick\n",
+    "    else:\n",
+    "        u = st.file_uploader(\"Upload\", type=[\"png\",\"jpg\",\"jpeg\",\"pdf\"])\n",
+    "        if u: path = save(u)\n",
+    "    if path:\n",
+    "        r = forensics.analyse_document(path)\n",
+    "        c1, c2 = st.columns([1,2])\n",
+    "        with c1: risk_badge(r[\"risk_band\"]); st.metric(\"Score\", f'{r[\"risk_score\"]:.3f}')\n",
+    "        with c2:\n",
+    "            st.info(r[\"recommended_action\"])\n",
+    "            for e in r[\"evidence\"]: st.markdown(\"- \" + e)\n",
+    "        st.image(str(path), use_container_width=True) if r[\"type\"]==\"image\" else None\n",
+    "        if r[\"type\"] == \"image\":\n",
+    "            ela, _ = forensics.error_level_analysis(path)\n",
+    "            viz, n, _ = forensics.copy_move_detect(path)\n",
+    "            t1, t2 = st.tabs([\"ELA\", f\"Copy-move ({n})\"])\n",
+    "            with t1: st.image(ela)\n",
+    "            with t2: st.image(cv2.cvtColor(viz, cv2.COLOR_BGR2RGB))\n",
+    "        if \"ml_prediction\" in r:\n",
+    "            ml = r[\"ml_prediction\"]; st.metric(\"RF verdict\", f\"{ml[\\'tamper_probability\\']:.1%}\")\n",
+    "        if \"cnn_prediction\" in r:\n",
+    "            cnn = r[\"cnn_prediction\"]; st.metric(\"CNN verdict\", f\"{cnn[\\'tamper_probability\\']:.1%}\")\n",
+    "        st.download_button(\"Audit JSON\", json.dumps(r, indent=2, default=str),\n",
+    "                            file_name=f\"audit_{path.stem}.json\")\n",
+    "        try:\n",
+    "            pdf = build_pdf_report(r, path)\n",
+    "            st.download_button(\"Audit PDF\", pdf, file_name=f\"audit_{path.stem}.pdf\")\n",
+    "        except Exception as e: st.caption(f\"PDF report: {e}\")\n",
+    "\n",
+    "with tab2:\n",
+    "    ups = st.file_uploader(\"Upload 2+ docs\", type=[\"png\",\"jpg\",\"pdf\"], accept_multiple_files=True)\n",
+    "    if ups and len(ups) >= 2:\n",
+    "        paths = [save(u) for u in ups]\n",
+    "        r = forensics.cross_doc_consistency(paths)\n",
+    "        risk_badge(r[\"consistency_band\"])\n",
+    "        st.metric(\"Mismatches\", r[\"mismatches\"])\n",
+    "        rows = []\n",
+    "        for f, v in r[\"field_results\"].items():\n",
+    "            rows.append({\"Field\":f, \"Status\":v[\"status\"], \"Similarity\":v[\"similarity\"]})\n",
+    "        st.dataframe(pd.DataFrame(rows), use_container_width=True)\n",
+    "\n",
+    "with tab3:\n",
+    "    default = Path.cwd() / (\"sample_data\" if not (Path.cwd()/\"data\").exists() else \"data\")\n",
+    "    folder = st.text_input(\"Folder\", value=str(default))\n",
+    "    if st.button(\"Audit\"):\n",
+    "        root = Path(folder); reports = []\n",
+    "        for p in root.rglob(\"*\"):\n",
+    "            if p.suffix.lower() in {\".png\",\".jpg\",\".jpeg\",\".pdf\"}:\n",
+    "                try: reports.append(forensics.analyse_document(p))\n",
+    "                except Exception as e: reports.append({\"file\":str(p),\"error\":str(e)})\n",
+    "        df = pd.DataFrame([{\"file\":r.get(\"file\"), \"band\":r.get(\"risk_band\"),\n",
+    "                            \"score\":r.get(\"risk_score\")} for r in reports])\n",
+    "        st.dataframe(df, use_container_width=True)\n",
+    "        st.download_button(\"CSV\", df.to_csv(index=False), file_name=\"audit_log.csv\")\n",
+    "'''\n",
+    "Path('app.py').write_text(APP_PY)\n",
+    "print('Wrote app.py')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "AUDIT_PY = '''\"\"\"audit_report.py - PDF report (auto-generated from notebook)\"\"\"\n",
+    "import io\n",
+    "from pathlib import Path\n",
+    "from reportlab.lib.pagesizes import A4\n",
+    "from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle\n",
+    "from reportlab.lib.units import cm\n",
+    "from reportlab.lib import colors\n",
+    "from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, Table, TableStyle\n",
+    "\n",
+    "BAND_C = {\"LOW\":colors.HexColor(\"#16a34a\"),\"MEDIUM\":colors.HexColor(\"#ca8a04\"),\n",
+    "          \"HIGH\":colors.HexColor(\"#ea580c\"),\"CRITICAL\":colors.HexColor(\"#dc2626\")}\n",
+    "\n",
+    "def build_pdf_report(report, source_path):\n",
+    "    source_path = Path(source_path); s = getSampleStyleSheet()\n",
+    "    s.add(ParagraphStyle(\"Title2\", parent=s[\"Title\"], fontSize=22,\n",
+    "                          textColor=colors.HexColor(\"#1e3a8a\")))\n",
+    "    buf = io.BytesIO()\n",
+    "    doc = SimpleDocTemplate(buf, pagesize=A4,\n",
+    "                             leftMargin=2*cm, rightMargin=2*cm,\n",
+    "                             topMargin=1.5*cm, bottomMargin=1.5*cm)\n",
+    "    story = [Paragraph(\"DOCSENTRY - DOCUMENT FORENSICS REPORT\", s[\"Title2\"]),\n",
+    "             Spacer(1, 0.3*cm)]\n",
+    "    meta = [[\"Document\", source_path.name],\n",
+    "            [\"Type\", report.get(\"type\",\"-\")],\n",
+    "            [\"Analysed at\", report.get(\"analysed_at\",\"-\")[:19].replace(\"T\",\" \")],\n",
+    "            [\"SHA-256\", report.get(\"sha256\",\"-\")[:32]+\"...\"]]\n",
+    "    t = Table(meta, colWidths=[4*cm, 13*cm])\n",
+    "    t.setStyle(TableStyle([(\"GRID\",(0,0),(-1,-1),0.4,colors.grey),(\"FONTSIZE\",(0,0),(-1,-1),9)]))\n",
+    "    story += [t, Spacer(1, 0.4*cm)]\n",
+    "    band_str = report.get(\"risk_band\",\"UNKNOWN\")\n",
+    "    bc = BAND_C.get(band_str, colors.grey)\n",
+    "    vt = Table([[Paragraph(f\\'<para alignment=\"center\"><font size=22 color=\"white\"><b>{band_str}</b></font></para>\\', s[\"Normal\"]),\n",
+    "                 Paragraph(f\\'<b>Risk score:</b> {report.get(\"risk_score\",\"-\")}<br/><b>Action:</b> {report.get(\"recommended_action\",\"-\")}\\', s[\"Normal\"])]],\n",
+    "                 colWidths=[5*cm, 12*cm])\n",
+    "    vt.setStyle(TableStyle([(\"BACKGROUND\",(0,0),(0,0), bc),\n",
+    "                              (\"BACKGROUND\",(1,0),(1,0), colors.HexColor(\"#f1f5f9\")),\n",
+    "                              (\"VALIGN\",(0,0),(-1,-1),\"MIDDLE\"),\n",
+    "                              (\"TOPPADDING\",(0,0),(-1,-1),12),(\"BOTTOMPADDING\",(0,0),(-1,-1),12)]))\n",
+    "    story.append(vt); story.append(Spacer(1, 0.4*cm))\n",
+    "    story.append(Paragraph(\"<b>Forensic evidence</b>\", s[\"Heading3\"]))\n",
+    "    for e in report.get(\"evidence\",[]): story.append(Paragraph(\"&bull; \"+e, s[\"Normal\"]))\n",
+    "    doc.build(story)\n",
+    "    buf.seek(0); return buf.read()\n",
+    "'''\n",
+    "Path('audit_report.py').write_text(AUDIT_PY)\n",
+    "# requirements + packages.txt for Streamlit Cloud\n",
+    "Path('requirements.txt').write_text('\\\\n'.join([\n",
+    "    'numpy','pandas','matplotlib','scikit-image','scikit-learn','joblib',\n",
+    "    'opencv-python-headless','Pillow','pytesseract','pdfplumber','pymupdf',\n",
+    "    'pikepdf','python-dateutil','streamlit','reportlab','tensorflow-cpu']))\n",
+    "Path('packages.txt').write_text('tesseract-ocr\\\\nlibtesseract-dev\\\\n')\n",
+    "print('Wrote audit_report.py, requirements.txt, packages.txt')\n",
+    "print()\n",
+    "print('Streamlit demo files are ready. Launch with:')\n",
+    "print('  streamlit run app.py')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 13. Launch the live app\n",
+    "\n",
+    "After running section 12, the supporting files exist at the repo root.\n",
+    "Open a terminal in this folder and run:\n",
+    "\n",
+    "```\n",
+    "streamlit run app.py\n",
+    "```\n",
+    "\n",
+    "Or to deploy to Streamlit Community Cloud (free public URL):\n",
+    "1. Push this folder to a public GitHub repo\n",
+    "2. Connect at https://share.streamlit.io\n",
+    "3. Pick the repo, main file `app.py`, click Deploy\n",
+    "\n",
+    "## 14. Where to go next\n",
+    "\n",
+    "- **Train CNN on real CASIA v2** - section 7, flip `TRAIN_CNN=True` on Colab GPU\n",
+    "- **Add signature verification** - Siamese network for borrower signatures\n",
+    "- **Wrap as FastAPI** - turn `analyse_document` into an HTTP endpoint\n",
+    "- **Grad-CAM overlays** - show which pixels the CNN flagged\n",
+    "\n",
+    "**Everything in this notebook is free, runs CPU-only by default, and demos end-to-end without any paid API call.**\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.10"
+  },
+  "colab": {
+   "provenance": [],
+   "toc_visible": true
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

forensics.py

@@ -0,0 +1,542 @@
+"""
+forensics.py - Document forensics core module
+
+Reusable analysis functions extracted from anomaly_detection_banking.ipynb.
+Imported by app.py (Streamlit) and the notebook.
+
+Public API:
+    analyse_document(path)           - end-to-end pipeline
+    score_image(path)                - image-only forensic score
+    error_level_analysis(path)       - ELA image + score
+    copy_move_detect(path)           - copy-move heatmap + match count
+    noise_inconsistency(path)        - noise heatmap + outlier ratio
+    exif_sanity(path)                - metadata flags
+    pdf_structural_audit(path)       - EOF count + producer/creator
+    pdf_font_audit(path)             - font count + flags
+    ocr_text(path)                   - OCR (no-op if Tesseract missing)
+    text_rule_checks(text)           - date/amount/IFSC sanity
+    extract_features(path)           - feature vector for ML model
+    predict_with_model(path)         - run trained Random Forest if present
+    generate_insights(score, sub, flags) - rule-based bullets
+    band(score)                      - score -> LOW/MEDIUM/HIGH/CRITICAL
+"""
+
+import os
+import io
+import re
+import math
+import json
+import hashlib
+import shutil
+import warnings
+from pathlib import Path
+from datetime import datetime
+
+import numpy as np
+import pandas as pd
+from PIL import Image, ImageChops, ImageEnhance
+import cv2
+import fitz                        # PyMuPDF
+import pytesseract
+
+warnings.filterwarnings("ignore")
+
+# -------------------------------------------------------------
+# Tesseract auto-detect (Windows-friendly)
+# -------------------------------------------------------------
+TESSERACT_OK = False
+for _c in [
+    shutil.which("tesseract"),
+    r"C:\Program Files\Tesseract-OCR\tesseract.exe",
+    r"C:\Program Files (x86)\Tesseract-OCR\tesseract.exe",
+    os.path.expanduser(r"~\AppData\Local\Programs\Tesseract-OCR\tesseract.exe"),
+]:
+    if _c and os.path.isfile(_c):
+        pytesseract.pytesseract.tesseract_cmd = _c
+        TESSERACT_OK = True
+        break
+
+# -------------------------------------------------------------
+# Image forensics
+# -------------------------------------------------------------
+def error_level_analysis(path, quality=90, scale=15):
+    orig = Image.open(path).convert("RGB")
+    buf = io.BytesIO()
+    orig.save(buf, "JPEG", quality=quality)
+    buf.seek(0)
+    resaved = Image.open(buf)
+    diff = ImageChops.difference(orig, resaved)
+    extrema = diff.getextrema()
+    max_diff = max([e[1] for e in extrema]) or 1
+    ela = ImageEnhance.Brightness(diff).enhance(scale * 255 / max_diff)
+    score = float(np.array(diff).mean())
+    return ela, score
+
+
+def copy_move_detect(path, min_dist=40, max_matches=80):
+    img = cv2.imread(str(path))
+    if img is None:
+        return None, 0, []
+    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
+    orb = cv2.ORB_create(nfeatures=2000)
+    kp, des = orb.detectAndCompute(gray, None)
+    if des is None or len(kp) < 10:
+        return img, 0, []
+    bf = cv2.BFMatcher(cv2.NORM_HAMMING)
+    matches = bf.knnMatch(des, des, k=10)
+    good = []
+    for m_list in matches:
+        for m in m_list[1:]:
+            p1 = kp[m.queryIdx].pt
+            p2 = kp[m.trainIdx].pt
+            d = math.hypot(p1[0] - p2[0], p1[1] - p2[1])
+            if d > min_dist and m.distance < 40:
+                good.append((p1, p2, d))
+    good = good[:max_matches]
+    out = img.copy()
+    for p1, p2, _ in good:
+        cv2.line(out, tuple(map(int, p1)), tuple(map(int, p2)), (0, 0, 255), 1)
+        cv2.circle(out, tuple(map(int, p1)), 3, (0, 255, 0), -1)
+        cv2.circle(out, tuple(map(int, p2)), 3, (0, 255, 0), -1)
+    return out, len(good), good
+
+
+def noise_inconsistency(path, block=32):
+    img = cv2.imread(str(path), cv2.IMREAD_GRAYSCALE)
+    if img is None:
+        return np.zeros((1, 1)), 0.0
+    H, W = img.shape
+    Hc, Wc = (H // block) * block, (W // block) * block
+    if Hc == 0 or Wc == 0:
+        return np.zeros((1, 1)), 0.0
+    img = img[:Hc, :Wc]
+    lap = cv2.Laplacian(img, cv2.CV_64F)
+    lap_blocks = (lap.reshape(Hc // block, block, Wc // block, block)
+                     .transpose(0, 2, 1, 3)
+                     .reshape(-1, block * block))
+    var = lap_blocks.var(axis=1)
+    z = (var - var.mean()) / (var.std() + 1e-9)
+    suspicious = (np.abs(z) > 2.5).sum() / max(1, len(z))
+    heat = np.abs(z).reshape(Hc // block, Wc // block)
+    return heat, float(suspicious)
+
+
+def exif_sanity(path):
+    try:
+        img = Image.open(path)
+        exif = img.getexif()
+    except Exception:
+        return ["cannot read image"]
+    if not exif:
+        return ["no EXIF metadata (re-saved or stripped)"]
+    tags = {Image.ExifTags.TAGS.get(k, k): v for k, v in exif.items()}
+    flags = []
+    sw = str(tags.get("Software", "")).lower()
+    for bad in ["photoshop", "gimp", "paint", "snapseed", "picsart"]:
+        if bad in sw:
+            flags.append("edited with " + bad)
+    if "DateTimeOriginal" in tags and "DateTime" in tags:
+        if tags["DateTimeOriginal"] != tags["DateTime"]:
+            flags.append("modified-time differs from original-time")
+    return flags or ["exif clean"]
+
+
+# -------------------------------------------------------------
+# PDF forensics
+# -------------------------------------------------------------
+def pdf_structural_audit(path):
+    raw = Path(path).read_bytes()
+    eofs = raw.count(b"%%EOF")
+    with fitz.open(path) as d:
+        info = d.metadata or {}
+        n_pages = d.page_count
+    flags = []
+    if eofs > 1:
+        flags.append(f"{eofs} EOF markers (incremental updates)")
+    prod = (info.get("producer") or "").lower()
+    crt  = (info.get("creator") or "").lower()
+    if prod and crt and prod != crt:
+        flags.append(f"producer/creator differ: {prod} vs {crt}")
+    for t in ["ilovepdf", "smallpdf", "pdfescape", "sejda", "foxit phantom"]:
+        if t in prod or t in crt:
+            flags.append("edited via consumer tool: " + t)
+    return {"pages": n_pages, "eof_markers": eofs,
+            "metadata": info, "flags": flags or ["clean"]}
+
+
+def pdf_font_audit(path):
+    fonts_per_page = []
+    with fitz.open(path) as d:
+        for page in d:
+            fonts_per_page.append({f[3] for f in page.get_fonts()})
+    all_fonts = set().union(*fonts_per_page) if fonts_per_page else set()
+    flags = []
+    if len(all_fonts) > 4:
+        flags.append("unusually high font count: " + str(len(all_fonts)))
+    return {"fonts": sorted(all_fonts), "flags": flags or ["ok"]}
+
+
+# -------------------------------------------------------------
+# OCR + text rules
+# -------------------------------------------------------------
+AMT_RE  = re.compile(r"(?<![A-Za-z])[-]?\d{1,3}(?:,\d{2,3})*(?:\.\d{1,2})?")
+DATE_RE = re.compile(r"(\d{1,2}[-/]\d{1,2}[-/]\d{2,4})")
+IFSC_RE = re.compile(r"\b[A-Z]{4}0[A-Z0-9]{6}\b")
+ACC_RE  = re.compile(r"\b\d{9,18}\b")
+
+
+def ocr_text(path):
+    if not TESSERACT_OK:
+        return ""
+    try:
+        return pytesseract.image_to_string(Image.open(path))
+    except Exception:
+        return ""
+
+
+def parse_amounts(text):
+    vals = []
+    for m in AMT_RE.findall(text):
+        try:
+            vals.append(float(m.replace(",", "")))
+        except ValueError:
+            pass
+    return vals
+
+
+def text_rule_checks(text):
+    if not text:
+        return {"n_dates": 0, "n_amounts": 0, "n_ifsc": 0,
+                "n_accounts": 0, "flags": ["ocr_skipped"]}
+    flags = []
+    dates = DATE_RE.findall(text)
+    ifsc  = IFSC_RE.findall(text)
+    accs  = ACC_RE.findall(text)
+    amts  = parse_amounts(text)
+    if dates:
+        try:
+            from dateutil import parser as dp
+            ds = [dp.parse(d, dayfirst=True) for d in dates]
+            if any(ds[i] > ds[i + 1] for i in range(len(ds) - 1)):
+                flags.append("dates not monotonic")
+        except Exception:
+            flags.append("unparseable dates")
+    if amts:
+        big_round = [a for a in amts if a >= 100000 and a % 100000 == 0]
+        if len(big_round) > 3:
+            flags.append(f"{len(big_round)} suspiciously round large amounts")
+    if accs and not ifsc:
+        flags.append("account number present but no IFSC")
+    return {"n_dates": len(dates), "n_amounts": len(amts),
+            "n_ifsc": len(ifsc), "n_accounts": len(accs),
+            "flags": flags or ["ok"]}
+
+
+# -------------------------------------------------------------
+# Scoring & insights
+# -------------------------------------------------------------
+WEIGHTS = {"ela": 0.20, "copy_move": 0.25, "noise": 0.15, "exif": 0.10,
+           "pdf_struct": 0.15, "text_rules": 0.10, "math": 0.05}
+
+INSIGHT_RULES = [
+    ("copy_move", 0.4, "Possible copy-paste forgery: repeated visual region. Inspect seal/signature area."),
+    ("ela",       0.4, "Compression artefacts inconsistent with a single-source scan. Likely re-saved after edits."),
+    ("noise",     0.4, "Localised noise inconsistency - common in image splicing."),
+    ("exif",      0.4, "Image metadata indicates edits in a photo-editor or stripped EXIF."),
+    ("pdf_struct", 0.4, "PDF structural anomalies detected (incremental edits / consumer-tool fingerprint)."),
+]
+
+ACTIONS = {
+    "LOW":      "Proceed with standard underwriting.",
+    "MEDIUM":   "Request additional verification documents.",
+    "HIGH":     "Escalate to fraud-risk team; manual review mandatory.",
+    "CRITICAL": "Block file; trigger investigation workflow.",
+}
+
+
+def band(score):
+    if score < 0.25: return "LOW"
+    if score < 0.50: return "MEDIUM"
+    if score < 0.75: return "HIGH"
+    return "CRITICAL"
+
+
+def score_image(path):
+    _, ela_s   = error_level_analysis(path)
+    _, n_cm, _ = copy_move_detect(path)
+    _, noise_r = noise_inconsistency(path)
+    exif_flags = exif_sanity(path)
+    sub = {"ela": min(ela_s / 25.0, 1.0),
+           "copy_move": min(n_cm / 50.0, 1.0),
+           "noise": min(noise_r * 4, 1.0),
+           "exif": 0.0 if exif_flags == ["exif clean"] else 0.6}
+    total = sum(WEIGHTS[k] * v for k, v in sub.items())
+    return total, sub, exif_flags
+
+
+def generate_insights(score, sub_scores, extra_flags=None):
+    bullets = []
+    for key, thresh, msg in INSIGHT_RULES:
+        if sub_scores.get(key, 0) >= thresh:
+            bullets.append(msg)
+    if extra_flags:
+        for f in extra_flags:
+            if f not in ("exif clean", "ok", "clean"):
+                bullets.append("Flag: " + str(f))
+    if not bullets:
+        bullets.append("No anomaly indicators above threshold.")
+    return {"risk_score": round(score, 3),
+            "risk_band": band(score),
+            "recommended_action": ACTIONS[band(score)],
+            "evidence": bullets}
+
+
+# -------------------------------------------------------------
+# ML feature extraction + prediction
+# -------------------------------------------------------------
+MODEL_PATH     = Path("models/forgery_rf.joblib")
+CNN_MODEL_PATH = Path("models/forgery_cnn.keras")
+CNN_META_PATH  = Path("models/forgery_cnn.meta.json")
+
+_CNN_CACHE = {"model": None, "meta": None, "tried": False}
+
+
+def _load_cnn():
+    """Lazy-load the CNN model only when first needed (avoids TF import cost)."""
+    if _CNN_CACHE["tried"]:
+        return _CNN_CACHE["model"], _CNN_CACHE["meta"]
+    _CNN_CACHE["tried"] = True
+    if not CNN_MODEL_PATH.exists():
+        return None, None
+    try:
+        import tensorflow as tf       # local import - heavy
+        _CNN_CACHE["model"] = tf.keras.models.load_model(CNN_MODEL_PATH)
+        if CNN_META_PATH.exists():
+            _CNN_CACHE["meta"] = json.loads(CNN_META_PATH.read_text())
+        else:
+            _CNN_CACHE["meta"] = {"image_size": 224, "class_names": ["originals", "tampered"]}
+    except Exception as e:
+        print("CNN load failed:", e)
+    return _CNN_CACHE["model"], _CNN_CACHE["meta"]
+
+
+def predict_with_cnn(path):
+    """Run the trained CNN if forgery_cnn.keras exists. Returns dict or None."""
+    model, meta = _load_cnn()
+    if model is None:
+        return None
+    img_size = meta.get("image_size", 224)
+    img = Image.open(path).convert("RGB").resize((img_size, img_size))
+    arr = np.array(img)[None, ...].astype(np.float32)
+    prob = float(model.predict(arr, verbose=0)[0, 0])
+    return {
+        "tamper_probability": round(prob, 3),
+        "verdict": "TAMPERED" if prob >= 0.5 else "GENUINE",
+        "model": "MobileNetV2 (CASIA v2 fine-tuned)",
+        "val_auc": (meta or {}).get("val_auc"),
+    }
+
+
+
+
+def extract_features(path):
+    from skimage.feature import graycomatrix, graycoprops
+    feats = {}
+    _, ela_score = error_level_analysis(path)
+    feats["ela_mean"] = ela_score
+    _, cm_count, _ = copy_move_detect(path)
+    feats["copy_move_matches"] = cm_count
+    _, noise_ratio = noise_inconsistency(path)
+    feats["noise_outlier_ratio"] = noise_ratio
+    feats["exif_clean"] = int(exif_sanity(path) == ["exif clean"])
+    img = cv2.imread(str(path), cv2.IMREAD_GRAYSCALE)
+    img_s = cv2.resize(img, (256, 256))
+    glcm = graycomatrix(img_s, [1], [0], 256, symmetric=True, normed=True)
+    feats["glcm_contrast"]    = float(graycoprops(glcm, "contrast")[0, 0])
+    feats["glcm_homogeneity"] = float(graycoprops(glcm, "homogeneity")[0, 0])
+    feats["glcm_energy"]      = float(graycoprops(glcm, "energy")[0, 0])
+    feats["glcm_correlation"] = float(graycoprops(glcm, "correlation")[0, 0])
+    col = cv2.imread(str(path))
+    if col is not None:
+        for i, ch in enumerate(["b", "g", "r"]):
+            hist = cv2.calcHist([col], [i], None, [32], [0, 256]).flatten()
+            hist = hist / (hist.sum() + 1e-9)
+            feats["hist_" + ch + "_entropy"] = float(-(hist * np.log2(hist + 1e-9)).sum())
+    return feats
+
+
+def predict_with_model(path, model_path=MODEL_PATH):
+    import joblib
+    if not Path(model_path).exists():
+        return None
+    bundle = joblib.load(model_path)
+    feats = extract_features(path)
+    x = pd.DataFrame([feats])[bundle["features"]]
+    p = bundle["model"].predict_proba(x)[0, 1]
+    return {"file": str(path), "tamper_probability": round(float(p), 3),
+            "verdict": "TAMPERED" if p >= 0.5 else "GENUINE",
+            "features": feats}
+
+
+# -------------------------------------------------------------
+# End-to-end pipeline
+# -------------------------------------------------------------
+def analyse_document(path):
+    path = Path(path)
+    ext = path.suffix.lower()
+    report = {"file": str(path),
+              "analysed_at": datetime.utcnow().isoformat() + "Z",
+              "sha256": hashlib.sha256(path.read_bytes()).hexdigest()}
+
+    if ext in (".png", ".jpg", ".jpeg", ".tif", ".tiff", ".bmp"):
+        report["type"] = "image"
+        s, sub, flags = score_image(path)
+        try:
+            txt = ocr_text(path)
+            text_rules = text_rule_checks(txt)
+            sub["text_rules"] = 0.0 if text_rules["flags"] == ["ok"] else 0.5
+            s = sum(WEIGHTS.get(k, 0) * v for k, v in sub.items())
+        except Exception as e:
+            text_rules = {"error": str(e)}
+        # Blend in RF prediction if model exists
+        try:
+            ml = predict_with_model(path)
+            if ml is not None:
+                s = 0.5 * s + 0.5 * ml["tamper_probability"]
+                report["ml_prediction"] = ml
+        except Exception as e:
+            report["ml_error"] = str(e)
+        # Blend in CNN prediction if model exists (weight rises with val_auc)
+        try:
+            cnn = predict_with_cnn(path)
+            if cnn is not None:
+                # If CNN AUC is known and high, give it more weight than rule-score
+                w = max(0.4, min(0.7, (cnn.get("val_auc") or 0.85)))
+                s = (1 - w) * s + w * cnn["tamper_probability"]
+                report["cnn_prediction"] = cnn
+        except Exception as e:
+            report["cnn_error"] = str(e)
+        insights = generate_insights(s, sub, flags + text_rules.get("flags", []))
+        report.update({"sub_scores": sub, "exif_flags": flags,
+                       "text_rules": text_rules, **insights})
+
+    elif ext == ".pdf":
+        report["type"] = "pdf"
+        audit = pdf_structural_audit(path)
+        fonts = pdf_font_audit(path)
+        sub = {"pdf_struct": 0.8 if audit["flags"] != ["clean"] else 0.1,
+               "text_rules": 0.6 if fonts["flags"] != ["ok"] else 0.1}
+        s = sum(WEIGHTS.get(k, 0) * v for k, v in sub.items())
+        insights = generate_insights(s, sub, audit["flags"] + fonts["flags"])
+        report.update({"sub_scores": sub, "pdf_audit": audit,
+                       "font_audit": fonts, **insights})
+    else:
+        report["type"] = "unsupported"
+        report["error"] = "extension " + ext + " not handled"
+    return report
+
+
+# -------------------------------------------------------------
+# Cross-document consistency (Sprint 2)
+# -------------------------------------------------------------
+NAME_RE = re.compile(r"(?:Name|Owner|Borrower|Holder|Account Holder)\s*[:\-]\s*([A-Z][A-Z\s.]{2,40})", re.IGNORECASE)
+DOB_RE  = re.compile(r"(?:DOB|Date of Birth|Born)\s*[:\-]\s*(\d{1,2}[-/]\d{1,2}[-/]\d{2,4})", re.IGNORECASE)
+ADDR_RE = re.compile(r"(?:Address|Village|Residence)\s*[:\-]\s*([A-Z0-9][A-Z0-9\s,.\-/]{3,80})", re.IGNORECASE)
+
+
+def _norm(s):
+    return re.sub(r"\s+", " ", (s or "").strip().upper())
+
+
+def extract_identity_fields(path):
+    """Pull name, DOB, address, account, IFSC from any document via OCR."""
+    if str(path).lower().endswith(".pdf"):
+        with fitz.open(path) as d:
+            text = "\n".join(page.get_text() for page in d)
+    else:
+        text = ocr_text(path)
+    fields = {
+        "name":    None, "dob": None, "address": None,
+        "account": None, "ifsc": None, "amounts": [],
+    }
+    if not text:
+        return fields, text
+    m = NAME_RE.search(text)
+    if m: fields["name"] = _norm(m.group(1))
+    m = DOB_RE.search(text)
+    if m: fields["dob"]  = _norm(m.group(1))
+    m = ADDR_RE.search(text)
+    if m: fields["address"] = _norm(m.group(1))
+    accs = ACC_RE.findall(text)
+    if accs: fields["account"] = accs[0]
+    ifsc = IFSC_RE.findall(text)
+    if ifsc: fields["ifsc"] = ifsc[0]
+    fields["amounts"] = parse_amounts(text)
+    return fields, text
+
+
+def _similarity(a, b):
+    """Simple ratio-based string similarity."""
+    if not a or not b:
+        return 0.0
+    from difflib import SequenceMatcher
+    return SequenceMatcher(None, a, b).ratio()
+
+
+def cross_doc_consistency(file_paths):
+    """Compare identity fields across 2+ documents. Return per-field verdict."""
+    if len(file_paths) < 2:
+        return {"error": "need at least 2 documents"}
+    extracts = []
+    for p in file_paths:
+        fields, _ = extract_identity_fields(p)
+        extracts.append({"file": str(p), "fields": fields})
+    # Compare each field across docs
+    field_results = {}
+    for field in ["name", "dob", "address", "account", "ifsc"]:
+        values = [e["fields"].get(field) for e in extracts]
+        present = [v for v in values if v]
+        if len(present) < 2:
+            field_results[field] = {
+                "status": "insufficient_data",
+                "values": values,
+                "similarity": None,
+            }
+            continue
+        # All-pairs similarity
+        sims = []
+        for i in range(len(present)):
+            for j in range(i + 1, len(present)):
+                sims.append(_similarity(present[i], present[j]))
+        min_sim = min(sims)
+        if min_sim >= 0.95:
+            status = "match"
+        elif min_sim >= 0.75:
+            status = "likely_match"
+        else:
+            status = "mismatch"
+        field_results[field] = {
+            "status": status,
+            "values": values,
+            "similarity": round(min_sim, 3),
+        }
+    # Aggregate risk
+    mismatches = sum(1 for r in field_results.values() if r["status"] == "mismatch")
+    likely    = sum(1 for r in field_results.values() if r["status"] == "likely_match")
+    risk_score = min(1.0, mismatches * 0.5 + likely * 0.2)
+    return {
+        "documents": extracts,
+        "field_results": field_results,
+        "mismatches": mismatches,
+        "likely_mismatches": likely,
+        "consistency_risk_score": round(risk_score, 3),
+        "consistency_band": band(risk_score),
+    }
+
+
+if __name__ == "__main__":
+    import sys
+    if len(sys.argv) < 2:
+        print("Usage: python forensics.py <file>")
+        sys.exit(1)
+    print(json.dumps(analyse_document(sys.argv[1]), indent=2, default=str))

models/forgery_rf.joblib

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f3d5924665984784844d47cae20b79373439173253289921a6608c76a0cd346
+size 243698

packages.txt

@@ -0,0 +1 @@
+tesseract-ocr\nlibtesseract-dev\n

requirements.txt

@@ -0,0 +1 @@
+numpy\npandas\nmatplotlib\nscikit-image\nscikit-learn\njoblib\nopencv-python-headless\nPillow\npytesseract\npdfplumber\npymupdf\npikepdf\npython-dateutil\nstreamlit\nreportlab\ntensorflow-cpu

sample_data/pdfs/agreement_000.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e3cc121f8054d518169c7cf1ed65f37420586301ae89b4fdd5535df61ed8e04
+size 1051

sample_data/pdfs/agreement_000_tampered.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abb944a9428840c93416b2cdaa85ee58851e523c6f67df6b3f1e0d3d0dc6c2f3
+size 1362