Spaces:
Running
Running
Enhance README with live deployment info and updates
Browse filesAdd live deployment section and update various details.
README.md
CHANGED
|
@@ -23,6 +23,17 @@ All language is analytically neutral and legally defensible.
|
|
| 23 |
|
| 24 |
---
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
## What It Builds
|
| 27 |
|
| 28 |
**Investigation Intelligence (Tab 1)**
|
|
@@ -58,14 +69,14 @@ GeM contracts, court filings, PIB releases, SEBI orders, and verified news.
|
|
| 58 |
bharatgraph/
|
| 59 |
scrapers/ 21 data collectors (Indian govt + international)
|
| 60 |
processing/ Name normalisation, entity resolution, pipeline
|
| 61 |
-
graph/ Neo4j schema, loader, Cypher queries
|
| 62 |
ai/ Risk scoring, NLP, graph analytics, 12 investigators
|
| 63 |
config/ 22 language configs, settings
|
| 64 |
-
api/ FastAPI REST + WebSocket (6 routes)
|
| 65 |
frontend/ HTML/CSS/JS dashboard, D3.js graph, dark/light theme
|
| 66 |
templates/ Jinja2 dossier template (PDF/HTML)
|
| 67 |
blockchain/ Audit log
|
| 68 |
-
|
| 69 |
```
|
| 70 |
|
| 71 |
---
|
|
@@ -76,11 +87,11 @@ bharatgraph/
|
|
| 76 |
|----------------|---------------------|------------------------------------|
|
| 77 |
| Source code | GitHub | Unlimited public repos |
|
| 78 |
| Backend API | Hugging Face Spaces | Docker SDK — no cold start |
|
| 79 |
-
| Frontend | GitHub Pages |
|
| 80 |
| Graph database | Neo4j AuraDB Free | 50K nodes, 175K relationships |
|
| 81 |
| CDN + DDoS | Cloudflare Free | Unlimited caching + DDoS |
|
| 82 |
| CI/CD | GitHub Actions | 2,000 min/month |
|
| 83 |
-
| Uptime monitor | UptimeRobot Free |
|
| 84 |
|
| 85 |
---
|
| 86 |
|
|
@@ -131,66 +142,51 @@ for cross-source entity resolution. Full pipeline orchestrator.
|
|
| 131 |
### Phase 3 — Graph Database
|
| 132 |
Neo4j schema: 7 node types, 6 relationship types, 10 constraint queries.
|
| 133 |
`MERGE` with stable MD5 IDs. 8 pre-built Cypher queries.
|
| 134 |
-
Connected to Neo4j AuraDB Free: `neo4j+s://1a34e3b8.databases.neo4j.io`
|
| 135 |
|
| 136 |
### Phase 4 — FastAPI Backend
|
| 137 |
FastAPI with lifespan, CORS, 6 route modules registered after `app = FastAPI()`,
|
| 138 |
-
WebSocket feed
|
| 139 |
|
| 140 |
### Phase 5 — Risk Scoring Engine
|
| 141 |
Composite 0–100 score. Weights: politician_company_overlap (0.35),
|
| 142 |
contract_concentration (0.25), audit_frequency (0.20), asset_anomaly (0.15),
|
| 143 |
-
criminal_case (0.05). `validate_language()` enforces neutral
|
| 144 |
|
| 145 |
-
### Phase 6 — Expanded Data Sources
|
| 146 |
-
|
| 147 |
-
All public HuggingFace models — no gating.
|
| 148 |
|
| 149 |
### Phase 7 — NLP Document Intelligence
|
| 150 |
spaCy `en_core_web_sm` NER. Benford's Law chi-squared test on affidavit assets.
|
| 151 |
-
Multilingual NER: `Davlan/bert-base-multilingual-cased-ner-hrl`.
|
| 152 |
Shadow draft detector: `all-MiniLM-L6-v2`, 93.35% alignment confirmed.
|
| 153 |
|
| 154 |
### Phase 8 — Advanced Graph Analytics
|
| 155 |
NetworkX: betweenness centrality, PageRank, Louvain community detection.
|
| 156 |
-
Circular ownership `simple_cycles()`
|
| 157 |
-
Ghost company scorer: GHOST_THRESHOLD=60 — Quick Win Pvt Ltd: 100/100 confirmed.
|
| 158 |
|
| 159 |
-
### Phase 9 — Eight New Indian Sources (21 Total)
|
| 160 |
NJDG (39 live records confirmed), ED, CVC, NCRB, LGD, IBBI, NGO Darpan, CPPP.
|
| 161 |
-
All have sample fallbacks when live source unavailable.
|
| 162 |
|
| 163 |
### Phase 10 — Multi-Investigator AI Engine
|
| 164 |
-
12 parallel investigators
|
| 165 |
-
|
| 166 |
-
`FakeSession` for offline testing. `validate_language()` on all output.
|
| 167 |
|
| 168 |
### Phase 11 — Multilingual Platform (22 Languages)
|
| 169 |
-
All 22 Indian scheduled languages. Unicode
|
| 170 |
-
|
| 171 |
-
Modi→मोदी/மோடி/మోదీ/ಮೋದಿ/മോദി cross-script search confirmed.
|
| 172 |
-
Risk levels pre-translated in 9 languages.
|
| 173 |
|
| 174 |
### Phase 12 — PDF Dossier Generator
|
| 175 |
-
SHA-256 integrity hash
|
| 176 |
-
|
| 177 |
-
Template: Indian tricolour design, 8 sections. 10,829 chars rendered confirmed.
|
| 178 |
-
`GET /export/pdf/{entity_id}` · `GET /verify/{hash}`
|
| 179 |
|
| 180 |
### Phase 13 — Production Frontend
|
| 181 |
HTML/CSS/JS — no React, no Node.js, no build step. Works from `file://`.
|
|
|
|
| 182 |
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
- `graph.js`: D3.js force-directed knowledge graph
|
| 189 |
-
- `app.js`: state management, 5 views (home/search/entity/feed/about)
|
| 190 |
-
- `index.html`: semantic HTML5 shell
|
| 191 |
-
|
| 192 |
-
Colours: Saffron #FF9933 · India Green #138808 · Ashoka Blue #000080 · Navy #0A0F2E.
|
| 193 |
-
Dark/light theme toggle with localStorage persistence.
|
| 194 |
|
| 195 |
---
|
| 196 |
|
|
@@ -198,14 +194,24 @@ Dark/light theme toggle with localStorage persistence.
|
|
| 198 |
|
| 199 |
| Phase | Title | Key Capability |
|
| 200 |
|-------|-------|----------------|
|
| 201 |
-
|
|
| 202 |
-
|
|
| 203 |
-
|
|
| 204 |
-
|
|
| 205 |
-
|
|
| 206 |
-
|
|
| 207 |
-
|
|
| 208 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
|
| 210 |
Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
|
| 211 |
|
|
@@ -217,12 +223,14 @@ Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
|
|
| 217 |
DataGov API 3,199 real MGNREGA records
|
| 218 |
CAG 30 audit report links from cag.gov.in
|
| 219 |
PIB 27 press releases from pib.gov.in
|
| 220 |
-
Wikidata Modi (Q1058), Gandhi (Q10218) confirmed
|
| 221 |
NJDG 39 court records confirmed live
|
| 222 |
-
Pipeline
|
| 223 |
SHA-256 Stable, unique, 64 chars — confirmed
|
| 224 |
Language det. Devanagari→hi, Tamil→ta, Latin→en — confirmed
|
| 225 |
-
|
|
|
|
|
|
|
| 226 |
```
|
| 227 |
|
| 228 |
---
|
|
@@ -233,34 +241,21 @@ Transliteration Modi→5 scripts — confirmed
|
|
| 233 |
git clone https://github.com/abinaze/bharatgraph.git
|
| 234 |
cd bharatgraph
|
| 235 |
python -m venv venv
|
| 236 |
-
source venv/Scripts/activate
|
| 237 |
pip install -r requirements.txt
|
| 238 |
cp .env.example .env
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
Edit `.env`:
|
| 242 |
-
```
|
| 243 |
-
DATAGOV_API_KEY=register_free_at_data.gov.in_user_register
|
| 244 |
-
NEO4J_URI=neo4j+s://your-instance.databases.neo4j.io
|
| 245 |
-
NEO4J_USER=neo4j
|
| 246 |
-
NEO4J_PASSWORD=your_password
|
| 247 |
-
```
|
| 248 |
-
|
| 249 |
-
Run:
|
| 250 |
-
```bash
|
| 251 |
python -m processing.pipeline --scrapers cag,gem,pib,myneta,mca
|
| 252 |
python -m graph.loader
|
| 253 |
uvicorn api.main:app --reload
|
| 254 |
```
|
| 255 |
|
| 256 |
-
Open `frontend/index.html`
|
| 257 |
|
| 258 |
---
|
| 259 |
|
| 260 |
## Git Workflow
|
| 261 |
|
| 262 |
-
One long-lived branch: `main`. All branches merge directly into `main`.
|
| 263 |
-
|
| 264 |
```
|
| 265 |
feature/phase-N-short-name New phase development
|
| 266 |
fix/issue-N-description Bug fix
|
|
@@ -269,6 +264,8 @@ docs/description Documentation only
|
|
| 269 |
|
| 270 |
Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
|
| 271 |
|
|
|
|
|
|
|
| 272 |
---
|
| 273 |
|
| 274 |
## Reference Documents
|
|
@@ -277,3 +274,5 @@ Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
|
|
| 277 |
- [Security Policy](SECURITY.md)
|
| 278 |
- [Phase Roadmap](PHASE_ROADMAP.md)
|
| 279 |
- [MIT License](LICENSE)
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
---
|
| 25 |
|
| 26 |
+
## Live Deployment
|
| 27 |
+
|
| 28 |
+
| Layer | URL |
|
| 29 |
+
|-------|-----|
|
| 30 |
+
| Frontend | https://abinaze.github.io/bharatgraph |
|
| 31 |
+
| API | https://abinazebinoy-bharatgraph.hf.space |
|
| 32 |
+
| Health | https://abinazebinoy-bharatgraph.hf.space/health |
|
| 33 |
+
| API Docs | https://abinazebinoy-bharatgraph.hf.space/docs |
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
|
| 37 |
## What It Builds
|
| 38 |
|
| 39 |
**Investigation Intelligence (Tab 1)**
|
|
|
|
| 69 |
bharatgraph/
|
| 70 |
scrapers/ 21 data collectors (Indian govt + international)
|
| 71 |
processing/ Name normalisation, entity resolution, pipeline
|
| 72 |
+
graph/ Neo4j schema, loader, Cypher queries, seed data
|
| 73 |
ai/ Risk scoring, NLP, graph analytics, 12 investigators
|
| 74 |
config/ 22 language configs, settings
|
| 75 |
+
api/ FastAPI REST + WebSocket (6 routes + admin)
|
| 76 |
frontend/ HTML/CSS/JS dashboard, D3.js graph, dark/light theme
|
| 77 |
templates/ Jinja2 dossier template (PDF/HTML)
|
| 78 |
blockchain/ Audit log
|
| 79 |
+
.github/ CI/CD workflows (test, deploy, daily scrape)
|
| 80 |
```
|
| 81 |
|
| 82 |
---
|
|
|
|
| 87 |
|----------------|---------------------|------------------------------------|
|
| 88 |
| Source code | GitHub | Unlimited public repos |
|
| 89 |
| Backend API | Hugging Face Spaces | Docker SDK — no cold start |
|
| 90 |
+
| Frontend | GitHub Pages | Static HTML Actions workflow |
|
| 91 |
| Graph database | Neo4j AuraDB Free | 50K nodes, 175K relationships |
|
| 92 |
| CDN + DDoS | Cloudflare Free | Unlimited caching + DDoS |
|
| 93 |
| CI/CD | GitHub Actions | 2,000 min/month |
|
| 94 |
+
| Uptime monitor | UptimeRobot Free | 5-min interval health checks |
|
| 95 |
|
| 96 |
---
|
| 97 |
|
|
|
|
| 142 |
### Phase 3 — Graph Database
|
| 143 |
Neo4j schema: 7 node types, 6 relationship types, 10 constraint queries.
|
| 144 |
`MERGE` with stable MD5 IDs. 8 pre-built Cypher queries.
|
|
|
|
| 145 |
|
| 146 |
### Phase 4 — FastAPI Backend
|
| 147 |
FastAPI with lifespan, CORS, 6 route modules registered after `app = FastAPI()`,
|
| 148 |
+
WebSocket feed. Version 0.12.0.
|
| 149 |
|
| 150 |
### Phase 5 — Risk Scoring Engine
|
| 151 |
Composite 0–100 score. Weights: politician_company_overlap (0.35),
|
| 152 |
contract_concentration (0.25), audit_frequency (0.20), asset_anomaly (0.15),
|
| 153 |
+
criminal_case (0.05). `validate_language()` enforces neutral output.
|
| 154 |
|
| 155 |
+
### Phase 6 — Expanded Data Sources (13 scrapers)
|
| 156 |
+
ICIJ HTML mode (BeautifulSoup). Wikidata SPARQL confirmed. Public HF models only.
|
|
|
|
| 157 |
|
| 158 |
### Phase 7 — NLP Document Intelligence
|
| 159 |
spaCy `en_core_web_sm` NER. Benford's Law chi-squared test on affidavit assets.
|
|
|
|
| 160 |
Shadow draft detector: `all-MiniLM-L6-v2`, 93.35% alignment confirmed.
|
| 161 |
|
| 162 |
### Phase 8 — Advanced Graph Analytics
|
| 163 |
NetworkX: betweenness centrality, PageRank, Louvain community detection.
|
| 164 |
+
Circular ownership `simple_cycles()`. Ghost company scorer: 100/100 confirmed.
|
|
|
|
| 165 |
|
| 166 |
+
### Phase 9 — Eight New Indian Sources (21 Total Scrapers)
|
| 167 |
NJDG (39 live records confirmed), ED, CVC, NCRB, LGD, IBBI, NGO Darpan, CPPP.
|
|
|
|
| 168 |
|
| 169 |
### Phase 10 — Multi-Investigator AI Engine
|
| 170 |
+
12 parallel investigators. SHA-256 report hash: stable, unique, 64 chars.
|
| 171 |
+
Synthesis: 3+ investigators agreeing = HIGH confidence. `FakeSession` for offline testing.
|
|
|
|
| 172 |
|
| 173 |
### Phase 11 — Multilingual Platform (22 Languages)
|
| 174 |
+
All 22 Indian scheduled languages. Unicode detection confirmed.
|
| 175 |
+
Helsinki-NLP public model. Modi→मोदी/தமிழ்/తెలుగ��/ಕನ್ನಡ/മലയാളം confirmed.
|
|
|
|
|
|
|
| 176 |
|
| 177 |
### Phase 12 — PDF Dossier Generator
|
| 178 |
+
SHA-256 integrity hash. Jinja2 + WeasyPrint (HTML fallback on Windows).
|
| 179 |
+
Indian tricolour design, 8 sections. 10,829 chars rendered confirmed.
|
|
|
|
|
|
|
| 180 |
|
| 181 |
### Phase 13 — Production Frontend
|
| 182 |
HTML/CSS/JS — no React, no Node.js, no build step. Works from `file://`.
|
| 183 |
+
D3.js force-directed graph. Dark/light theme. 5 views: home/search/entity/feed/about.
|
| 184 |
|
| 185 |
+
### Phase 14 — Zero Cold-Start Deployment ✓ LIVE
|
| 186 |
+
Hugging Face Spaces Docker SDK. No cold start on public spaces.
|
| 187 |
+
Service worker cache-first for static assets. GZipMiddleware (60-80% compression).
|
| 188 |
+
`POST /admin/seed` endpoint loads 10 politicians, 5 companies, 3 contracts.
|
| 189 |
+
Frontend on GitHub Pages via GitHub Actions static workflow.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 190 |
|
| 191 |
---
|
| 192 |
|
|
|
|
| 194 |
|
| 195 |
| Phase | Title | Key Capability |
|
| 196 |
|-------|-------|----------------|
|
| 197 |
+
| **15** | **Mathematical Intelligence Engine** | Spectral graph analysis, Fourier timeline, path signatures, persistent homology, expanded Benford, mutual information + causal ranking |
|
| 198 |
+
| **16** | **Evidence Connection Map + Deep Investigation** | Investigation connector map with relationship labels/strength/source, 6-layer recursive engine, clickable evidence panel, hidden-link suggestions, recursive find-more button |
|
| 199 |
+
| **17** | **Security Hardening + Provenance** | Rate limiting, CSP/HSTS headers, SHA-256 audit chain, full provenance layer on every node/edge, deterministic artifact IDs |
|
| 200 |
+
| **18** | **Self-Learning + Case Memory** | Schema adaptation, pattern candidate discovery, case memory library, human-in-the-loop merge queue, investigation replay |
|
| 201 |
+
| **19** | **Affidavit Wealth Trajectory** | Kalman filter across 5 election cycles, unexplained wealth residual scoring, asset disappearance detection |
|
| 202 |
+
| **20** | **Biography Engine** | Complete chronological life timeline from all 28 sources, temporal convergence detection, narrative generation |
|
| 203 |
+
| **21** | **Benami Detection** | Graph embeddings, proxy network analysis, family name matching, director age anomaly |
|
| 204 |
+
| **22** | **Procurement DNA + Cartel Detection** | Bid document fingerprinting, price ratio analysis, cover bid detection, vendor co-bidding network |
|
| 205 |
+
| **23** | **Revolving Door + TBML Detection** | Career graph, cooling-off violations, trade-based money laundering flags |
|
| 206 |
+
| **24** | **Linguistic Fingerprinting** | Burrows' Delta authorship attribution across govt documents, template reuse detection |
|
| 207 |
+
| **25** | **Policy-Benefit Causal Analysis** | Granger causality, transfer entropy, Cumulative Abnormal Contract Award |
|
| 208 |
+
| **26** | **Adversarial Counterevidence + Competing Hypotheses** | Forced disproof search, hypothesis A/B/C mode, evidence scorecard |
|
| 209 |
+
| **27** | **Multi-Agent Debate Engine** | iMAD hesitation detection, 3-round structured debate, anti-drift, adaptive routing, consensus + dissent tracking |
|
| 210 |
+
| **28** | **Dark Pattern Detection** | PrefixSpan sequential pattern mining, 6 pre-defined high-risk patterns |
|
| 211 |
+
| **29** | **Source-Drift + Historical Analysis** | Wayback Machine credibility, archive-gap recovery, temporal slice viewer, claim survival scoring |
|
| 212 |
+
| **30** | **Predictive Risk + Auto-Prioritisation** | ARIMA + Random Forest trajectory, 6-month forecast, lead priority scores |
|
| 213 |
+
| **31** | **Geospatial Verification** | Sentinel-2 satellite imagery, NDVI change detection, build-completion mismatch |
|
| 214 |
+
| **32** | **Identity Fusion + Multimedia OSINT** | Alias-to-email linking, credential reuse detection, persona clustering, background-object analysis |
|
| 215 |
|
| 216 |
Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
|
| 217 |
|
|
|
|
| 223 |
DataGov API 3,199 real MGNREGA records
|
| 224 |
CAG 30 audit report links from cag.gov.in
|
| 225 |
PIB 27 press releases from pib.gov.in
|
| 226 |
+
Wikidata Modi (Q1058), Gandhi (Q10218) confirmed live
|
| 227 |
NJDG 39 court records confirmed live
|
| 228 |
+
Pipeline 28 nodes loaded — 2 politicians, 2 companies, 20 audits, 4 press
|
| 229 |
SHA-256 Stable, unique, 64 chars — confirmed
|
| 230 |
Language det. Devanagari→hi, Tamil→ta, Latin→en — confirmed
|
| 231 |
+
Search Working on production HF Space
|
| 232 |
+
Frontend Live at abinaze.github.io/bharatgraph
|
| 233 |
+
API Live at abinazebinoy-bharatgraph.hf.space
|
| 234 |
```
|
| 235 |
|
| 236 |
---
|
|
|
|
| 241 |
git clone https://github.com/abinaze/bharatgraph.git
|
| 242 |
cd bharatgraph
|
| 243 |
python -m venv venv
|
| 244 |
+
source venv/Scripts/activate
|
| 245 |
pip install -r requirements.txt
|
| 246 |
cp .env.example .env
|
| 247 |
+
# Edit .env with your Neo4j URI, user, password and DATAGOV_API_KEY
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 248 |
python -m processing.pipeline --scrapers cag,gem,pib,myneta,mca
|
| 249 |
python -m graph.loader
|
| 250 |
uvicorn api.main:app --reload
|
| 251 |
```
|
| 252 |
|
| 253 |
+
Open `frontend/index.html` — no server needed.
|
| 254 |
|
| 255 |
---
|
| 256 |
|
| 257 |
## Git Workflow
|
| 258 |
|
|
|
|
|
|
|
| 259 |
```
|
| 260 |
feature/phase-N-short-name New phase development
|
| 261 |
fix/issue-N-description Bug fix
|
|
|
|
| 264 |
|
| 265 |
Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
|
| 266 |
|
| 267 |
+
Never resolve merge conflicts in the GitHub web editor. Resolve locally then push.
|
| 268 |
+
|
| 269 |
---
|
| 270 |
|
| 271 |
## Reference Documents
|
|
|
|
| 274 |
- [Security Policy](SECURITY.md)
|
| 275 |
- [Phase Roadmap](PHASE_ROADMAP.md)
|
| 276 |
- [MIT License](LICENSE)
|
| 277 |
+
- [Phase Roadmap](PHASE_ROADMAP.md)
|
| 278 |
+
- [MIT License](LICENSE)
|