abinazebinoy commited on
Commit
58cd023
·
unverified ·
1 Parent(s): bd91d61

Enhance README with live deployment info and updates

Browse files

Add live deployment section and update various details.

Files changed (1) hide show
  1. README.md +64 -65
README.md CHANGED
@@ -23,6 +23,17 @@ All language is analytically neutral and legally defensible.
23
 
24
  ---
25
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## What It Builds
27
 
28
  **Investigation Intelligence (Tab 1)**
@@ -58,14 +69,14 @@ GeM contracts, court filings, PIB releases, SEBI orders, and verified news.
58
  bharatgraph/
59
  scrapers/ 21 data collectors (Indian govt + international)
60
  processing/ Name normalisation, entity resolution, pipeline
61
- graph/ Neo4j schema, loader, Cypher queries
62
  ai/ Risk scoring, NLP, graph analytics, 12 investigators
63
  config/ 22 language configs, settings
64
- api/ FastAPI REST + WebSocket (6 routes)
65
  frontend/ HTML/CSS/JS dashboard, D3.js graph, dark/light theme
66
  templates/ Jinja2 dossier template (PDF/HTML)
67
  blockchain/ Audit log
68
- tests/ Tests
69
  ```
70
 
71
  ---
@@ -76,11 +87,11 @@ bharatgraph/
76
  |----------------|---------------------|------------------------------------|
77
  | Source code | GitHub | Unlimited public repos |
78
  | Backend API | Hugging Face Spaces | Docker SDK — no cold start |
79
- | Frontend | GitHub Pages | Unlimited static hosting |
80
  | Graph database | Neo4j AuraDB Free | 50K nodes, 175K relationships |
81
  | CDN + DDoS | Cloudflare Free | Unlimited caching + DDoS |
82
  | CI/CD | GitHub Actions | 2,000 min/month |
83
- | Uptime monitor | UptimeRobot Free | 50 monitors, 5-min intervals |
84
 
85
  ---
86
 
@@ -131,66 +142,51 @@ for cross-source entity resolution. Full pipeline orchestrator.
131
  ### Phase 3 — Graph Database
132
  Neo4j schema: 7 node types, 6 relationship types, 10 constraint queries.
133
  `MERGE` with stable MD5 IDs. 8 pre-built Cypher queries.
134
- Connected to Neo4j AuraDB Free: `neo4j+s://1a34e3b8.databases.neo4j.io`
135
 
136
  ### Phase 4 — FastAPI Backend
137
  FastAPI with lifespan, CORS, 6 route modules registered after `app = FastAPI()`,
138
- WebSocket feed, Pydantic models. Version 0.12.0.
139
 
140
  ### Phase 5 — Risk Scoring Engine
141
  Composite 0–100 score. Weights: politician_company_overlap (0.35),
142
  contract_concentration (0.25), audit_frequency (0.20), asset_anomaly (0.15),
143
- criminal_case (0.05). `validate_language()` enforces neutral analytical output.
144
 
145
- ### Phase 6 — Expanded Data Sources
146
- 13 scrapers total. ICIJ HTML mode (BeautifulSoup). Wikidata SPARQL confirmed.
147
- All public HuggingFace models — no gating.
148
 
149
  ### Phase 7 — NLP Document Intelligence
150
  spaCy `en_core_web_sm` NER. Benford's Law chi-squared test on affidavit assets.
151
- Multilingual NER: `Davlan/bert-base-multilingual-cased-ner-hrl`.
152
  Shadow draft detector: `all-MiniLM-L6-v2`, 93.35% alignment confirmed.
153
 
154
  ### Phase 8 — Advanced Graph Analytics
155
  NetworkX: betweenness centrality, PageRank, Louvain community detection.
156
- Circular ownership `simple_cycles()` 3-node cycle confirmed.
157
- Ghost company scorer: GHOST_THRESHOLD=60 — Quick Win Pvt Ltd: 100/100 confirmed.
158
 
159
- ### Phase 9 — Eight New Indian Sources (21 Total)
160
  NJDG (39 live records confirmed), ED, CVC, NCRB, LGD, IBBI, NGO Darpan, CPPP.
161
- All have sample fallbacks when live source unavailable.
162
 
163
  ### Phase 10 — Multi-Investigator AI Engine
164
- 12 parallel investigators in `ThreadPoolExecutor`. SHA-256 report hash: stable,
165
- unique, 64 chars. Synthesis: 3+ investigators agreeing = HIGH confidence.
166
- `FakeSession` for offline testing. `validate_language()` on all output.
167
 
168
  ### Phase 11 — Multilingual Platform (22 Languages)
169
- All 22 Indian scheduled languages. Unicode script detection confirmed
170
- (Devanagari→hi, Tamil→ta). Helsinki-NLP/opus-mt-en-hi (public, no gating).
171
- Modi→मोदी/மோடி/మోదీ/ಮೋದಿ/മോദി cross-script search confirmed.
172
- Risk levels pre-translated in 9 languages.
173
 
174
  ### Phase 12 — PDF Dossier Generator
175
- SHA-256 integrity hash per report. Verified tamper detection working.
176
- Jinja2 + WeasyPrint (HTML fallback on Windows, full PDF on Linux).
177
- Template: Indian tricolour design, 8 sections. 10,829 chars rendered confirmed.
178
- `GET /export/pdf/{entity_id}` · `GET /verify/{hash}`
179
 
180
  ### Phase 13 — Production Frontend
181
  HTML/CSS/JS — no React, no Node.js, no build step. Works from `file://`.
 
182
 
183
- - `design-system.css`: CSS custom properties token layer, dark + light themes
184
- - `components.css`: 25+ component classes
185
- - `router.js`: hash-based client-side routing with named params
186
- - `api.js`: typed API client for all FastAPI endpoints
187
- - `components.js`: HTMLElement factory functions (React-component pattern)
188
- - `graph.js`: D3.js force-directed knowledge graph
189
- - `app.js`: state management, 5 views (home/search/entity/feed/about)
190
- - `index.html`: semantic HTML5 shell
191
-
192
- Colours: Saffron #FF9933 · India Green #138808 · Ashoka Blue #000080 · Navy #0A0F2E.
193
- Dark/light theme toggle with localStorage persistence.
194
 
195
  ---
196
 
@@ -198,14 +194,24 @@ Dark/light theme toggle with localStorage persistence.
198
 
199
  | Phase | Title | Key Capability |
200
  |-------|-------|----------------|
201
- | 14 | Zero Cold-Start Deployment | HF Spaces + Cloudflare + Service Worker |
202
- | 15 | Mathematical Intelligence | Path signatures, Fourier, persistent homology |
203
- | 16 | Evidence Connection Map | 6-layer deep investigation, clickable graph |
204
- | 17 | Security Hardening | Rate limiting, audit chain, CSP headers |
205
- | 18 | Self-Learning System | Schema adaptation, pattern discovery |
206
- | 19 | Affidavit Trajectory | Kalman filter unexplained wealth detection |
207
- | 20 | Biography Engine | Complete life timeline from all 28 sources |
208
- | 21+ | Advanced Forensics | Benami, Procurement DNA, Revolving Door |
 
 
 
 
 
 
 
 
 
 
209
 
210
  Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
211
 
@@ -217,12 +223,14 @@ Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
217
  DataGov API 3,199 real MGNREGA records
218
  CAG 30 audit report links from cag.gov.in
219
  PIB 27 press releases from pib.gov.in
220
- Wikidata Modi (Q1058), Gandhi (Q10218) confirmed
221
  NJDG 39 court records confirmed live
222
- Pipeline 47 records processed in 15 seconds
223
  SHA-256 Stable, unique, 64 chars — confirmed
224
  Language det. Devanagari→hi, Tamil→ta, Latin→en — confirmed
225
- Transliteration Modi→5 scripts confirmed
 
 
226
  ```
227
 
228
  ---
@@ -233,34 +241,21 @@ Transliteration Modi→5 scripts — confirmed
233
  git clone https://github.com/abinaze/bharatgraph.git
234
  cd bharatgraph
235
  python -m venv venv
236
- source venv/Scripts/activate # Windows Git Bash
237
  pip install -r requirements.txt
238
  cp .env.example .env
239
- ```
240
-
241
- Edit `.env`:
242
- ```
243
- DATAGOV_API_KEY=register_free_at_data.gov.in_user_register
244
- NEO4J_URI=neo4j+s://your-instance.databases.neo4j.io
245
- NEO4J_USER=neo4j
246
- NEO4J_PASSWORD=your_password
247
- ```
248
-
249
- Run:
250
- ```bash
251
  python -m processing.pipeline --scrapers cag,gem,pib,myneta,mca
252
  python -m graph.loader
253
  uvicorn api.main:app --reload
254
  ```
255
 
256
- Open `frontend/index.html` in your browser — no server needed.
257
 
258
  ---
259
 
260
  ## Git Workflow
261
 
262
- One long-lived branch: `main`. All branches merge directly into `main`.
263
-
264
  ```
265
  feature/phase-N-short-name New phase development
266
  fix/issue-N-description Bug fix
@@ -269,6 +264,8 @@ docs/description Documentation only
269
 
270
  Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
271
 
 
 
272
  ---
273
 
274
  ## Reference Documents
@@ -277,3 +274,5 @@ Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
277
  - [Security Policy](SECURITY.md)
278
  - [Phase Roadmap](PHASE_ROADMAP.md)
279
  - [MIT License](LICENSE)
 
 
 
23
 
24
  ---
25
 
26
+ ## Live Deployment
27
+
28
+ | Layer | URL |
29
+ |-------|-----|
30
+ | Frontend | https://abinaze.github.io/bharatgraph |
31
+ | API | https://abinazebinoy-bharatgraph.hf.space |
32
+ | Health | https://abinazebinoy-bharatgraph.hf.space/health |
33
+ | API Docs | https://abinazebinoy-bharatgraph.hf.space/docs |
34
+
35
+ ---
36
+
37
  ## What It Builds
38
 
39
  **Investigation Intelligence (Tab 1)**
 
69
  bharatgraph/
70
  scrapers/ 21 data collectors (Indian govt + international)
71
  processing/ Name normalisation, entity resolution, pipeline
72
+ graph/ Neo4j schema, loader, Cypher queries, seed data
73
  ai/ Risk scoring, NLP, graph analytics, 12 investigators
74
  config/ 22 language configs, settings
75
+ api/ FastAPI REST + WebSocket (6 routes + admin)
76
  frontend/ HTML/CSS/JS dashboard, D3.js graph, dark/light theme
77
  templates/ Jinja2 dossier template (PDF/HTML)
78
  blockchain/ Audit log
79
+ .github/ CI/CD workflows (test, deploy, daily scrape)
80
  ```
81
 
82
  ---
 
87
  |----------------|---------------------|------------------------------------|
88
  | Source code | GitHub | Unlimited public repos |
89
  | Backend API | Hugging Face Spaces | Docker SDK — no cold start |
90
+ | Frontend | GitHub Pages | Static HTML Actions workflow |
91
  | Graph database | Neo4j AuraDB Free | 50K nodes, 175K relationships |
92
  | CDN + DDoS | Cloudflare Free | Unlimited caching + DDoS |
93
  | CI/CD | GitHub Actions | 2,000 min/month |
94
+ | Uptime monitor | UptimeRobot Free | 5-min interval health checks |
95
 
96
  ---
97
 
 
142
  ### Phase 3 — Graph Database
143
  Neo4j schema: 7 node types, 6 relationship types, 10 constraint queries.
144
  `MERGE` with stable MD5 IDs. 8 pre-built Cypher queries.
 
145
 
146
  ### Phase 4 — FastAPI Backend
147
  FastAPI with lifespan, CORS, 6 route modules registered after `app = FastAPI()`,
148
+ WebSocket feed. Version 0.12.0.
149
 
150
  ### Phase 5 — Risk Scoring Engine
151
  Composite 0–100 score. Weights: politician_company_overlap (0.35),
152
  contract_concentration (0.25), audit_frequency (0.20), asset_anomaly (0.15),
153
+ criminal_case (0.05). `validate_language()` enforces neutral output.
154
 
155
+ ### Phase 6 — Expanded Data Sources (13 scrapers)
156
+ ICIJ HTML mode (BeautifulSoup). Wikidata SPARQL confirmed. Public HF models only.
 
157
 
158
  ### Phase 7 — NLP Document Intelligence
159
  spaCy `en_core_web_sm` NER. Benford's Law chi-squared test on affidavit assets.
 
160
  Shadow draft detector: `all-MiniLM-L6-v2`, 93.35% alignment confirmed.
161
 
162
  ### Phase 8 — Advanced Graph Analytics
163
  NetworkX: betweenness centrality, PageRank, Louvain community detection.
164
+ Circular ownership `simple_cycles()`. Ghost company scorer: 100/100 confirmed.
 
165
 
166
+ ### Phase 9 — Eight New Indian Sources (21 Total Scrapers)
167
  NJDG (39 live records confirmed), ED, CVC, NCRB, LGD, IBBI, NGO Darpan, CPPP.
 
168
 
169
  ### Phase 10 — Multi-Investigator AI Engine
170
+ 12 parallel investigators. SHA-256 report hash: stable, unique, 64 chars.
171
+ Synthesis: 3+ investigators agreeing = HIGH confidence. `FakeSession` for offline testing.
 
172
 
173
  ### Phase 11 — Multilingual Platform (22 Languages)
174
+ All 22 Indian scheduled languages. Unicode detection confirmed.
175
+ Helsinki-NLP public model. Modi→मोदी/தமிழ்/తెలుగ��/ಕನ್ನಡ/മലയാളം confirmed.
 
 
176
 
177
  ### Phase 12 — PDF Dossier Generator
178
+ SHA-256 integrity hash. Jinja2 + WeasyPrint (HTML fallback on Windows).
179
+ Indian tricolour design, 8 sections. 10,829 chars rendered confirmed.
 
 
180
 
181
  ### Phase 13 — Production Frontend
182
  HTML/CSS/JS — no React, no Node.js, no build step. Works from `file://`.
183
+ D3.js force-directed graph. Dark/light theme. 5 views: home/search/entity/feed/about.
184
 
185
+ ### Phase 14 Zero Cold-Start Deployment LIVE
186
+ Hugging Face Spaces Docker SDK. No cold start on public spaces.
187
+ Service worker cache-first for static assets. GZipMiddleware (60-80% compression).
188
+ `POST /admin/seed` endpoint loads 10 politicians, 5 companies, 3 contracts.
189
+ Frontend on GitHub Pages via GitHub Actions static workflow.
 
 
 
 
 
 
190
 
191
  ---
192
 
 
194
 
195
  | Phase | Title | Key Capability |
196
  |-------|-------|----------------|
197
+ | **15** | **Mathematical Intelligence Engine** | Spectral graph analysis, Fourier timeline, path signatures, persistent homology, expanded Benford, mutual information + causal ranking |
198
+ | **16** | **Evidence Connection Map + Deep Investigation** | Investigation connector map with relationship labels/strength/source, 6-layer recursive engine, clickable evidence panel, hidden-link suggestions, recursive find-more button |
199
+ | **17** | **Security Hardening + Provenance** | Rate limiting, CSP/HSTS headers, SHA-256 audit chain, full provenance layer on every node/edge, deterministic artifact IDs |
200
+ | **18** | **Self-Learning + Case Memory** | Schema adaptation, pattern candidate discovery, case memory library, human-in-the-loop merge queue, investigation replay |
201
+ | **19** | **Affidavit Wealth Trajectory** | Kalman filter across 5 election cycles, unexplained wealth residual scoring, asset disappearance detection |
202
+ | **20** | **Biography Engine** | Complete chronological life timeline from all 28 sources, temporal convergence detection, narrative generation |
203
+ | **21** | **Benami Detection** | Graph embeddings, proxy network analysis, family name matching, director age anomaly |
204
+ | **22** | **Procurement DNA + Cartel Detection** | Bid document fingerprinting, price ratio analysis, cover bid detection, vendor co-bidding network |
205
+ | **23** | **Revolving Door + TBML Detection** | Career graph, cooling-off violations, trade-based money laundering flags |
206
+ | **24** | **Linguistic Fingerprinting** | Burrows' Delta authorship attribution across govt documents, template reuse detection |
207
+ | **25** | **Policy-Benefit Causal Analysis** | Granger causality, transfer entropy, Cumulative Abnormal Contract Award |
208
+ | **26** | **Adversarial Counterevidence + Competing Hypotheses** | Forced disproof search, hypothesis A/B/C mode, evidence scorecard |
209
+ | **27** | **Multi-Agent Debate Engine** | iMAD hesitation detection, 3-round structured debate, anti-drift, adaptive routing, consensus + dissent tracking |
210
+ | **28** | **Dark Pattern Detection** | PrefixSpan sequential pattern mining, 6 pre-defined high-risk patterns |
211
+ | **29** | **Source-Drift + Historical Analysis** | Wayback Machine credibility, archive-gap recovery, temporal slice viewer, claim survival scoring |
212
+ | **30** | **Predictive Risk + Auto-Prioritisation** | ARIMA + Random Forest trajectory, 6-month forecast, lead priority scores |
213
+ | **31** | **Geospatial Verification** | Sentinel-2 satellite imagery, NDVI change detection, build-completion mismatch |
214
+ | **32** | **Identity Fusion + Multimedia OSINT** | Alias-to-email linking, credential reuse detection, persona clustering, background-object analysis |
215
 
216
  Full detail in [Phase Roadmap](PHASE_ROADMAP.md).
217
 
 
223
  DataGov API 3,199 real MGNREGA records
224
  CAG 30 audit report links from cag.gov.in
225
  PIB 27 press releases from pib.gov.in
226
+ Wikidata Modi (Q1058), Gandhi (Q10218) confirmed live
227
  NJDG 39 court records confirmed live
228
+ Pipeline 28 nodes loaded 2 politicians, 2 companies, 20 audits, 4 press
229
  SHA-256 Stable, unique, 64 chars — confirmed
230
  Language det. Devanagari→hi, Tamil→ta, Latin→en — confirmed
231
+ Search Working on production HF Space
232
+ Frontend Live at abinaze.github.io/bharatgraph
233
+ API Live at abinazebinoy-bharatgraph.hf.space
234
  ```
235
 
236
  ---
 
241
  git clone https://github.com/abinaze/bharatgraph.git
242
  cd bharatgraph
243
  python -m venv venv
244
+ source venv/Scripts/activate
245
  pip install -r requirements.txt
246
  cp .env.example .env
247
+ # Edit .env with your Neo4j URI, user, password and DATAGOV_API_KEY
 
 
 
 
 
 
 
 
 
 
 
248
  python -m processing.pipeline --scrapers cag,gem,pib,myneta,mca
249
  python -m graph.loader
250
  uvicorn api.main:app --reload
251
  ```
252
 
253
+ Open `frontend/index.html` — no server needed.
254
 
255
  ---
256
 
257
  ## Git Workflow
258
 
 
 
259
  ```
260
  feature/phase-N-short-name New phase development
261
  fix/issue-N-description Bug fix
 
264
 
265
  Commit prefixes: `feat`, `fix`, `docs`, `test`, `chore`, `refactor`.
266
 
267
+ Never resolve merge conflicts in the GitHub web editor. Resolve locally then push.
268
+
269
  ---
270
 
271
  ## Reference Documents
 
274
  - [Security Policy](SECURITY.md)
275
  - [Phase Roadmap](PHASE_ROADMAP.md)
276
  - [MIT License](LICENSE)
277
+ - [Phase Roadmap](PHASE_ROADMAP.md)
278
+ - [MIT License](LICENSE)