Spaces:
Build error
Build error
Commit History
-
Wire get_graph_depth() into 5 internal investigator queries (R-5, partial)
What: 5 hardcoded [*1..2] variable-length Cypher traversals -- 3 in ai/deep_investigator.py (_layer_3_patterns x2, _layer_4_timeline x1), 1 in ai/investigators/historical_investigator.py, 1 in api/routes/risk.py's ghost-company detection -- now interpolate int(get_graph_depth()) instead of a hardcoded 2, using the same f-string-into-Cypher-variable-length-path pattern ai/graph_analytics.py already established elsewhere in this codebase (Cypher does not accept *min..max as a query parameter -- this project's own BUG-1 fix already established that literal string interpolation is required here, not ). Why these 5 specifically, and why this is genuinely safe (unlike investigation_layers, deliberately NOT touched in this commit): every one of these 5 sites was hardcoded at exactly *1..2, which is also PROFILE['low']'s own graph_depth value. That means a low-tier/desktop install sees ZERO behavioral change from this commit -- it already was getting depth-2 traversal today, for everyone, reg…f064888 -
Wire get_cache_ttl() into _STATS_TTL and _VERIFY_TTL (R-5, partial)
What: two hardcoded cache-freshness constants -- api/main.py's _STATS_TTL (was 60.0) and api/dependencies.py's _VERIFY_TTL (was 30.0) -- now read config.model_selector.get_cache_ttl(), the same previously- dead function R-5 named. Pure caching-freshness/overhead trade-off in both cases: changes how OFTEN cached data is recomputed, never WHAT gets computed or returned. Why only 2 of R-5's 4 dead functions, not all: investigated all four rather than assuming R-5 was uniformly a design question or uniformly mechanical -- it's neither, it's two very different things wearing the same label. - get_batch_size(): confirmed genuinely dead. Grepped the whole codebase for any batching pattern (batch_size, BATCH_SIZE, chunked iteration) -- zero hits anywhere outside model_selector.py/ runtime_profile.py themselves. Nothing to wire it into; unchanged. - get_graph_depth() / get_investigation_layers(): found REAL existing call sites (hardcoded *1..2 Cypher patterns across deep_in…dd8fe80
-
Extend provenance badges to search results (D8)
What: search results now carry and display source_type, same as the profile page from the previous commit. Three pieces: - api/models.py: SearchResult gets a source_type field (Optional, same convention as risk_level). - api/routes/search.py: all 22 LABEL_QUERIES RETURN clauses, the fulltext-index query, and both result-construction code paths (fulltext success + label-scan fallback) now select/pass source_type through. Applied as one global, count-asserted substitution across the 22 near-identical query strings rather than 22 separate manual edits -- same reasoning as the desktop-mode chdir fix: many genuinely uniform call sites are better fixed once, centrally, when the change to each is identical. - frontend/js/components.js: ResultCard renders the badge by calling Components.ProvenanceBadge(...).outerHTML rather than duplicating its label map inline -- one source of truth for the 4 SOURCE_TYPES labels, and it inherits that component's textContent-based (not in…c91bada -
Add provenance badges to the entity profile view (D8)
What: a colored badge next to the entity-type badge on every profile page, showing graph/schema.py's SOURCE_TYPES value (OFFICIAL today; UNOFFICIAL/COMMUNITY/USER_CREATED are provenance for D7's future user-import paths). Mirrors the existing badge/risk-badge pattern exactly -- same component style (Components.ProvenanceBadge, container appended post-render), same CSS structure, reuses existing color tokens rather than inventing a new palette (OFFICIAL reuses badge--company's green, COMMUNITY reuses badge--contract's blue, USER_CREATED reuses risk-badge--MODERATE's amber, UNOFFICIAL reuses risk-badge--NONE's neutral gray). One commit, not several, because the two halves are inert alone: the backend field with no UI renders nothing, and the UI with no field has nothing to render. Real gap found and fixed, not assumed: source_type was already being read off every node (p = dict(pol['p'])) but silently dropped by the Politician and Company profile handlers' hand-curated overview dicts -…66bf6f4 -
Migrate admin-secret generation to try the OS keychain first (D6)
What: _generate_and_persist_admin_secret() in api/routes/admin.py now calls config.credentials.set_credential("ADMIN_SECRET", ...) first, falling back to config.settings.set_local_setting() (config.json) only if the keychain write fails or isn't available. _require_admin() needed zero changes -- it already reads via os.getenv("ADMIN_SECRET", ""), which transparently picks up whatever config.settings injected regardless of source. Subtlety specifically verified rather than assumed: set_local_setting() already injects its value into os.environ itself, but set_credential() (the keychain path) does not -- that's load_credentials_into_environ()'s job, which only runs once at process startup. Without an explicit os.environ["ADMIN_SECRET"] = new_secret on the keychain-success branch, a secret generated via the keychain would be correct on disk/in-keychain but invisible to the rest of that same running process until next restart. Added explicitly; confirmed by test. An existing install with …188acc4
-
fix: desktop-mode-adapt dossier output; unify 3 hardcoded copies of its path (P-1)
Root cause: the dossier/export flow had this same base path hardcoded independently in THREE places, all of which need to agree for the export endpoint to work at all: 1. ai/dossier_generator.py's generate() computed its own <project_root>/data as a path-traversal allowlist (a pre-existing H-07/CodeQL security fix) and joined output_dir onto <project_root> directly. 2. api/routes/export.py passed the literal string "data/processed" as that function's output_dir argument. 3. api/routes/export.py ALSO independently recomputed <project_root>/data/processed as its OWN separate, independent path-traversal allowlist (a second H-07/CodeQL fix) to validate the path generate() handed back before serving the file. New finding while implementing this: (1) and (3) were ALREADY subtly inconsistent with each other before any desktop-mode change -- (1) allowed anything under <project_root>/data/ (e.g. data/samples), while (3) allowed only <project_root>/data/…7650d33 -
fix: centralize + desktop-mode-adapt the logs directory (P-1)
New finding while implementing this: config/settings.py's LOGS_PATH constant was dead code -- grepped every consumer, zero hits anywhere outside its own definition. The 3 real log-writing call sites each independently hardcoded or recomputed their own path instead: - scrapers/base_scraper.py: a bare CWD-relative "logs" string - api/middleware/audit_logger.py: __file__-relative, 3 dirname() calls up to project root - blockchain/audit_chain.py: __file__-relative, 2 dirname() calls up to project root (different file depth, same target) All 3 resolve to the identical directory TODAY only because Docker's fixed WORKDIR makes CWD == project root -- a coincidence, not a guarantee. A desktop build has no such guarantee: a double-clicked app's CWD is unpredictable, so the CWD-relative scraper logs and the __file__-relative audit logs could silently end up in two different, unrelated directories the moment this ships as a desktop app. Fix: LOGS_PATH is now desktop-mode-aware (same…6d12c24
-
feat(D1): local config-file loading + desktop-mode admin-auth hardening
Phase D1 scope (PHASE_ROADMAP.md): local-config-file loading added alongside .env, and the DEBUG_MODE admin-auth bypass replaced with a desktop-appropriate default per SECURITY.md's Desktop Application Threat Model. config/settings.py: added get_app_data_dir() (OS-appropriate config dir), _load_local_config()/set_local_setting() (JSON file at <app-data>/config.json). Mirrors exactly how load_dotenv() already works -- injects values into os.environ so every existing os.getenv(...) call site in the codebase picks them up transparently, without needing to touch api/dependencies.py or any scraper. Precedence: real env var (including .env) always wins; local config only fills gaps. A missing/malformed local config file degrades to a warning, never a crash. api/routes/admin.py: _require_admin() now branches on BHARATGRAPH_DESKTOP_MODE. When unset (every current deployment), web behavior is byte-for-byte unchanged. When set, the DEBUG_MODE bypass path is skipped entirely -- a missing ADMIN_…e26a730
-
cleanup(input_validator): remove dead ALLOWED_CHARS regex
Root cause: a module-level character-allowlist regex sat directly next to the two checks that ARE live in dispatch() (MAX_QUERY_LEN, CYPHER_INJECTION), but dispatch() never references it. Confirmed via grep: zero references anywhere else in the repository either. The 'obvious' fix -- wire it into dispatch() -- was checked and rejected. The allowlist's excluded-character set rejects '&', which appears in real, well-known Indian company names this platform needs to search. Verified: 'Larsen & Toubro' and 'M&M' both fail to match. Enabling it as-is would have been a functional regression. Verified: py_compile passes.04bce6d -
fix: version-string drift (F-8) and dead ALLOWED_CHARS regex (F-9)
F-8: GET / and GET /health now read app.version instead of hand-copying the literal a third and fourth time (GET / was stale at 0.31.5). F-9: removed ALLOWED_CHARS -- unreferenced anywhere, and its character set rejects real company names like 'Larsen & Toubro' (verified directly), so wiring it in blind would have been a regression. Verified: py_compile passes on both files.a520836
-
fix(timeline route): invalid Cypher syntax '{id: }' and bare LIMIT
Root cause: GET /timeline/{entity_id} -- the entry point for the EvidencePanel timeline tab -- used '{id: }' (map literal with no value) and a bare 'LIMIT' clause with no bound value. Both are invalid Cypher. This is the most severe instance of this pattern in the codebase besides graph/loader.py (next commit): unlike every other occurrence, this route has NO exception handling at all around the session.run() call, so it returns an unhandled 500 on every single request. It has not yet been hit by real end users only because the frontend has no timeline tab to call it from yet (a separate, already-documented frontend gap) -- the moment that tab is built, every load of it will 500. Evidence: grep -rn '{[a-zA-Z_]*: *}' plus a direct read of the bare 'LIMIT' clause in context; the params dict already passes limit=limit * 3, confirming the intent was always to bind it via $limit. Verified: py_compile passes.4ccfced -
fix(forensics route): invalid Cypher syntax '{id:}' in 3 query blocks
Root cause: same malformed map-literal pattern as the risk.py commit, three more instances in api/routes/forensics.py: - GET /forensics/benfords/{entity_id} (line 136): affidavit fetch - GET /forensics/shadow-draft (line 187): submission-text fetch - GET /forensics/shadow-draft (line 191): bill-text fetch All three are wrapped in try/except Exception blocks that return {"status": "error", ...} on failure -- syntactically valid-looking JSON, no operator-visible signal that the underlying query never parsed. Evidence: grep -rn '{[a-zA-Z_]*: *}', each instance opened in context before fixing (not just pattern-matched). Verified: py_compile passes; confirmed zero remaining '{id:}' instances in this file after the fix.3f1a145 -
fix(risk route): invalid Cypher syntax + wrong field names in 3 Phase-34 blocks
Root cause: three session.run() calls in GET /risk/{entity_id} used the malformed map literal '{id:}' (no value after the colon), which is not valid Cypher and fails to parse on every call. All three sit inside a try/except Exception block, so the failure is silent -- the endpoint returns normal-looking JSON with the corresponding risk factor simply absent, with no operator-visible signal anything is wrong. Evidence: grep -rn '{[a-zA-Z_]*: *}' across the tree, each instance read in context (not just pattern-matched) before fixing: - Benford's Law block (line 176): '(p {id:})' -> '(p {id: $id})' - Ghost-company block (line 208): '(n {id:})' -> '(n {id: $id})' - Shadow-director block (line 243): '(n {id:})' -> '(n {id: $id})' Second, independent bug fixed in the same commit (same block, same root symptom -- a risk factor that can never fire): the ghost-company query fetched co.employee_count AS emp and co.registered_capital_crore AS cap, but GhostCompanyDetector.score_company() r…146a572
-
fix(search): remove dead 'pressrelease' LABEL_QUERIES branch
Every LABEL_QUERIES entry returns columns id/name/state/party except 'pressrelease', which returned entity_id/entity_type/name/source instead. The shared result loop does: if not r.get("id") or not r.get("name"): continue so this branch's rows never had a truthy 'id' key and were silently dropped, 100% of the time. 'pib' already does this job correctly with matching column names. The frontend never sent type=pressrelease (it's unreachable through the UI), so this has no observable effect on real traffic. Note for anyone hitting the API directly: filter_type lookup tries an exact key match first, so an explicit ?type=pressrelease request will now fall through the prefix-match logic, find nothing, and fall back to searching across ALL types -- rather than silently returning zero rows as it did before. That's a behavior change worth knowing about, even though it's strictly better than the old silent-drop.7fd981e
-
chore: bump version to 0.36.0 -- Phase 35 complete
Phase 35 -- Entity Intelligence Dashboard Frontend entity view: 3 tabs -> 6 tabs Before: Findings | Relationship Graph | Evidence Locker After: Findings | Relationship Graph | Timeline | Forensics | Network Analysis | Evidence Locker Frontend entity view: 3 API calls -> 6 API calls Added: Api.timeline(), Api.graphAnalytics(), Api.investigate() New page: #/forensics Network-wide ghost company + shadow director + circular ownership dashboard Nav link added to desktop and mobile navbar Home stats: 4 counters -> 6 counters Added: Audit Reports count + total Relationship count api.js: Api.investigate() method added Tests: 113 total (15 new, was 98)8549189
-
chore: bump version to 0.35.0 -- Phase 34 complete
Phase 34 -- Forensic Intelligence: Wire 8 Detection Engines 8 fully-built AI engines that were importing nothing now have API routes: ai/benfords_analyzer.py -> GET /forensics/benfords/{entity_id} + new RiskFactor in GET /risk/{entity_id} ai/circular_ownership.py -> GET /forensics/circular-ownership ai/ghost_company.py -> GET /forensics/ghost-companies + new RiskFactor in GET /risk/{entity_id} ai/shadow_director.py -> GET /forensics/shadow-directors + new RiskFactor in GET /risk/{entity_id} ai/shadow_draft_detector.py -> GET /forensics/shadow-draft ai/self_learning/*.py -> GET /self-learning/patterns|weights|audit|schema ai/case_memory/case_store.py-> GET /case-memory/stats|similar POST /case-memory/false-positive ai/explainer.py -> wired into /risk/{entity_id} explanation New routes: 3 (forensics, self_learning, case_memo…820c26f -
feat(phase-34): register forensics, self_learning, case_memory routers
Added 3 route modules to the single-line routes import (CI maintained). Registered all 3 with app.include_router after timeline.router. New endpoints now live: /forensics/* -- 5 forensic detection endpoints /self-learning/* -- 4 self-learning endpoints /case-memory/* -- 3 case memory endpoints70e41ad -
feat(phase-34): add api/routes/case_memory.py with 3 endpoints
GET /case-memory/stats -- case count, pattern breakdown, false positive rates GET /case-memory/similar -- find past cases with matching finding types POST /case-memory/false-positive -- record false positives to improve accuracy CaseStore was fully implemented in ai/case_memory/case_store.py. It was imported by zero routes and had no feedback mechanism. The false-positive endpoint closes the learning loop: users flag bad findings, WeightOptimizer picks them up and reduces the offending indicator weight on next optimize() run.538036c -
feat(phase-34): add api/routes/self_learning.py with 4 endpoints
GET /self-learning/patterns -- PatternLearner discovers new investigation motifs GET /self-learning/weights -- WeightOptimizer current investigator weights GET /self-learning/audit -- SelfAudit scraper health check (30s timeout) GET /self-learning/schema -- SchemaLearner pending fields not in schema.py All 4 modules in ai/self_learning/ were fully implemented. None were reachable via any API endpoint. Now wired.5fdaa50 -
feat(phase-34): add api/routes/forensics.py with 5 endpoints
GET /forensics/circular-ownership -- detect shell company ownership rings GET /forensics/ghost-companies -- companies with contracts but no activity GET /forensics/shadow-directors -- directors of 10+ companies / shared addresses GET /forensics/benfords/{entity_id} -- Benford Law analysis on asset declarations GET /forensics/shadow-draft -- policy text similarity to lobbying documents All 5 endpoints use pre-built detection engines in ai/ that were fully implemented but imported by zero routes. Each returns gracefully if the underlying data (affidavits, company filings) is not yet ingested.48337b7 -
feat(phase-34): wire BenfordsAnalyzer + GhostCompanyDetector + ShadowDirectorDetector into risk scorer
Three fully-built detection engines were importing nothing and calling no API routes. Now they contribute new RiskFactor entries: benfords_law_anomaly: Runs Benford Law chi-squared test on affidavit asset values. chi2 > 15.5 (p<0.05) flags fabricated or heavily rounded figures. Weight 0.20, max contribution 20 points. ghost_company_association: Scores linked companies for ghost indicators (no employees, minimal capital, high contract volume). Each ghost company adds 12 points. Weight 0.24, max contribution 24 points. high_directorship_count: Flags entities directing 10+ companies (shadow director pattern). Weight 0.15, max contribution 15 points. Also wired the explainer module for structured natural-language output.4ac80cb
-
chore: bump version to 0.34.0 -- Phase 33 complete
Phase 33 -- Graph Intelligence API New API endpoints: GET /graph/analytics/{entity_id} -- betweenness + pagerank + communities GET /graph/centrality/betweenness -- top 20 network brokers (full graph) GET /graph/centrality/pagerank -- top 20 influential entities GET /graph/communities -- institutional cluster detection GET /timeline/{entity_id} -- chronological event feed GET /timeline/{entity_id}/by-year -- events bucketed by year Frontend fix: EvidencePanel timeline tab was calling /profile/ (no events). Now calls /timeline/ which returns real chronological events. New client methods: Api.timeline(), Api.timelineByYear() Api.graphAnalytics(), Api.graphBetweenness() Api.graphPagerank(), Api.graphCommunities() Tests: 78 total (was 59). 19 new tests for GraphAnalytics + timeline helpers.1b77cde -
feat(phase-33): register timeline.router in api/main.py
Added 'timeline' to the single-line routes import (CI requirement). Registered app.include_router(timeline.router) after resolve.router. /timeline/* endpoints are now live. The EvidencePanel timeline tab will receive real events instead of always showing empty.5457f68 -
feat(phase-33): add api/routes/timeline.py with 2 endpoints
GET /timeline/{entity_id} -- all time-stamped events for an entity sorted newest-first (feeds EvidencePanel) GET /timeline/{entity_id}/by-year -- same events bucketed by year The EvidencePanel timeline tab calls /profile/{entity_id} and receives a ProfileResponse with no events field -- the timeline always shows empty. This dedicated /timeline endpoint queries all connected nodes that have any date field (order_date, date, filing_date, scraped_at, year) and returns structured events with category, title, amount, source. Categories: contract, audit, enforcement, financial, legal, political, electoral, regulatory, vigilance, corporate, other.4435778 -
feat(phase-33): add 4 graph analytics endpoints to api/routes/graph.py
GET /graph/analytics/{entity_id} -- betweenness + pagerank + communities for an entity local sub-graph GET /graph/centrality/betweenness -- top 20 brokers in full graph GET /graph/centrality/pagerank -- top 20 influential entities in full graph GET /graph/communities -- detect institutional clusters All 4 endpoints use the pre-built GraphAnalytics class (ai/graph_analytics.py) which uses NetworkX under the hood. Graceful ImportError if networkx not installed -- returns {status: error} instead of 500.5ac9ba3
-
chore: bump version to 0.33.0 -- Phase 32 complete
Phase 32 -- Entity Resolution v2: Canonical Identity Engine New endpoints: GET /resolve -- normalise + canonicalise a name GET /resolve/alias -- O(1) alias graph lookup POST /resolve/batch -- merge duplicates in a list (max 500 records) GET /resolve/stats -- alias graph statistics Infrastructure: AliasGraph singleton loaded at startup from data/processed/alias_graph.json Pipeline rebuilds alias graph after every resolve_dataset() run 15 new tests added (59 total, was 44)681bc37 -
feat(phase-32): load AliasGraph at startup + register resolve.router
AliasGraph.load() is now called in the lifespan startup block (before yield) so /resolve/alias lookups work immediately on the first request. If the alias_graph.json file does not yet exist (before first pipeline run), the failure is caught and logged as a warning -- API starts normally. app.include_router(resolve.router) is registered after runtime.router so all /resolve/* endpoints are live.02e886f -
feat(phase-32): add AliasGraph import and singleton to api/main.py
Added imports for AliasGraph and EntityResolverV2 after api.models import. Added module-level singletons _alias_graph = AliasGraph() and _resolver_v2 = EntityResolverV2(threshold=0.82) so they are created once and reused across all requests rather than per-request instantiation.d0cb2c0
-
feat(phase-32): register resolve.router in api/main.py
Added 'resolve' to the single-line routes import (CI requirement maintained). Registered app.include_router(resolve.router) after runtime.router. The /resolve/* endpoints are now served at the same base URL as all other BharatGraph API routes.e36916d -
feat(phase-32): add api/routes/resolve.py with 4 endpoints
GET /resolve?name=...&kind=person -- normalise + canonicalise a single name GET /resolve/alias?name=... -- O(1) lookup in pre-built alias graph POST /resolve/batch -- resolve list of records (max 500), merge duplicates GET /resolve/stats -- alias graph statistics The /resolve/alias and /resolve/stats endpoints share the _alias_graph singleton loaded at startup from data/processed/alias_graph.json. The /resolve/batch endpoint creates a temporary resolver for the request at the caller's threshold.d5dc03e
-
fix(M-09): confirm dirname guard in save_json; bump version to 0.32.0
M-09 was already correctly fixed in a prior sprint with: dir_path = os.path.dirname(filepath) if dir_path: os.makedirs(dir_path, exist_ok=True) The sprint 8 verification reported FAIL because the check searched for the variable name '_save_dir' but the actual fix used 'dir_path'. Both are valid fixes -- no code change needed, verification logic corrected. Version: 0.31.x -> 0.32.0e492711
-
fix(NEW-3): expand _FEED_LABELS from 6 to 14 node types
Live feed only showed AuditReport, EnforcementAction, RegulatoryOrder, ElectoralBond, Contract, VigilanceCircular. Politician updates, Company registrations, CourtCases, Tenders, NGO actions, PressReleases, InsolvencyOrders, and SanctionedEntity records never appeared in the feed even when the graph was fully populated.1a8094a -
fix(B-01): define _stats_lock -- NameError crashed GET /stats on every call
api/main.py used 'with _stats_lock:' at line 146 but the lock object was never created and threading was never imported. Every call to GET /stats raised NameError: name '_stats_lock' is not defined, completely breaking the stats endpoint and the homepage statistics panel.05ed738
-
fix(BACKEND-2): add threading.Lock for _stats_cache -- prevents concurrent full-graph scans
112882a
-
chore: bump version to 0.31.5 -- Sprint 7 complete (31 commits, 30+ bugs fixed)
fc19538 -
fix(SEARCH-2,SEARCH-3): add datagov/crimereport filter buttons; add Arabic script U+0600-U+06FF to ALLOWED_CHARS for Urdu/Sindhi search
297cc9e -
fix(SEARCH-1): call setup_schema() in API lifespan startup -- fulltext index created on fresh deployment without waiting for daily scrape
aa36a1a -
fix(BACKEND-1): move rate limiter eviction AFTER current key pruning -- prevents losing rate-limit tracking on burst-after-idle patterns
9d85e1e -
fix(LOGIC-1): remove 2 duplicate pressrelease keys in LABEL_QUERIES; fix LIMIT $lim -> $limit to match query runner kwarg
b348d13 -
fix(SEC-6): reject empty-string ADMIN_SECRET -- empty string was falsy and bypassed authentication check
4040e0f -
fix(CodeQL#16-17): use validated real_out path for FileResponse and media_type check in export.py
673a953 -
fix(CodeQL#23): log exception type only in investigation.py -- full exception message may expose Neo4j URI
7a6c1ee -
fix(FEED-3): replace deprecated asyncio.get_event_loop() with get_running_loop() in WebSocket handler
c157cb6
-
fix(SyntaxError): remove duplicate stats cache block in api/main.py -- global must precede all use of variable
b20b3a7
-
fix(H-07): add 60s cache for /stats endpoint -- prevents full MATCH(n) scan on every homepage load
5ead1fe -
fix(M-08): add pressrelease alias in LABEL_QUERIES so frontend filter button works
9b088fe
-
chore: bump version to 0.31.4 -- Sprint 5 complete (36 bugs fixed across 36 commits)
1e8b330 -
fix(H-07): add 60s cache to /stats endpoint -- prevents full graph scan (MATCH (n)) on every homepage load
903be0d -
fix(M-08): add 'pressrelease' alias key in LABEL_QUERIES -- frontend filter button now correctly filters to PressRelease nodes
2d87a38 -
fix(M-03,M-06,M-08): add mobile CSS overrides for inline grid layouts; add NONE risk level for score=0; clamp risk score to 0-100
b0e0450