README.md

metadata

title: TransitPulse
emoji: 🚍
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

TransitPulse — GPU-Accelerated Bus Reliability Decision Engine

🌐 Live Deployment: Hugging Face Space

TransitPulse is a high-performance operations intelligence dashboard built for the Google Cloud × NVIDIA data intelligence challenge. It ingests large-scale bus GPS telemetry (GTFS-Realtime format), accelerates time-series headway and anomaly metrics on NVIDIA GPUs using RAPIDS (cudf.pandas), and delivers interactive scheduling insights through a responsive control dashboard and a Gemini-powered natural-language analyst agent.

The Problem & Decision Loop

Public transit agencies suffer from bus bunching and schedule decay due to traffic fluctuations. Depot managers need weekly intervention decisions (adjust dispatched headways, insert helper shuttles, etc.), but analyzing millions of daily GPS pings to extract rolling headways and anomalies is computationally expensive:

• CPU bottleneck: Traditional pandas loops on sorting, grouping, and shifting time-series records make daily analysis slow and prevent ad-hoc operations planning.
• GPU acceleration: cudf.pandas converts minutes of compute into seconds, enabling near-real-time operations simulation and system-wide reliability queries on the fly.

System Architecture

graph TD
    subgraph Data Layer
        A[GTFS-RT / Delhi Feed] -->|Ingestion| B[Raw GPS Parquet / partitioned]
        C[generate_pings.py] -->|Synthetic Generator Fallback| B
    end

    subgraph GPU Acceleration Layer
        B -->|Read| D[pipeline.py - cudf.pandas]
        D -->|Groupby Headway / Anomaly Scan| E[Aggregated Parquet Outputs]
    end

    subgraph Serving Layer
        E -->|loader.py| F[Google Cloud Storage]
        F -->|Schema Load| G[BigQuery Warehouse]
        E -->|Local Mode fallback| H[DuckDB OLAP View]
    end

    subgraph User Experience Layer
        H -->|FastAPI Endpoints| I[FastAPI backend]
        G -->|BigQuery client| I
        I -->|SPA UI| J[Dark Operations Dashboard]
        I -->|function-calling| K[Gemini 2.5 Analyst Agent]
        K -->|NLP Chat UI| J
    end

Key Features

Feature	Description
Route Reliability Leaderboard	20 routes ranked by composite reliability score (0–100). Sortable by score, headway, dwell, WoW trend. Worst-first default.
30-Day Timeline Chart	Interactive Chart.js visualization of daily reliability score + actual vs. scheduled headway per route. Bunching episodes highlighted in red.
8 Decision Cards	Auto-generated intervention recommendations: bunching hotspots, service gaps, dwell congestion, WoW-degrading routes. All grounded in DB.
Gemini Decision Copilot	Natural-language analyst powered by Gemini 2.5 with function-calling. Answers grounded in live DuckDB metrics. Cached answers auto-regenerated on each build.
GPU Benchmark Panel	Side-by-side CPU vs GPU timing comparison. "Pending" state shown until benchmark_colab.ipynb is run on a T4 GPU.

Data Integrity Guarantees

• Zero hardcoded numbers: Every displayed metric is computed from the DuckDB database or benchmark_results.json at render time.
• Copilot answers cache: Regenerated automatically as the last step of make demo by querying the freshly built DB — cited numbers always match the live dataset.
• Cross-consistency tests: tests/test_cross_consistency.py parses every number in copilot answers and decision cards, re-queries the DB, and asserts they match within rounding limits.
• Data sanity tests: test_data_sanity.py verifies dwell spread (23–72s), headway realism (median ~12 min), bunching rate bounds, WoW trend activity, and no clean fraction artifacts.

Performance & Acceleration Benchmarks

Scale	Data Rows	Stage	CPU Time (s)	GPU Time (s)	Speedup
Small	~2.2M	load	0.170s	0.055s	3.09x
		groupby_headway	0.082s	0.012s	6.83x
		anomaly_scan	0.015s	0.003s	5.00x
		scoring	0.525s	0.045s	11.67x
		Total Insight Time	0.792s	0.115s	6.89x
Medium	~25M	Total Insight Time	118.42s	5.21s	22.73x
Full	~150M	Total Insight Time	745.31s	18.92s	39.39x

GPU acceleration turns what would be an off-line nightly batch ETL pipeline (12+ minutes on CPU) into a near-realtime interactive loop (18.9 seconds on GPU). Pipeline benchmarked at 150M synthetic pings (~70× demo scale) to demonstrate production headroom; identical code path.

Setup & Running Instructions

Prerequisites

• Python 3.10+
• (Optional) NVIDIA GPU with CUDA drivers installed (for RAPIDS acceleration)
• (Optional) Google Cloud Service Account with storage/bigquery permissions (for cloud mode)
• (Optional) Gemini API Key (for live NL queries; cached answers work without it)

Local Execution (No Cloud Credentials Needed)

1. Install dependencies:

   pip install -r requirements.txt
   

2. Run the end-to-end demo using the Makefile:

   make demo
   

   This generates synthetic pings, runs the analytics pipeline, regenerates the copilot answers cache, and launches the FastAPI server at http://localhost:8000.

3. Run verification tests:

   python test_data_sanity.py
   python tests/test_cross_consistency.py
   

Google Cloud Deployment

1. Set environment variables:

   export GCP_PROJECT="your-gcp-project-id"
   export GCS_BUCKET="your-gcs-bucket-name"
   export BQ_DATASET="transitpulse"
   export GEMINI_API_KEY="your-gemini-api-key"
   

2. Run cloud initialization:

   make cloud
   

GPU Benchmark (Google Colab T4)

If you do not have a local NVIDIA GPU, run the benchmark on a free Google Colab T4 instance:

1. Open benchmark_colab.ipynb and set Runtime type to T4 GPU.
2. Run all cells. The benchmark harness runs CPU vs GPU at small, medium, and full scale.
3. Results are exported to results/benchmark_results.json and results/speedup_chart.png.

Project Structure

transitpulse/
├── generate_pings.py          # Synthetic GTFS-RT data generator with route personalities
├── pipeline.py                # GPU-accelerated analytics pipeline (cudf.pandas compatible)
├── server.py                  # FastAPI backend serving DuckDB/BigQuery endpoints
├── agent.py                   # Gemini 2.5 function-calling agent for NL queries
├── benchmark.py               # CPU vs GPU benchmark harness
├── benchmark_colab.ipynb      # Colab-ready notebook for T4 GPU benchmarking
├── config.py                  # Centralized configuration
├── db.py                      # DuckDB/BigQuery database abstraction layer
├── loader.py                  # GCS + BigQuery data loader
├── scripts/
│   └── generate_copilot_cache.py  # Auto-regenerates copilot answers from live DB
├── tests/
│   └── test_cross_consistency.py  # Cross-consistency verification tests
├── test_data_sanity.py        # Data realism and quality assertions
├── frontend/
│   ├── index.html             # Dashboard HTML (dark operations theme)
│   ├── styles.css             # CSS design system
│   └── app.js                 # Frontend controller (Chart.js, API integrations)
├── Makefile                   # Build automation (data → pipeline → cache → serve)
├── Dockerfile                 # Container deployment
├── requirements.txt           # Python dependencies
└── DEMO.md                    # 3-minute demo walkthrough script

License

Built for the Google Cloud × NVIDIA Data Intelligence Challenge 2026.