DataPilot-AI-Agent / README.md
dineshb's picture
Finalize secure production deployment
dd5fa80 verified
|
Raw
History Blame Contribute Delete
14.9 kB
metadata
title: DataPilot AI Agent
emoji: 📊
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: true
license: mit
short_description: Evidence-grounded autonomous data science and ML copilot

Why DataPilot AI?

Most “AI data scientist” demos upload a CSV, run preprocessing on the entire dataset, compare a few models, and ask an LLM to write an impressive-sounding summary. That is fast—but it can leak test information, exaggerate confidence, and produce insights with no numerical provenance.

DataPilot AI treats trust as a feature. It coordinates a stateful agent team that audits the data, plans the experiment, builds leakage-safe pipelines, compares models, challenges the winner, explains predictive signals, and exports reproducible artifacts. Every displayed metric comes from deterministic computation. The optional LLM may improve wording; it cannot create new numbers or execute code.

Recruiter five-minute test

  1. Open the app.
  2. Keep Iris classification selected.
  3. Click Run autonomous analysis.
  4. Inspect the model comparison, critic decision, feature importance and full agent trace.
  5. Download the fitted pipeline, model card and standalone report.

No account, upload or API key is required.


Product capabilities

Stage What DataPilot does Evidence produced
Data intake Accepts bounded CSV, TSV, Excel, JSON, Parquet or packaged demos row/column limits and validated schema
Data quality Detects missingness, duplicates, target gaps, imbalance, outliers and leakage-like fields evidence registry with IDs, source and method
EDA Uses DuckDB and pandas for compact profiles, cardinality and correlations dataset profile and statistical summary
Planning Infers classification/regression, primary metric, validation strategy and risk controls typed Pydantic analysis plan
Feature engineering Builds numeric and categorical transformers inside the model pipeline transformation plan and fitted pipeline
Modeling Selects linear, forest, extra-trees and optional XGBoost models by training-only CV CV mean/stability and one-time untouched test results
Evaluation Applies thresholds and validation-consistency checks critic approval, rejection reasons and retry count
Explainability Uses SHAP only when the optional dependency and fitted estimator are compatible; otherwise permutation importance ranked predictive signals and caveats
Reporting Produces a dashboard, evidence-backed narrative and portable artifacts HTML report, model card, JSON, joblib pipeline
Follow-up Answers questions from persisted run evidence bounded, non-hallucinatory responses

Real agentic orchestration

The “agents” are typed LangGraph nodes combining deterministic Python computation with optional, evidence-bounded LLM narration. The critic controls a conditional edge: weak or unstable analysis returns to the modeling node before explanation is allowed.

flowchart LR
    A["CSV / Parquet / Demo"] --> B["Data Quality Agent"]
    B --> C["EDA Agent"]
    C --> D["Statistical Agent"]
    D --> E["Planning Agent"]
    E --> F["Feature Engineering Agent"]
    F --> G["Modeling Agent"]
    G --> H{"Evaluation / Critic"}
    H -->|"Reject + retry"| G
    H -->|"Approve"| I["Explainability Agent"]
    I --> J["Executive Insights Agent"]
    J --> K["Report · Model Card · Pipeline · Evidence"]

The Streamlit Agent trace tab shows every completed node, its duration and decision.


Leakage-safe ML design

pipeline = Pipeline(
    [
        ("preprocessor", ColumnTransformer(...)),
        ("model", candidate_model),
    ]
)

# Imputers, encoders and scalers learn only from training folds.
pipeline.fit(x_train, y_train)
  • Split occurs before learned preprocessing.
  • Cross-validation refits the complete pipeline in every fold.
  • Classification defaults to balanced accuracy and stratification where possible.
  • Candidates are selected only by training-partition CV; the selected model touches the test set once.
  • Target-like names and identifier cardinality are flagged for human review.
  • Predictive importance is never described as causality.

Dashboard experience

The Streamlit application is summary-first and useful before any upload:

  • Built-in demos: Iris, Breast Cancer and Diabetes Progression.
  • Executive overview: findings, recommendations and quality risks.
  • Model laboratory: candidate comparison, CV stability and critic gate.
  • Explainability: interactive Plotly feature-importance view.
  • Agent trace: visible orchestration and retry behavior.
  • Artifacts and Q&A: downloads, evidence registry and run-specific questions.

DataPilot production architecture

The public demo URL is intentionally not claimed until a monitored deployment exists. Add a real browser capture under assets/ together with the deployment URL after release validation.


Technology stack

Layer Technology
Agent orchestration LangGraph typed state and conditional routing
User interface Streamlit and Plotly
API FastAPI and Pydantic
Analytical engine DuckDB, pandas and NumPy
ML scikit-learn; optional XGBoost and Optuna
Explainability optional SHAP with permutation fallback
Persistence SQLAlchemy; SQLite locally, PostgreSQL in production
Experiment tracking structured agent trace; optional MLflow/OpenTelemetry
Artifacts local storage interface, ready for S3/MinIO replacement
Delivery Docker Compose, Render Blueprint and Streamlit Cloud
Quality Pytest, Ruff, coverage, compile checks and Docker builds in GitHub Actions

Repository structure

DataPilot-AI/
├── datapilot/
│   ├── config.py             # typed environment configuration and limits
│   ├── data.py               # safe readers, DuckDB overview and sample datasets
│   ├── quality.py            # quality, leakage, imbalance, outlier and drift checks
│   ├── modeling.py           # leakage-safe pipelines and model comparison
│   ├── workflow.py           # LangGraph agent graph and critic loop
│   ├── insights.py           # deterministic + optional evidence-bounded narrative
│   ├── reports.py            # HTML report, model card, pipeline and JSON export
│   ├── persistence.py        # SQL run store and artifact interface
│   ├── observability.py      # optional MLflow integration
│   └── safety.py             # expression-only AST security policy
├── api/main.py               # versioned FastAPI application
├── worker/main.py            # optional isolated calculation worker
├── tests/                    # quality, safety, API and end-to-end workflow tests
├── docs/                     # architecture, deployment, security and governance
├── app.py                    # canonical premium Streamlit implementation
├── streamlit_app.py          # Streamlit Cloud compatibility shim
├── Dockerfile*               # non-root UI, API and worker images
├── docker-compose.yml        # constrained local multi-service stack
├── render.yaml               # Render API deployment blueprint
└── .github/workflows/ci.yml  # lint, tests, coverage, compile and image builds

Quick start

Local Streamlit demo

git clone https://github.com/dineshbarri/DataPilot-AI.git
cd DataPilot-AI

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate

pip install -e ".[dev]"
streamlit run app.py

For the validated Python 3.12 reference environment, use pip install -r requirements.lock. The lock snapshot is refreshed after dependency updates and tested across supported interpreters.

Open http://localhost:8501.

Full AI/AutoML/observability extras

pip install -e ".[all,dev]"

The default installation intentionally stays deployable on modest public-demo infrastructure.

FastAPI

uvicorn api.main:app --reload --port 8000

Open http://localhost:8000/docs.

Example:

curl -X POST http://localhost:8000/v1/analyze/sample \
  -H "Content-Type: application/json" \
  -d "{\"sample\":\"iris\"}"

The API returns 202 Accepted with a job ID. Poll GET /v1/jobs/{job_id} and cancel queued work with DELETE /v1/jobs/{job_id}. Set API_KEY in deployed environments and send it as X-API-Key. The bundled in-process queue is for development; use the production topology in docs/ARCHITECTURE.md for durable execution.

Docker

docker compose up --build

The optional worker runs with no network, a read-only filesystem, dropped capabilities, memory/CPU/PID limits and expression-only AST validation.


Configuration

Copy .env.example to .env.

Variable Default Purpose
DATABASE_URL SQLite Set a PostgreSQL URL for persistent production runs
ARTIFACT_ROOT artifacts Root for reports, pipelines and model cards
MAX_UPLOAD_MB 25 Public upload protection
MAX_ROWS 100000 Maximum rows per analysis
MAX_COLUMNS 250 Maximum feature width
MAX_CATEGORIES_PER_FEATURE 100 Bound categorical expansion
MAX_ENCODED_FEATURES 5000 Refuse unsafe estimated encoded width
API_KEY empty Optional API authentication; required for public deployment
REQUESTS_PER_MINUTE 30 Per-client API rate limit
MAX_CRITIC_RETRIES 1 Conditional modeling retry budget
ENABLE_MLFLOW false Enable optional experiment tracking
GEMINI_API_KEY empty Optional narrative refinement only

No credential is embedded in the repository.


Testing and engineering quality

pip install -e ".[dev]"
ruff check .
pytest --cov=datapilot --cov=api --cov-fail-under=75
python -m compileall datapilot api worker app.py streamlit_app.py

CI runs Python 3.11, 3.12 and 3.13, enforces coverage, builds the package and all three images, audits dependencies, scans containers with Trivy, and performs secret detection.


Deployment

Streamlit Community Cloud

  • Entrypoint: streamlit_app.py
  • Python: 3.12
  • Secrets: none required; GEMINI_API_KEY is optional
  • Default recruiter path: bundled demo dataset

Render / Railway / Fly.io

Deploy Dockerfile.api, attach PostgreSQL, and configure durable object storage if artifacts must survive container replacement. See the deployment guide.

Production recommendation

The included in-process job manager makes local requests non-blocking. For durable production jobs, replace it with a queue and separate workers, PostgreSQL state, S3-compatible artifacts, and short-lived sandboxed workers for any future code-execution capability.


Responsible-use boundaries

DataPilot is an exploratory decision-support system, not an automatic production approval authority. Before consequential use, complete:

  • target and leakage review
  • out-of-time and segment evaluation
  • privacy and retention assessment
  • fairness and disparate-impact evaluation
  • domain and legal approval
  • monitoring, rollback and retraining ownership

See Model Governance and Security.


Roadmap

  • Background job queue and live progress streaming
  • S3/MinIO artifact adapter with signed downloads
  • Native PostgreSQL checkpoints for resumable LangGraph runs
  • Optuna study dashboard and experiment comparison
  • Time-series and clustering task families
  • Fairness and segment-performance report
  • Data-contract and schema-drift registry
  • Authenticated multi-tenant workspace

Creator

Dinesh Barri

AI Engineer building agentic systems, data products, RAG applications and production-oriented machine-learning workflows.

GitHub LinkedIn


License

Released under the MIT License.

If this project helps you, please star the repository and share the live demo.