DataPilot-AI-Agent / README.md
dineshb's picture
Finalize secure production deployment
dd5fa80 verified
|
Raw
History Blame Contribute Delete
14.9 kB
---
title: DataPilot AI Agent
emoji: "📊"
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: true
license: mit
short_description: Evidence-grounded autonomous data science and ML copilot
---
<!-- Hugging Face production deployment: bounded Gemini REST and server-only secrets enabled. -->
<div align="center">
<img src="https://capsule-render.vercel.app/api?type=rect&color=0:21154F,48:6045D8,100:19A98E&height=190&section=header&text=DataPilot%20AI&fontSize=48&fontColor=ffffff&animation=fadeIn&desc=Autonomous%20Data%20Science%20%C2%B7%20Evidence-Grounded%20ML%20%C2%B7%20Explainable%20Decisions&descSize=17&descAlignY=72" width="100%" alt="DataPilot AI banner">
<p>
<img src="https://readme-typing-svg.herokuapp.com/?font=Fira+Code&size=16&duration=2600&pause=850&color=6D4AFF&center=true&vCenter=true&width=780&lines=CSV+or+Parquet+%E2%86%92+Quality+Audit+%E2%86%92+Validated+Model;8+Specialized+Agents+%7C+Stateful+LangGraph+Orchestration;Leakage-Safe+Pipelines+%7C+Critic+Retry+Loop+%7C+SHAP-Ready;Streamlit+%7C+FastAPI+%7C+DuckDB+%7C+scikit-learn" alt="DataPilot AI capabilities">
</p>
[![Python](https://img.shields.io/badge/Python-3.11--3.13-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
[![LangGraph](https://img.shields.io/badge/LangGraph-Stateful_Agents-1C1C1C?style=for-the-badge)](https://langchain-ai.github.io/langgraph/)
[![Streamlit](https://img.shields.io/badge/Streamlit-Recruiter_Demo-FF4B4B?style=for-the-badge&logo=streamlit&logoColor=white)](https://datapilot-ai-agent.streamlit.app/)
[![FastAPI](https://img.shields.io/badge/FastAPI-Production_API-009688?style=for-the-badge&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
[![CI](https://github.com/dineshbarri/DataPilot-AI/actions/workflows/ci.yml/badge.svg)](https://github.com/dineshbarri/DataPilot-AI/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/License-MIT-F5C518?style=for-the-badge)](LICENSE)
**[API Docs](#fastapi)** ·
**[Architecture](docs/ARCHITECTURE.md)** ·
**[Security](docs/SECURITY.md)** ·
**[Model Governance](docs/MODEL_GOVERNANCE.md)** ·
**[Benchmarks](docs/BENCHMARKS.md)** ·
**[API Examples](docs/API_EXAMPLES.md)**
</div>
---
## Why DataPilot AI?
Most “AI data scientist” demos upload a CSV, run preprocessing on the entire dataset, compare
a few models, and ask an LLM to write an impressive-sounding summary. That is fast—but it can
leak test information, exaggerate confidence, and produce insights with no numerical provenance.
**DataPilot AI treats trust as a feature.** It coordinates a stateful agent team that audits the
data, plans the experiment, builds leakage-safe pipelines, compares models, challenges the
winner, explains predictive signals, and exports reproducible artifacts. Every displayed metric
comes from deterministic computation. The optional LLM may improve wording; it cannot create
new numbers or execute code.
### Recruiter five-minute test
1. Open the app.
2. Keep **Iris classification** selected.
3. Click **Run autonomous analysis**.
4. Inspect the model comparison, critic decision, feature importance and full agent trace.
5. Download the fitted pipeline, model card and standalone report.
No account, upload or API key is required.
---
## Product capabilities
| Stage | What DataPilot does | Evidence produced |
|---|---|---|
| Data intake | Accepts bounded CSV, TSV, Excel, JSON, Parquet or packaged demos | row/column limits and validated schema |
| Data quality | Detects missingness, duplicates, target gaps, imbalance, outliers and leakage-like fields | evidence registry with IDs, source and method |
| EDA | Uses DuckDB and pandas for compact profiles, cardinality and correlations | dataset profile and statistical summary |
| Planning | Infers classification/regression, primary metric, validation strategy and risk controls | typed Pydantic analysis plan |
| Feature engineering | Builds numeric and categorical transformers inside the model pipeline | transformation plan and fitted pipeline |
| Modeling | Selects linear, forest, extra-trees and optional XGBoost models by training-only CV | CV mean/stability and one-time untouched test results |
| Evaluation | Applies thresholds and validation-consistency checks | critic approval, rejection reasons and retry count |
| Explainability | Uses SHAP only when the optional dependency and fitted estimator are compatible; otherwise permutation importance | ranked predictive signals and caveats |
| Reporting | Produces a dashboard, evidence-backed narrative and portable artifacts | HTML report, model card, JSON, joblib pipeline |
| Follow-up | Answers questions from persisted run evidence | bounded, non-hallucinatory responses |
---
## Real agentic orchestration
The “agents” are typed LangGraph nodes combining deterministic Python computation with optional,
evidence-bounded LLM narration. The critic controls a conditional edge:
weak or unstable analysis returns to the modeling node before explanation is allowed.
```mermaid
flowchart LR
A["CSV / Parquet / Demo"] --> B["Data Quality Agent"]
B --> C["EDA Agent"]
C --> D["Statistical Agent"]
D --> E["Planning Agent"]
E --> F["Feature Engineering Agent"]
F --> G["Modeling Agent"]
G --> H{"Evaluation / Critic"}
H -->|"Reject + retry"| G
H -->|"Approve"| I["Explainability Agent"]
I --> J["Executive Insights Agent"]
J --> K["Report · Model Card · Pipeline · Evidence"]
```
The Streamlit **Agent trace** tab shows every completed node, its duration and decision.
---
## Leakage-safe ML design
```python
pipeline = Pipeline(
[
("preprocessor", ColumnTransformer(...)),
("model", candidate_model),
]
)
# Imputers, encoders and scalers learn only from training folds.
pipeline.fit(x_train, y_train)
```
- Split occurs before learned preprocessing.
- Cross-validation refits the complete pipeline in every fold.
- Classification defaults to balanced accuracy and stratification where possible.
- Candidates are selected only by training-partition CV; the selected model touches the test set once.
- Target-like names and identifier cardinality are flagged for human review.
- Predictive importance is never described as causality.
---
## Dashboard experience
The Streamlit application is summary-first and useful before any upload:
- **Built-in demos:** Iris, Breast Cancer and Diabetes Progression.
- **Executive overview:** findings, recommendations and quality risks.
- **Model laboratory:** candidate comparison, CV stability and critic gate.
- **Explainability:** interactive Plotly feature-importance view.
- **Agent trace:** visible orchestration and retry behavior.
- **Artifacts and Q&A:** downloads, evidence registry and run-specific questions.
![DataPilot production architecture](docs/architecture.svg)
The public demo URL is intentionally not claimed until a monitored deployment exists. Add a
real browser capture under `assets/` together with the deployment URL after release validation.
---
## Technology stack
| Layer | Technology |
|---|---|
| Agent orchestration | LangGraph typed state and conditional routing |
| User interface | Streamlit and Plotly |
| API | FastAPI and Pydantic |
| Analytical engine | DuckDB, pandas and NumPy |
| ML | scikit-learn; optional XGBoost and Optuna |
| Explainability | optional SHAP with permutation fallback |
| Persistence | SQLAlchemy; SQLite locally, PostgreSQL in production |
| Experiment tracking | structured agent trace; optional MLflow/OpenTelemetry |
| Artifacts | local storage interface, ready for S3/MinIO replacement |
| Delivery | Docker Compose, Render Blueprint and Streamlit Cloud |
| Quality | Pytest, Ruff, coverage, compile checks and Docker builds in GitHub Actions |
---
## Repository structure
```text
DataPilot-AI/
├── datapilot/
│ ├── config.py # typed environment configuration and limits
│ ├── data.py # safe readers, DuckDB overview and sample datasets
│ ├── quality.py # quality, leakage, imbalance, outlier and drift checks
│ ├── modeling.py # leakage-safe pipelines and model comparison
│ ├── workflow.py # LangGraph agent graph and critic loop
│ ├── insights.py # deterministic + optional evidence-bounded narrative
│ ├── reports.py # HTML report, model card, pipeline and JSON export
│ ├── persistence.py # SQL run store and artifact interface
│ ├── observability.py # optional MLflow integration
│ └── safety.py # expression-only AST security policy
├── api/main.py # versioned FastAPI application
├── worker/main.py # optional isolated calculation worker
├── tests/ # quality, safety, API and end-to-end workflow tests
├── docs/ # architecture, deployment, security and governance
├── app.py # canonical premium Streamlit implementation
├── streamlit_app.py # Streamlit Cloud compatibility shim
├── Dockerfile* # non-root UI, API and worker images
├── docker-compose.yml # constrained local multi-service stack
├── render.yaml # Render API deployment blueprint
└── .github/workflows/ci.yml # lint, tests, coverage, compile and image builds
```
---
## Quick start
### Local Streamlit demo
```bash
git clone https://github.com/dineshbarri/DataPilot-AI.git
cd DataPilot-AI
python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate
pip install -e ".[dev]"
streamlit run app.py
```
For the validated Python 3.12 reference environment, use `pip install -r requirements.lock`.
The lock snapshot is refreshed after dependency updates and tested across supported interpreters.
Open <http://localhost:8501>.
### Full AI/AutoML/observability extras
```bash
pip install -e ".[all,dev]"
```
The default installation intentionally stays deployable on modest public-demo infrastructure.
### FastAPI
```bash
uvicorn api.main:app --reload --port 8000
```
Open <http://localhost:8000/docs>.
Example:
```bash
curl -X POST http://localhost:8000/v1/analyze/sample \
-H "Content-Type: application/json" \
-d "{\"sample\":\"iris\"}"
```
The API returns `202 Accepted` with a job ID. Poll `GET /v1/jobs/{job_id}` and cancel queued
work with `DELETE /v1/jobs/{job_id}`. Set `API_KEY` in deployed environments and send it as
`X-API-Key`. The bundled in-process queue is for development; use the production topology in
`docs/ARCHITECTURE.md` for durable execution.
### Docker
```bash
docker compose up --build
```
The optional worker runs with no network, a read-only filesystem, dropped capabilities,
memory/CPU/PID limits and expression-only AST validation.
---
## Configuration
Copy `.env.example` to `.env`.
| Variable | Default | Purpose |
|---|---|---|
| `DATABASE_URL` | SQLite | Set a PostgreSQL URL for persistent production runs |
| `ARTIFACT_ROOT` | `artifacts` | Root for reports, pipelines and model cards |
| `MAX_UPLOAD_MB` | `25` | Public upload protection |
| `MAX_ROWS` | `100000` | Maximum rows per analysis |
| `MAX_COLUMNS` | `250` | Maximum feature width |
| `MAX_CATEGORIES_PER_FEATURE` | `100` | Bound categorical expansion |
| `MAX_ENCODED_FEATURES` | `5000` | Refuse unsafe estimated encoded width |
| `API_KEY` | empty | Optional API authentication; required for public deployment |
| `REQUESTS_PER_MINUTE` | `30` | Per-client API rate limit |
| `MAX_CRITIC_RETRIES` | `1` | Conditional modeling retry budget |
| `ENABLE_MLFLOW` | `false` | Enable optional experiment tracking |
| `GEMINI_API_KEY` | empty | Optional narrative refinement only |
No credential is embedded in the repository.
---
## Testing and engineering quality
```bash
pip install -e ".[dev]"
ruff check .
pytest --cov=datapilot --cov=api --cov-fail-under=75
python -m compileall datapilot api worker app.py streamlit_app.py
```
CI runs Python 3.11, 3.12 and 3.13, enforces coverage, builds the package and all three images,
audits dependencies, scans containers with Trivy, and performs secret detection.
---
## Deployment
### Streamlit Community Cloud
- Entrypoint: `streamlit_app.py`
- Python: 3.12
- Secrets: none required; `GEMINI_API_KEY` is optional
- Default recruiter path: bundled demo dataset
### Render / Railway / Fly.io
Deploy `Dockerfile.api`, attach PostgreSQL, and configure durable object storage if artifacts
must survive container replacement. See [the deployment guide](docs/DEPLOYMENT.md).
### Production recommendation
The included in-process job manager makes local requests non-blocking. For durable production
jobs, replace it with a queue and separate workers, PostgreSQL state, S3-compatible artifacts,
and short-lived sandboxed workers for any future code-execution capability.
---
## Responsible-use boundaries
DataPilot is an exploratory decision-support system, not an automatic production approval
authority. Before consequential use, complete:
- target and leakage review
- out-of-time and segment evaluation
- privacy and retention assessment
- fairness and disparate-impact evaluation
- domain and legal approval
- monitoring, rollback and retraining ownership
See [Model Governance](docs/MODEL_GOVERNANCE.md) and [Security](docs/SECURITY.md).
---
## Roadmap
- [ ] Background job queue and live progress streaming
- [ ] S3/MinIO artifact adapter with signed downloads
- [ ] Native PostgreSQL checkpoints for resumable LangGraph runs
- [ ] Optuna study dashboard and experiment comparison
- [ ] Time-series and clustering task families
- [ ] Fairness and segment-performance report
- [ ] Data-contract and schema-drift registry
- [ ] Authenticated multi-tenant workspace
---
## Creator
### Dinesh Barri
AI Engineer building agentic systems, data products, RAG applications and production-oriented
machine-learning workflows.
[![GitHub](https://img.shields.io/badge/GitHub-dineshbarri-181717?style=for-the-badge&logo=github)](https://github.com/dineshbarri)
[![LinkedIn](https://img.shields.io/badge/LinkedIn-Dinesh_Barri-0A66C2?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/dinesh-barri-7654b010b)
---
## License
Released under the [MIT License](LICENSE).
If this project helps you, please star the repository and share the live demo.