| # DS Tools Expansion — Visualization & Modeling (Proposal) |
|
|
| **Status:** PROPOSAL — for team/mentor review; §6 items need Harry (Go/dedorch) + FE coordination. |
| **Date:** 2026-07-07 · **Branch context:** `pr/12`. |
| **Companions:** [REPO_STATUS.md](REPO_STATUS.md) (current built state) · [DEV_PLAN.md](DEV_PLAN.md) |
| (§4 #26/#27 deferred charts/images — this doc un-defers #26 with a concrete design) · |
| [API_CONTRACT_BE_PYTHON.md](API_CONTRACT_BE_PYTHON.md) (contract that §6 extends). |
| **Research basis:** industry/lit review 2026-07-07 — sources in §9. |
|
|
| --- |
|
|
| ## 0. Executive summary |
|
|
| **The ask.** Data Eyond answers analytical questions with text + markdown tables. The product story |
| ("junior data scientist that hands back a decision-ready deliverable", CRISP-DM) requires two |
| capabilities the chat surface can't express today: **charts** and **models** (forecast / clusters / |
| anomalies / drivers). This doc proposes what to add, on what stack, and in what order. |
|
|
| **The one architectural decision that matters.** DS agents in the market split into two families: |
|
|
| 1. **Sandboxed code generation** — the LLM writes Python, an isolated sandbox executes it, charts |
| come back as PNG files (ChatGPT Advanced Data Analysis, Julius, LangChain deep-agents; infra = |
| E2B/Modal/Firecracker microVMs). Maximum flexibility; weakest governance — per Gartner's 2026 |
| agentic-analytics taxonomy this is "Level 1: output varies per run, no governance." |
| 2. **Declarative specs over governed tools** — the LLM (or deterministic code) emits a validated |
| JSON *specification*; deterministic code executes it against governed data (Databricks Genie, |
| Snowflake Cortex Analyst, ThoughtSpot Spotter; research systems LIDA and chat2plot). Bounded |
| flexibility; repeatable, auditable, no arbitrary code execution. |
|
|
| **Recommendation: family 2.** It is literally our existing architecture — the query engine already |
| does *constrained spec (QueryIR) → validator → deterministic compiler → guarded executor*. |
| Visualization and modeling should extend that spine, not bolt a code sandbox onto it. Sandboxed |
| codegen is re-evaluated only when curated tools hit an expressiveness ceiling (§7, M2). |
|
|
| **Feature set (phased):** |
|
|
| | Phase | Feature | User sees | New LLM calls | |
| |---|---|---|---| |
| | **V1** | Deterministic charts from existing `analyze_*` results (trend→line, aggregate→bar, correlation→heatmap, …) | Interactive chart under the answer, chart in traceability, chart in report | 0 | |
| | **V2** | Chart-aware planning: `render_chart` tool + LLM-picked `ChartSpec` for "plot X vs Y" asks and chart-edit turns | Charts on demand, editable ("make it a pie") | 0–1 (structured output) | |
| | **M1** | Modeling tools: `analyze_forecast`, `analyze_cluster`, `analyze_anomaly`, `analyze_driver` | Predictions with confidence bands, segments, outliers, ranked drivers — each with metrics, caveats, and a chart | 0 (planner already budgeted) | |
| | **M2** | *(deferred)* Hosted code sandbox for the long tail | Arbitrary analyses | n/a — decision gate in §7 | |
|
|
| **Stack (and the one-line why — full rationale §4):** |
|
|
| | Concern | Pick | Why | |
| |---|---|---| |
| | Chart artifact format | **Plotly Figure JSON**, compiled from a small pydantic `ChartSpec` | Team already decided Plotly-JSON (DEV_PLAN #26); `plotly==5.24.1` + `kaleido==0.2.1` already pinned in `pyproject.toml`; `react-plotly.js` fits the React/Vite FE; kaleido gives server-side PNG for the deferred PPT/PDF report export | |
| | Chart storage/delivery | Python-owned **`message_charts`** JSONB table + **`GET /api/v1/charts`** — the `message_traceability` pattern reused verbatim | SSE stays text-only (house rule); FE fetches artifacts on `done`, exactly like traceability today | |
| | Forecasting | **statsmodels** (ETS / SARIMAX, `seasonal_decompose`) | The standard agent-tool library for TS; interpretable, CPU-cheap, no new heavy deps beyond itself | |
| | Clustering / anomaly / drivers | **scikit-learn** (KMeans+silhouette, IsolationForest, regularized linear/logistic + permutation importance) | Already in the venv transitively (sentence-transformers) — pin it explicitly; interpretable models only | |
| | Explicitly NOT now | code sandbox (E2B/Modal), AutoGluon/FLAML AutoML, prophet, deep-learning TS, Vega-Lite | §4.3 rejected-alternatives table | |
|
|
| **Infrastructure delta (the "other than the chatbot interface" part):** one new Python-owned dedorch |
| table (DDL handoff to Harry), one new GET endpoint (contract addition), an FE chart renderer |
| (react-plotly.js) + artifact fetch on `done`, 2 new Python deps (`statsmodels`, explicit |
| `scikit-learn`), and a `chart` output kind in the tool contract (coordinate: `src/tools/contracts.py` |
| is tool-team-owned). **No sandbox service, no GPU, no new datastore, no change to the SSE stream.** |
|
|
| **Effort (rough):** V1 ≈ 4–5 dev-days Python + 2–3 FE · V2 ≈ 3 · M1 ≈ 6–8. V1 is demo-visible fastest. |
|
|
| --- |
|
|
| ## 1. How the field does it (what the research says) |
|
|
| ### 1.1 Three architecture families |
|
|
| **A. Sandboxed code interpreters** (ChatGPT ADA, Claude analysis tool, Julius, LangChain |
| deep-agents reference). LLM writes pandas/matplotlib code; a sandbox (E2B, Modal, Daytona, |
| LangSmith Sandbox — Firecracker microVM isolation) executes it; PNGs/files come back. |
| *Strengths:* unbounded expressiveness — any analysis pandas can do. |
| *Weaknesses:* non-repeatable ("output varies per run"), un-auditable code paths, prompt-injection → |
| code-execution risk, real infra (microVMs, warm pools, credential isolation — LangChain's own docs: |
| "avoid adding credentials to the sandbox"), and results that bypass any governance layer. The 2026 |
| Gartner-derived maturity taxonomy places these at **Level 1** precisely because of governance. |
|
|
| **B. Declarative specs over governed data** (Databricks Genie, Snowflake Cortex Analyst, |
| ThoughtSpot Spotter, Amazon Q in QuickSight — **Level 2/3**). The LLM's only job is to emit a |
| constrained artifact (SQL against a semantic layer, or a chart/analysis spec); execution is |
| deterministic platform code. Research systems converge here for viz: **LIDA** (Microsoft) runs a |
| staged pipeline — data *Summarizer* → *Goal Explorer* → *VisGenerator* with generate-validate-repair; |
| **chat2plot** generates *"declarative visualization specs in JSON rather than Python code"* for |
| *"more secure execution, as the LLM does not directly generate code"*, validated by |
| structured-output/function-calling, rendered by plotly or altair. |
| *Strengths:* repeatable, auditable, cheap, safe; specs are storable/editable/versionable artifacts. |
| *Weaknesses:* bounded expressiveness — you can only draw/fit what the spec grammar covers. |
|
|
| **C. Investigative/agentic analytics** (Tellius, Qlik Predict — **Level 3/4**): multi-step |
| decomposition of metric changes with quantified attribution ("why did revenue dip?"), ML under the |
| hood (segment comparison, variance decomposition), narrative output. Architecturally these are |
| family B + a planner + statistical tooling — *not* codegen. |
|
|
| ### 1.2 Where Data Eyond already sits |
|
|
| The repo is a family-B system with a family-C planner: |
| `data_catalog` = the semantic/governance layer · QueryIR + `IRValidator` + SqlCompiler + read-only |
| executor = the constrained-spec pipeline · Planner→TaskRunner→Assembler = the investigative loop · |
| `report_inputs` → versioned reports = the audit trail. The 2026 platform comparison's core critique — |
| Level-1 tools "sacrifice audit trail, role-controlled, repeatable execution" — is exactly the |
| trade-off we already refused when we built IR validation instead of LLM-SQL. Viz and ML should |
| follow the same refusal. |
|
|
| ### 1.3 ML in analyst agents specifically |
|
|
| Published agent systems for business TS/ML (sktime LLM workflows, TimeCopilot, DCATS) wrap |
| **statsmodels / scikit-learn estimators as tools** with the LLM planning which tool to call — |
| not writing model code. Interpretability drives library choice: ETS/ARIMA with confidence |
| intervals, KMeans with silhouette, permutation importance — things an executive-facing narrative |
| can explain and a report can defend. Heavy AutoML (AutoGluon, FLAML) appears in Kaggle-style |
| agents (MLE-bench, AIDE), not analyst products. |
|
|
| --- |
|
|
| ## 2. What we add and why (feature detail) |
|
|
| Product gaps these close, mapped to CRISP-DM (the report's own structure): |
|
|
| 1. **Charts (V1/V2)** — *Data Understanding + Evaluation.* Trend, composition, correlation and |
| distribution questions are answered today with tables the user must mentally plot. Every |
| comparable product renders charts; our reports' "EDA" section is tables-only. V1 needs **zero |
| new LLM calls**: each registered `analyze_*` already returns typed, structured output |
| (`ToolOutput.kind ∈ table|stats|series`) that maps rule-deterministically to a chart type. |
| 2. **Forecast (M1, `analyze_forecast`)** — *Modeling.* "What will sales look like next quarter?" |
| is currently answered by `analyze_trend` (descriptive slope only). ETS/SARIMAX with a holdout |
| backtest (MAPE/sMAPE reported) + CI bands is the minimum credible answer. |
| 3. **Clustering (M1, `analyze_segment` upgrade → `analyze_cluster`)** — *Modeling.* "What kinds of |
| customers do we have?" KMeans on scaled numerics, k by silhouette, cluster profile table + |
| PCA-2D scatter. Note the taxonomy already reserved `analyze_segment` (built, unregistered) — |
| this either upgrades it or registers a sibling; decide with the tool owner. |
| 4. **Anomaly detection (M1, `analyze_anomaly`)** — *Evaluation.* "Anything unusual last month?" |
| IsolationForest (tabular) / STL-residual z-score (time series); flagged-rows table + marked |
| chart. Also the seed of the Level-4 "proactive monitoring" story later. |
| 5. **Driver analysis (M1, `analyze_driver`)** — the Level-3 differentiator: "what's driving |
| churn/the dip?" Regularized linear/logistic fit + permutation importance → ranked-driver table. |
| Complements `analyze_contribution` (arithmetic decomposition) with statistical attribution. |
| 6. **Model artifacts in reports** — every M1 tool writes metrics + caveats into its |
| `AnalysisRecord`, so reports gain honest Modeling/Evaluation sections for free (charts embed as |
| kaleido PNG when the deferred PPT/PDF export lands — same artifact, two renderings). |
|
|
| Non-goals now: dashboards, scheduled/proactive monitoring, model persistence/registry (each run |
| fits in-request on ≤10k retrieved rows), deep-learning anything, cross-source joins. |
|
|
| --- |
|
|
| ## 3. Architecture (how) |
|
|
| ### 3.1 Charts — V1 data flow (no LLM) |
|
|
| ``` |
| structured_flow turn (unchanged): |
| Planner → TaskRunner → results_snapshot {task_id → ToolOutput} |
| │ |
| ▼ NEW, deterministic, never-throw |
| ChartBuilder.build(results_snapshot, plan) |
| rule table: analyze_trend→line · analyze_aggregate→bar/grouped |
| analyze_correlation→heatmap · analyze_descriptive→histogram |
| analyze_comparison→grouped bar · analyze_contribution→pareto |
| → ChartSpec (pydantic, ≤1 per substantive task) |
| │ |
| ▼ |
| SpecCompiler → plotly.graph_objects.Figure (schema-validated |
| by construction) → fig.to_json(), downsample >2k pts/trace, |
| payload cap ~1 MB |
| │ |
| ▼ |
| ChartStore.save → dedorch `message_charts` (Python-owned JSONB, one row per chart, |
| keyed analysis_id+message_id+order) — flushed alongside the traceability flush, |
| before `done` (same 8-site discipline; error turns write nothing) |
| │ |
| SSE stream: UNCHANGED (text-only) ──▶ done{message_id} |
| │ |
| FE on done ──▶ GET /api/v1/charts?analysis_id&message_id → [{chart_id, spec, figure_json, |
| title, source_task_id}] ──▶ react-plotly.js render under the answer |
| ``` |
|
|
| Design rules carried over from the house style: **never-throw** (a chart failure degrades to |
| no-chart, never kills the turn) · charts derive **only from executed tool results** (never from |
| LLM text — grounded by construction, LIDA's "data-faithful" property) · traceability's `tool_calls` |
| entries gain a `chart_id` ref so provenance and artifact stay correlated. |
|
|
| ### 3.2 Charts — V2 (`render_chart` tool + ChartSpec-by-LLM) |
| |
| - Register `render_chart` in the planner registry (Pattern A: takes `data` = `${t<id>}` + spec |
| params) so "plot revenue by region as a pie" becomes a plannable step. `ToolOutput` gains |
| `kind="chart"` — **one-line Literal change in tool-team-owned `contracts.py` + an Assembler |
| branch; coordinate with the tool owner before building.** |
| - Where the rule table is ambiguous or the user asked for a specific viz, ONE structured-output |
| LLM call emits `ChartSpec` (chat2plot's exact trick: constrained pydantic schema via function |
| calling — the LLM never writes Plotly JSON, so invalid output is a validation error with one |
| repair retry, mirroring the Planner's re-prompt loop). |
| - Chart-edit turns ("make it horizontal") load the stored spec, apply the delta, re-compile, |
| save a new chart row — spec-as-artifact is what makes edits cheap (family-B dividend). |
|
|
| ### 3.3 Modeling tools — M1 |
|
|
| All four are composite tools in the existing taxonomy — Pattern A inputs, `ToolOutput` outputs, |
| registered in `analytics_registry()`, planner-visible with prompt-grade descriptions: |
|
|
| | Tool | Method (all CPU, interpretable) | Output (`kind`) | Auto-caveats | |
| |---|---|---|---| |
| | `analyze_forecast` | statsmodels ETS; SARIMAX when seasonality detected; naive-seasonal fallback | `series` (history + forecast + CI) + chart | holdout MAPE/sMAPE; "≥2 seasons or fallback"; missing-period warning | |
| | `analyze_cluster` | sklearn scale→KMeans, k∈2..8 by silhouette | `table` (profiles) + PCA scatter chart | silhouette score; "clusters are descriptive, not causal" | |
| | `analyze_anomaly` | IsolationForest (tabular) / STL residual z (TS) | `table` (flagged rows) + marked chart | contamination assumption; top-N only | |
| | `analyze_driver` | standardized ridge/logistic + permutation importance | `stats` (ranked drivers) + bar chart | R²/AUC on holdout; "association ≠ causation" | |
|
|
| Safety/robustness rails (same philosophy as `DbExecutor`): row cap (inherits the 10k retrieve |
| cap) + feature cap (≤20 numeric) · fixed `random_state` (repeatable runs — the family-B promise) · |
| `asyncio.to_thread` + 30s wall-clock timeout · never-throw (failure → `kind="error"`, TaskRunner |
| degrade-and-continue does the rest) · metrics/caveats copied verbatim into the `AnalysisRecord` |
| (Assembler narrates them; it never invents numbers — existing rule). |
|
|
| ### 3.4 What explicitly does NOT change |
|
|
| Router intents (structured_flow already covers "plot/forecast/segment" asks — verify with new |
| `eval/intent` cases, not new intents) · SSE event shape (charts are fetched, not streamed) · |
| QueryIR/compiler/executor (ML runs on already-retrieved DataFrames) · report floor · guardrail |
| layers. |
| |
| --- |
| |
| ## 4. Stack rationale (why these picks) |
| |
| ### 4.1 Plotly JSON over the alternatives |
| |
| | Option | Verdict | Reasoning | |
| |---|---|---| |
| | **Plotly Figure JSON** (pick) | ✅ | Already a pinned dep (`plotly==5.24.1`, `kaleido==0.2.1`) and already the team lean (DEV_PLAN #26 "Plotly→JSON, not matplotlib PNG"). `graph_objects` construction = schema validation for free. `react-plotly.js` is mature for the React/Vite FE. **kaleido closes the report loop**: same figure → interactive JSON in chat, PNG in PPT/PDF export (DEV_PLAN deferred "PPT preferred"). | |
| | Vega-Lite | ❌ for now | The research favorite (LIDA/chat2plot support it; tighter grammar, smaller specs) — *when the LLM writes the spec*. In our design the LLM writes a tiny `ChartSpec` and a deterministic compiler emits the figure, so Vega-Lite's LLM-ergonomics advantage mostly evaporates, and it would add an FE renderer + Python compiler we don't have. `ChartSpec` is renderer-agnostic (chat2plot precedent) — a Vega compiler can be added later without touching tools. | |
| | matplotlib/seaborn PNG | ❌ | Static, non-interactive, heavier payloads, no client theming, and the team already rejected it (#26). Keep matplotlib only as kaleido's export path. | |
| | Mermaid/ASCII in markdown | ❌ | Not data-viz grade. | |
| |
| ### 4.2 statsmodels + scikit-learn over the alternatives |
| |
| - **statsmodels**: the default in every surveyed agent-tools stack for forecasting; ETS/SARIMAX are |
| explainable to executives and run in milliseconds on our row caps. *(New dep — needs sign-off.)* |
| - **scikit-learn**: KMeans/IsolationForest/linear models cover cluster/anomaly/driver with |
| interpretable outputs. Already in the venv **transitively** via sentence-transformers — pin it |
| explicitly the moment we import it (transitive deps are not a contract). |
| - **Rejected:** prophet (heavy dep, maintenance-mode, marginal gain over ETS at our scale) · |
| AutoGluon/FLAML (GPU-hungry, opaque ensembles — wrong for decision-ready narratives; that's |
| Kaggle-agent gear) · sktime/TimeCopilot (nice unified API, but another abstraction layer over the |
| two libs we'd still be running; revisit if tool count grows) · LLM-as-forecaster (research shows |
| it underperforms classical baselines on numeric TS; we use the LLM to *plan and narrate*, never |
| to produce numbers — existing Assembler rule). |
| |
| ### 4.3 No code sandbox (the biggest "why not") |
| |
| Codegen would give the long tail (custom feature engineering, exotic plots) but costs exactly what |
| our architecture is sold on: arbitrary LLM code vs our five-layer read-only defense; per-run |
| variance vs `report_inputs` repeatability; PNG blobs vs auditable specs; and real infra we don't |
| have — HF Spaces is one Docker container (no Firecracker microVMs; in-process `exec()` of LLM code |
| is a non-starter against our own guardrail posture). Every hosted option (E2B, Modal, Daytona) |
| means a new external service + credential-isolation design. **Decision gate for M2 (§7):** revisit |
| only when a logged backlog of user asks provably doesn't fit the curated tools. |
|
|
| --- |
|
|
| ## 5. Implementation plan (task-table-ready) |
|
|
| Statuses: ⬜ not started (all — proposal). Owners are suggestions. |
|
|
| | # | Phase | Task | Owner | Note | |
| |---|---|---|---|---| |
| | 1 | V1 | `ChartSpec` pydantic + rule table + `SpecCompiler` (plotly), downsampling + size caps | Rifqi | pure Python, no seams touched | |
| | 2 | V1 | `ChartBuilder` hook in `chat_handler._run_slow_path` after TaskRunner; never-throw | Rifqi | mirrors traceability accumulation | |
| | 3 | V1 | `message_charts` ORM + `ChartStore` (save/list); flush wired at the traceability flush sites | Rifqi | Python-owned, like `message_traceability` | |
| | 4 | V1 | **DDL handoff to Harry**: `message_charts` (uuid id, analysis_id FK, message_id, user_id, `chart` jsonb {spec, figure, title, source_task_id, order}, created_at) | Rifqi → Harry | plural name, uuid ids — house rules | |
| | 5 | V1 | `GET /api/v1/charts?analysis_id&message_id` + contract § in API_CONTRACT_BE_PYTHON.md | Rifqi | mirror traceability endpoint; 404 semantics same | |
| | 6 | V1 | FE: react-plotly.js renderer + fetch-on-done; Go passthrough if FE→Go→Python | FE + Harry | coordination, not Python work | |
| | 7 | V1 | traceability `tool_calls[].chart_id` ref; local tests (spec compile goldens, store, endpoint) | Rifqi | tests stay local | |
| | 8 | V2 | `render_chart` ToolSpec + `kind="chart"` in contracts.py | **tool owner** + Rifqi | contracts are tool-team-owned — coordinate first | |
| | 9 | V2 | ChartSpec-by-LLM (structured output + 1 repair retry) for explicit viz asks; chart-edit turns | Rifqi | chat2plot pattern | |
| | 10 | V2 | `eval/intent` cases for plot/forecast phrasing (EN+ID); chart-rule goldens in eval or tests | Rifqi + Sofhia | protocol: eval before router-adjacent claims | |
| | 11 | M1 | `uv add statsmodels` + pin scikit-learn (sign-off needed) | Rifqi | manual §6.4 | |
| | 12 | M1 | `analyze_forecast` (+ backtest + caveats) → registry + planner prompt row + few-shot | Rifqi + tool owner | taxonomy fit review | |
| | 13 | M1 | `analyze_cluster` / `analyze_anomaly` / `analyze_driver` (same template) | split | one PR each, stacked | |
| | 14 | M1 | Report generator: Modeling/Evaluation sections consume new record fields; kaleido PNG path stub for PPT export | Sofhia (report) / Rifqi | ties into deferred report-formats work | |
| | 15 | M2 | ⏸️ sandbox decision gate: collect can't-answer asks in traceability meta; revisit with evidence | — | deferred by design | |
|
|
| Sequencing: 1–7 ship V1 end-to-end (demo-visible; FE renderer is the only external dependency — |
| until it lands, Swagger shows the JSON). 8–10 next. 11–14 after taxonomy review with the tool |
| owner. Estimates: V1 ≈ 4–5 Python dev-days + 2–3 FE; V2 ≈ 3; M1 ≈ 6–8. |
|
|
| ## 6. Coordination & infra deltas (the handoff list) |
|
|
| 1. **Harry / dedorch:** `message_charts` migration (task #4 DDL). Optional later: `charts_count` |
| hint in the `done` payload — contract change, batch it with the next contract rev. |
| 2. **FE:** react-plotly.js (+ theme config), fetch-on-done, render under answer bubble; chart |
| panel in the report preview later. |
| 3. **Tool team:** `kind="chart"` Literal + `render_chart` spec review; `analyze_segment` |
| upgrade-vs-sibling decision (§2.3). |
| 4. **Deps (Rifqi sign-off):** `statsmodels` new; `scikit-learn` explicit pin. Both CPU wheels, |
| no image-size drama on HF. |
| 5. **No change requested from:** router intents, SSE shape, Redis, Langfuse, guardrails. |
|
|
| ## 7. Risks & mitigations |
|
|
| | Risk | Mitigation | |
| |---|---| |
| | Chart payloads bloat the DB / FE | downsample >2k pts/trace; ~1 MB cap; store spec always, figure optionally recompilable | |
| | Wrong chart type annoys users | V1 rules only for unambiguous mappings; else no chart (no-chart beats bad-chart); V2 adds the LLM picker | |
| | Forecast on garbage data → confident nonsense | hard preconditions (min periods, regular frequency) → refuse-with-reason into the record; backtest metric always shown; naive fallback labeled | |
| | Never-throw hides chart/model failures | same answer as traceability: failure reason lands in the tool span + record caveats, visible in `/traceability` | |
| | PII in chart labels/tooltips | charts render only executed result data (same exposure class as today's tables); `pii_flag` columns excluded from V2 LLM spec-picking context (prompt-plane rule holds) | |
| | Scope creep toward AutoML/sandbox | M2 gate requires logged evidence of unmet asks, not vibes | |
|
|
| ## 8. Open questions |
|
|
| 1. `message_charts` vs folding chart JSON into `message_traceability` — separate table recommended |
| (different consumer, different lifecycle, report reuse by `chart_id`), but Harry may prefer one |
| migration. **Decide at DDL handoff.** |
| 2. Does the FE want figure JSON (render-ready, bigger) or spec+data (smaller, FE compiles)? |
| Recommend figure JSON first — dumbest possible FE integration. |
| 3. `analyze_segment` (existing, unregistered) vs new `analyze_cluster` — tool-owner call. |
| 4. Report PNG embedding: inline base64 in markdown vs chart_id placeholder resolved at export — |
| recommend placeholder; decide with report-formats work. |
| 5. Do forecast/cluster asks change `structured_flow` routing confidence? Add eval cases first |
| (task #10) — data before prompt edits. |
|
|
| ## 9. Sources (reviewed 2026-07-07) |
|
|
| - Tellius — [Best AI Data Analysis Agents in 2026: 12 platforms compared](https://www.tellius.com/resources/blog/best-ai-data-analysis-agents-in-2026-12-platforms-compared-for-nl-to-sql-autonomous-investigation-and-governance) (maturity levels, governance trade-offs, Gartner 2026 agentic-analytics framing) |
| - Microsoft Research — [LIDA: grammar-agnostic visualization generation with LLMs](https://microsoft.github.io/lida/) · [paper](https://aclanthology.org/2023.acl-demo.11/) (staged pipeline, generate-validate-repair, data-faithfulness) |
| - [chat2plot](https://github.com/nyanp/chat2plot) (declarative JSON specs over codegen; structured-output validation; "more secure execution, as LLM does not directly generate code") |
| - LangChain — [deep-agents data analysis reference](https://docs.langchain.com/oss/python/deepagents/data-analysis) (the codegen-family architecture: sandbox backends E2B/Modal/Daytona, PNG artifacts, credential isolation) |
| - Modal — [Best code execution sandboxes for AI agents 2026](https://modal.com/resources/best-code-execution-sandboxes-ai-agents) (sandbox infra requirements: microVM isolation, warm pools) |
| - [Vega-Lite](https://vega.github.io/vega-lite/) · [Plotly vs Vega comparison thread](https://github.com/plotly/documentation/issues/1330) (template-based vs grammar-based spec trade-off) |
| - [sktime LLM workflows](https://medium.com/@benedikt_heidrich/can-we-do-time-series-analysis-with-llm-powered-workflows-using-sktime-12b19cf39376) · [TimeCopilot / agentic forecasting survey](https://arxiv.org/html/2508.04231v1) (statsmodels/sklearn-as-tools pattern; LLM plans, classical models compute) |
|
|