docs/DSAGENT_DEV_GUIDE.md

Lexsi DS Agent — Developer / KT Guide

A code-level companion to DSAGENT_OVERVIEW.md (which is the what/why). This is the how: file map, control flow, the contracts you'll touch, configuration, and the full benchmark-running mechanics. Read the overview first, then this.

---

1. Mental model in one paragraph

One question → one AgentLoop.run() → a ReAct loop: build a system prompt (framing + active-dataset summary + tool catalog), ask the LLM for a JSON decision ({"thought","tool","args"} or {"thought","final_answer"}), validate args with the tool's pydantic schema, run the tool against a shared AgentContext, append the observation to history, repeat until a final answer or the step budget. Tools read and write a shared ctx.cache scratchpad. The data substrate is always DuckDB; tabular ML runs on remote Lexsi pods via the SDK. The loop is generic — it has no opinion on which tools to call.

---

2. Repo layout (code-oriented)

lexsi_ds/
  agent/
    loop.py            AgentLoop.run() — the ReAct orchestrator (start here)
    context.py         AgentContext, DatasetHandle, TableInfo, ColumnInfo
    prompts.py         PLANNER_SYSTEM, render_tool_catalog, tool-selector (intent routing)
    datasource.py      DuckDB handle loaders + multi-DB materialization + DB cache
    tools/
      base.py          Tool / ToolSpec / ToolResult contract (the interface)
      __init__.py      REGISTRY: dict[str, Tool]  (30 tools)
      <name>.py        one tool per file, each exports `TOOL: Tool`
      _tabular_common.py  shared helpers for the model-lifecycle tools
                          (resolve_tab_project, active_model_name, df_records, …)
    kg_index.py        embedding/lexical index over the PKDD knowledge graph
    table_index.py     embedding index over table schemas (schema linking, big DBs)
    session.py         SessionContext + multi-turn merge
    turn_classifier.py classify a turn: new / refine / abandon
    reference_resolver.py  resolve "it/that/those" against prior turns
    samples.py         curated demo questions (by_id, SAMPLES) — UI chips + tests
  llm/client.py        LLMClient protocol, StubLLMClient, LexsiTextClient, cache, timeout
  schema/
    raw.py             PKDD DDL introspection (introspect())
    context_graph.py   ContextGraph — the banking knowledge graph
  data/loader.py       PKDD TSVs -> artifacts/financial.duckdb
  paths.py             DUCKDB_PATH, ARTIFACTS_DIR, etc.

app/gradio_app.py      the UI (build_ui(); demo.launch on GRADIO_PORT)

benchmark/
  dab/
    adapter.py         parse a DAB dataset -> DatasetHandle + tasks (+ DB cache key)
    runner.py          run agent per DAB task, score Pass@1 via the task's validate.py
    DataAgentBench/    the (gitignored) upstream checkout + DB files
  pkdd_runner.py       run agent over bench/pkdd/questions.yaml, auto-score
  datagen/             template pipeline that GENERATES bench/pkdd/*.yaml

bench/pkdd/
  questions.yaml       40 single-turn bench items (analytic/predictive/adversarial)
  sessions.yaml        multi-turn sessions
  p2_feature_engineering.yaml   10 feature-eng items (separate `assertions` schema)
benchmark/fixtures/pkdd_seed.duckdb   pinned PKDD snapshot the bench scores against

scripts/
  repro_tabicl/                  TabICL inference + per-case XAI failure repro
  repro_model_lifecycle/         model-lifecycle SDK probe harness (self-contained):
    data/                        the PKDD CSVs (local copy)
    repro_model_lifecycle.py             TabICL-first (clean)
    repro_model_lifecycle_xgboost_first.py  XGBoost-first (#05-013 cascade repro)

tests/                 pytest suite (test_b_tools, test_v1_multiturn, test_dab_*, …)
docs/                  design + results + this guide

---

3. The run lifecycle (loop.py)

AgentLoop(ctx, registry=REGISTRY, max_steps=18).run(question) -> AgentRun

run(question):
  1. (multi-turn) classify_turn() -> new/refine/abandon; resolve_reference()
     - "abandon" short-circuits with a polite answer.
  2. ctx.cache["question"] = question
  3. system = _render_system(...)                 # built ONCE per run, re-sent every step
       - _select_tools(question): LLM intent classifier -> render only relevant tool group(s)
       - render_dataset_block(ctx) + render_tool_catalog(tools) + PLANNER_SYSTEM
  4. history = ["USER QUESTION: ..."]
  5. while n < max_steps:
       user_msg = "\n\n".join(history) + "You have {max_steps-n} tool calls left..."
       decision = parse(ctx.llm.complete(system, user_msg))
       if "final_answer": finalize + return
       tool = registry.get(decision["tool"])
       result = _call_tool(tool, decision["args"])   # pydantic-validate then tool.run()
       history.append("TOOL CALL ...\nOBSERVATION: " + result.summary[:1500])
       # anti-loop: same tool fails 2x in a row -> inject "[LOOP GUARD] re-plan" directive
       if result.meta.get("pause_for_user"): return (clarify pause)   # interactive only
  6. budget exhausted -> _force_final(): ask once for a final answer

Key methods: run, _render_system, _select_tools, _call_tool, _force_final. Dispatch always uses the full self.registry; the tool selector only narrows what the planner sees in the prompt, never what it can call.

Data classes:

• AgentRun — run_id, question, dataset_id, steps[], final_answer, ok, error, pending_clarification, system_prompt.
• Step — n, thought, tool, args, result(ToolResult), raw_llm_text, latency_s, prompt. The full per-turn user message lives on each Step (so trajectories are reproducible).

---

4. Core data structures (context.py)

AgentContext — built once per run, passed to every tool:

• dataset: DatasetHandle, run_id
• org / text_project / tab_project / tab_projects — Lexsi SDK handles (may be None offline)
• llm — an LLMClient
• cache: dict — run-scoped scratchpad (see §6)
• session — optional SessionContext (multi-turn)
• interactive: bool — True in the UI (clarify pauses); False in bench/headless (clarify falls back to its default)
• .duck(read_only=None) — open a DuckDB connection on dataset.duckdb_path (read-only for bundled, writable otherwise)

DatasetHandle — id, kind, duckdb_path, tables: list[TableInfo], connector_id, kg. kind ∈ {bundled, upload, connector, attached}. Everything funnels through ctx.duck() → duckdb_path, so "switch dataset" = build a new handle and assign ctx.dataset.

TableInfo = name, columns: list[ColumnInfo], n_rows, comment. ColumnInfo = name, dtype, comment, sample (the sample value is the per-column representative value the planner sees in the schema — populated at describe time).

---

5. The tool contract — how to add a tool (tools/base.py)

A tool is any object with .spec: ToolSpec and .run(args, ctx) -> ToolResult.

# lexsi_ds/agent/tools/my_tool.py
from pydantic import BaseModel, Field
from lexsi_ds.agent.context import AgentContext
from lexsi_ds.agent.tools.base import Tool, ToolResult, ToolSpec

class MyToolArgs(BaseModel):
    table_or_label: str = Field(..., description="...")   # match sibling arg names!

def _run(args: MyToolArgs, ctx: AgentContext) -> ToolResult:
    con = ctx.duck()
    ...
    return ToolResult(ok=True, summary="short text the LLM sees (<~200 tok)",
                      payload={"dataframe": df})           # rich data; LLM never sees it

class _MyTool:
    spec = ToolSpec(name="my_tool", description="...(LLM sees verbatim)...",
                    args_schema=MyToolArgs, returns="what the observation looks like")
    def run(self, args, ctx): return _run(args, ctx)

TOOL: Tool = _MyTool()

Then register it: add from .my_tool import TOOL as my_tool_tool and an entry in REGISTRY in tools/init.py. If it fits an intent group, add it to _TOOL_GROUPS in prompts.py so the tool selector surfaces it (see §8).

Contract notes:

• summary is the only thing the LLM sees; payload is for the UI / downstream tools.
• Return ok=False with an actionable summary on expected failures — the planner reads it and re-plans. Unexpected exceptions propagate; the loop wraps them.
• Arg-name convention: sibling tools that take a source use table_or_label. Match it (or alias via pydantic AliasChoices) — mismatches show up as validation_error and waste planner steps (this bit us; see transform_column).
• Args are validated in _call_tool via tool.spec.args_schema.model_validate(raw_args).

---

6. The cache contract (ctx.cache)

Run-scoped (session-scoped in multi-turn). Producers → consumers, by key:

Key	Producer	Consumer
question	loop	tool selector
tool_intents::<q>	_select_tools	(memoize the intent classification)
sql_result:<label>	run_sql	profile_data, sample_rows, transform_column, predict
last_sql_result, last_sql	run_sql	PKDD scorer, downstream
profile:<label>	profile_data	train_tabular_model, dq checks
last_train_columns	train_tabular_model	predict (schema alignment)
last_predictions, last_predict_tag, last_model_id	predict	explain_prediction, evaluate_predictions
last_eval	evaluate_predictions	summarize_result

When adding a tool that hands data downstream, write a documented key here.

---

7. Data layer (datasource.py)

Handle loaders (each returns a DatasetHandle):

• load_pkdd_handle() — bundled PKDD (DUCKDB_PATH = artifacts/financial.duckdb), KG attached.
• load_upload_handle(spec) — CSV/Parquet → per-session DuckDB.
• load_s3_connector_handle(spec) — read_parquet(s3://…) views.
• load_attached_handle(specs, *, dataset_id, target_path=None) — fold heterogeneous DBs into one DuckDB.
• load_postgres_connector_handle / load_mongo_connector_handle — live DB connectors (reuse the materialize helpers).

Multi-DB fold-in (_build_attached_duckdb): each source becomes <alias>_<table>:

• sqlite/duckdb → ATTACH + CREATE TABLE AS SELECT (_materialize_file_db)
• postgres → restore the .sql dump into an embedded pgserver, copy via postgres scanner (_materialize_postgres_dump); tears down the pgdata dir after.
• mongo → read .bson server-lessly, flatten nested → JSON text (_materialize_mongo_dump).

Schema sampling: _describe_tables → _column_samples runs one SELECT * LIMIT 5 per table and stores a representative value on ColumnInfo.sample. This is what lets the planner see "Date: VARCHAR sample='31 Dec 1986, 00:00'" and parse with TRY_CAST/strptime instead of naive casts.

DB cache (build-once) — load_attached_handle(target_path=...): if the master exists at target_path, reuse it; else materialize once (atomic .building → rename). Each run gets an isolated working copy (shutil.copyfile → artifacts/sessions/attach_*.duckdb) so agent writes never mutate the master. The DAB adapter computes the master path + content-fingerprint (see §13).

Session files live in artifacts/sessions/ via _new_session_db(); the DAB runner deletes the per-dataset working copy when done (_cleanup_working_copy).

---

8. Prompts + the token-lean tool selector (prompts.py)

• PLANNER_SYSTEM — the ReAct framing + hard rules + JSON response format.
• render_tool_catalog(registry) — renders each tool as ## name + description + one-line arg signature (_compact_args, not full JSON Schema — ~64% fewer tokens)
  • Returns. Enums/Literals keep their choices.
• render_dataset_block(ctx) / render_session_context(...) — dataset summary + multi-turn prefix.
• Tool selector (gated, LLM-based): _select_tools (in loop.py) calls a one-shot classifier (TOOL_INTENT_SYSTEM) once per run, cached. parse_tool_intents → subset_for_intents(intents, registry) renders core tools + the union of matched groups (_CORE_TOOLS, _TOOL_GROUPS); ambiguous/multi-intent → full catalog. Toggle with LEXSI_TOOL_SELECTOR=0. Dispatch is always against the full registry.

---

9. LLM client (llm/client.py)

• LLMClient protocol: .complete(system, user) -> LLMResult(text, raw).
• Implementations: StubLLMClient (offline sentinels), EchoGoldLLMClient (tests), LexsiTextClient (real; LexsiTextClient.from_env()), built via factory("lexsi"|"stub").
• Caching: in-memory LRU (LEXSI_LLM_CACHE_MAX, default 256) keyed on (model, provider, max_tokens, system, user); optional disk layer when LEXSI_LLM_CACHE_DIR is set (the bench runners set it → replays across runs). Disable with LEXSI_LLM_CACHE=0.
• Timeout/retry (important): every gateway call goes through _call_with_timeout (daemon-thread guard, LEXSI_LLM_TIMEOUT_S default 180, LEXSI_LLM_RETRIES default 1). Without it a stalled gateway request freezes the whole run — this was a real incident.
• Model-family handling: GPT-5 / o-series reject max_tokens; the client POSTs directly with max_completion_tokens (_post_chat_with_completion_tokens).

---

10. Multi-turn

SessionContext (session.py) seeds ctx.cache from the session at run start and lifts changes back at the end. classify_turn (turn_classifier.py) labels each turn new/refine/abandon; resolve (reference_resolver.py) resolves anaphora and hands the planner an explicit reference.

---

11. Configuration (environment variables)

Required for the real LLM path: SDK_ACCESS_TOKEN, LEXSI_ORG_NAME, LEXSI_WORKSPACE_NAME, LEXSI_TEXT_PROJECT_NAME, LEXSI_TEXT_PROVIDER, LEXSI_TEXT_MODEL. Optional: LEXSI_TEXT_PROVIDER_API_KEY, LEXSI_TEXT_MAX_TOKENS.

Tabular: LEXSI_TABULAR_PROJECT_NAME (per-run projects created on train, so usually unset).

Var	Default	Purpose
LEXSI_TOOL_SELECTOR	1	gate the LLM tool selector (0 = always full catalog)
LEXSI_LLM_TIMEOUT_S	180	hard timeout per gateway call
LEXSI_LLM_RETRIES	1	retries on timeout/failure
LEXSI_LLM_CACHE	1	in-memory LLM cache on/off
LEXSI_LLM_CACHE_DIR	—	enable on-disk LLM cache (bench runners set this)
LEXSI_LLM_CACHE_MAX	256	LRU size
DUCKDB_PATH	artifacts/financial.duckdb	bundled PKDD DB
DAB_ROOT	benchmark/dab/DataAgentBench	DAB checkout
GRADIO_PORT	7860 (7850 in Docker)	UI port
LEXSI_UI_RUN_TIMEOUT_S, LEXSI_UI_HEARTBEAT_S, LEXSI_UI_SSR	—	UI tuning
LEXSI_TRAIN_TIMEOUT_S, LEXSI_PREDICT_TIMEOUT_S, LEXSI_XAI_BUDGET_S	—	Lexsi pod waits
LEXSI_ROUNDTRIP_MODEL/_PROVIDER/_DISABLE	—	datagen Stage-4 back-translator

(Connector creds — PG_*, MONGO_URI, AWS_*, MYSQL_* — are read by the live connect_datalake paths.)

---

12. Local dev setup

# deps (extras: lexsi, ui, dev, embeddings, connectors)
uv sync --extra lexsi --extra ui --extra dev --extra embeddings --extra connectors

# bootstrap the bundled PKDD DuckDB (one-time, ~10s)
uv run python -m lexsi_ds.data.loader

# drive the agent from Python (offline stub LLM)
uv run python -c "
from lexsi_ds.agent import AgentContext, AgentLoop
from lexsi_ds.agent.datasource import load_pkdd_handle
from lexsi_ds.llm.client import factory
ctx = AgentContext(dataset=load_pkdd_handle(), run_id='local', llm=factory('stub'))
print(AgentLoop(ctx=ctx).run('Top 5 districts by number of accounts.').final_answer)
"

# UI
uv run python -m app.gradio_app          # http://localhost:7860

# tests (LLM-gated ones skip without the Lexsi env)
uv run pytest -q

---

13. Benchmark mechanisms (all of them)

There are three harnesses. All write full trajectories so failures are debuggable.

13a. DataAgentBench (external; Pass@1) — benchmark/dab/

What it is: 17 datasets across sqlite/duckdb/postgres/mongo; each task ships a question + ground_truth.csv + its own validate.py. We score Pass@1 by calling that validate.py on the agent's final answer.

Code:

• adapter.py: load_dataset(dir) → DabDataset (parses db_config.yaml into AttachSpecs via parse_db_config, loads each task's validate.py via _load_validate). build_handle(dataset, cache_dir=...) → load_attached_handle with a content-keyed master path (<cache_dir>/<dataset>__<_fingerprint>.duckdb).
• runner.py: main → run_dataset (builds the handle once per dataset, loops tasks) → run_task (composes the question with the DAB description + table-name mapping, runs AgentLoop, calls validate). DB working copy cleaned per dataset (_cleanup_working_copy).

Setup + run:

git clone https://github.com/ucbepic/DataAgentBench benchmark/dab/DataAgentBench
uv sync --extra connectors        # needed for the postgres/mongo datasets
export SDK_ACCESS_TOKEN=… LEXSI_WORKSPACE_NAME=… LEXSI_TEXT_PROJECT_NAME=…  # + org/model/provider

# one dataset / one task / sweep all
uv run python -m benchmark.dab.runner --dataset DEPS_DEV_V1 --llm lexsi
uv run python -m benchmark.dab.runner --dataset DEPS_DEV_V1 --query query1 --llm lexsi
uv run python -m benchmark.dab.runner --all --llm lexsi

# the sqlite/duckdb "core" (5 datasets / 17 tasks, no connectors extra), real Pass@1
uv run python -m benchmark.dab.runner --all --db-types sqlite,duckdb --llm lexsi \
  --iterations 3 --max-steps 20 --log-dir artifacts/dab_runs/run1

Flags: --dataset/--all, --db-types t1,t2 (filter --all), --query, --iterations, --max-steps (default 12), --use-hints, --log-dir, --db-cache-dir/--no-db-cache, --llm-cache-dir/--no-llm-cache, -v.

Caching (two layers, both speed re-runs):

• DB cache artifacts/dab_dbs/ — each dataset's fold-in materialized once; reused across runs; per-run working copy auto-deleted.
• LLM disk cache artifacts/llm_cache/ — gateway responses keyed on the prompt; a failed sweep resumes cheaply.

Outputs: per-task trace JSON in --log-dir (<dataset>_<query>_iter<N>.json — full steps, prompts, raw LLM output, payloads, validator verdict) + a console scoreboard (per-dataset pass counts, wall vs agent time, LLM reuse). Latest results writeup: dab_results_v1.md; integration detail: dab_integration.md.

13b. PKDD-Curated (internal) — benchmark/pkdd_runner.py

What it is: our own suite over the bundled PKDD dataset, scored against the pinned fixture benchmark/fixtures/pkdd_seed.duckdb. Question schema in benchmark/datagen/types.py (Row); the live set is bench/pkdd/questions.yaml (40 items: analytic, predictive, adversarial).

Scoring (pkdd_runner.py, by scoring_rule):

• row_set_equality (analytic) → _score_row_set: agent's last_sql_result vs gold_sql executed on the fixture, as a multiset of sorted-value rows (column-rename tolerant; lenient scalar fallback).
• exact_match (numeric) → _score_numeric: reference value within tolerance.
• judge_rubric (adversarial) → _score_behavioral: heuristic — fail if it used a forbidden_tool or answered when it should clarify/refuse.
• ndcg@k (predictive) → captured but not auto-scored unless --include-predictive (those train real models, slow). Plan precision/recall computed for every row.

Run:

export SDK_ACCESS_TOKEN=… LEXSI_WORKSPACE_NAME=… LEXSI_TEXT_PROJECT_NAME=…  # + org/model/provider
uv run python -m benchmark.pkdd_runner --llm lexsi --log-dir artifacts/pkdd_runs
# scope / debug
uv run python -m benchmark.pkdd_runner --tier analytic --limit 5 --llm lexsi
uv run python -m benchmark.pkdd_runner --include-predictive --llm lexsi

Flags: --tier, --include-predictive, --limit, --llm, --log-dir, --llm-cache-dir/--no-llm-cache, -v. Outputs: per-row trace JSON + a per-tier scoreboard (pass/scored + plan p/r). The agent queries the fixture (handle from load_pkdd_handle() with duckdb_path repointed to the fixture).

Note: bench/pkdd/sessions.yaml (multi-turn) and p2_feature_engineering.yaml (separate assertions schema) are not run by pkdd_runner yet.

13c. Datagen pipeline (generates the PKDD questions) — benchmark/datagen/

Template-based; authoring is per-template, so ~10 templates yield 500+ rows. Stages: column inventory → templates → instantiate → execute gold SQL on the pinned fixture → LLM round-trip verify → predictive enrichment → multi-turn compose → emit YAML.

uv run python -m benchmark.datagen pin-fixture        # snapshot artifacts/financial.duckdb
uv run python -m benchmark.datagen run --skip-roundtrip   # fast (skip LLM Stage 4)
uv run python -m benchmark.datagen run                    # full (needs Lexsi text env)

Every emitted row carries fixture_sha; a scoreboard from a different sha isn't comparable. The grading design (5 axes, tiers, pass^k) lives in v1_benchmarking.md.

13d. SDK repro / probe scripts — scripts/repro_*

Standalone, self-contained scripts that hit the live Lexsi SDK against a real trained model and report OK/FAIL per call. They're how we validate the SDK-backed tools and file platform bugs. Each catches every call so one run produces a complete report; each leaves its project on the platform for Activity-Log inspection.

• scripts/repro_tabicl/ — reproduces TabICL model_inference + per-case XAI failures (empty Exception on T4 pods, inference hang/timeout, case_predict ValidationError). Ships its own PKDD CSVs in data/.
• scripts/repro_model_lifecycle/ — probe harness for the model-lifecycle tools (list_available_models, select_active_model, compare_models, check_drift, monitor_performance) + the future explanation methods. Trains TabICL + XGBoost on the (local) PKDD CSVs and calls each SDK method. Two scripts: repro_model_lifecycle.py (TabICL-first, the clean path) and repro_model_lifecycle_xgboost_first.py (XGBoost-first, reproduces the #05-013 cascade). Run: uv run python scripts/repro_model_lifecycle/repro_model_lifecycle.py (env: SDK_ACCESS_TOKEN, LEXSI_WORKSPACE_NAME, LEXSI_ORG_NAME).

Live SDK findings from the lifecycle repro (last run 13 OK / 4 FAIL):

• #05-013 — classic-ML (XGBoost) explainability scans every stored column, including the server-injected tag column, and fails float() on a non-numeric one. The SDK uploads clean data + an explicit feature_include (tabular.py:285,368); explainability ignores it. Model still builds (non-fatal), but upload_data raises and rolls back the upload → cascade (train_model → "Upload files first", predict upload → "Project Config is required"). TabICL's foundation path doesn't trip it. Fix for the agent: treat #05-013 as success-with-warning, and train foundation-first / drop or encode non-numeric columns.
• evals_tabular(<foundation model>) → Exception: 'model' (KeyError) — works for classic ML, breaks for TabICL. Blocks compare_models on foundation models.
• get_model_performance(...) → 'utf-8' codec can't decode … — response decode bug. Blocks monitor_performance.
• per-case XAI (case_predict, get_feature_importance) needs (a) the model active + an inference run first (else "Inference status inactive"), and (b) SHAP computed at train (xai=["shap"]) — which itself trips #05-013 for classic ML. case_predict(risk_policies=True) works without SHAP.

The five lifecycle tools degrade gracefully against these (per-model error capture in compare_models; actionable ok=False in monitor_performance). Record new findings in sdk_issues.md.

---

14. Deploy

• Docker (Dockerfile): python:3.10-slim, CPU-only torch, pinned deps, bakes artifacts/financial.duckdb (the PKDD source TSVs are Git-LFS pointers), serves Gradio on 7850. Build for the platform arch: docker buildx build --platform linux/amd64 --provenance=false --sbom=false -t bplexsi/lexsi-ds-agent:prod_vN --push .
• HF Space — a Gradio Space (README.md frontmatter sdk: gradio, app_file: app/gradio_app.py); the Dockerfile is ignored there.
• Lexsi platform — config.yaml: one container on port 7850; the real SDK_ACCESS_TOKEN goes in the platform env tab (the repo keeps a placeholder).

---

15. Sharp edges (things that bit us)

• No LLM timeout = frozen run. Always keep _call_with_timeout in the path (§9).
• Tool arg-name drift. The planner reuses sibling names (table_or_label, source_columns, prompt); mismatched schemas → validation_error loops. Alias them.
• DataFrame truthiness. x = cache.get(a) or cache.get(b) raises on a DataFrame; use explicit is None.
• String-typed dates/numbers. Source columns are often VARCHAR; rely on the ColumnInfo.sample + the TRY_CAST/try_strptime guidance in the text_to_sql prompt.
• DB cache contamination. The master must stay pristine — agents write to the working copy, never the master (§7).
• artifacts/sessions/ growth. Working copies/pgdata are cleaned now; if you add a loader, clean up after it.
• Single-run bench numbers are noisy (DAB sits in an 8–9/17 band, 11/17 ceiling). Use --iterations and read the per-task/failure breakdown, not the headline.

---

16. "Where do I look for…"

Want to…	Go to
change the planner behavior / rules	prompts.py (PLANNER_SYSTEM)
add/modify a tool	tools/<name>.py + tools/__init__.py (+ _TOOL_GROUPS)
change which tools the planner sees	prompts.py selector (_CORE_TOOLS, _TOOL_GROUPS)
connect a new datasource	datasource.py (loaders) + connect_datalake tool
change LLM / caching / timeout	llm/client.py
run/extend DAB	benchmark/dab/{adapter,runner}.py
run/extend PKDD	benchmark/pkdd_runner.py + bench/pkdd/questions.yaml
regenerate PKDD questions	benchmark/datagen/
model-lifecycle tools (list/select/compare/drift/monitor)	tools/{list_available_models,select_active_model,compare_models,check_drift,monitor_performance}.py + tools/_tabular_common.py
probe the live SDK / file an SDK bug	scripts/repro_model_lifecycle/, scripts/repro_tabicl/ → docs/sdk_issues.md
the UI	app/gradio_app.py
roadmap / what's next	v1_tools.md, tools_remaining_and_dab_hardening.md