docs/overview/Phase_2.md

# Phase 2 Delta Guide (vs Phase 1 Baseline)

## Executive Summary (For Leadership)
Phase 2 represents a substantial platform maturity jump from a baseline single-path agent into a production-oriented, evaluable, and auditable system.

### Business Outcomes
- Higher delivery confidence through stronger validation and failure detection (evaluation v2 + consistency checks).
- Better explainability for stakeholders via expanded trace, reporting, and visualization surfaces.
- Improved demo and communication readiness through executive-focused output and narrative tooling.
- Stronger submission and release posture with clearer packaging, reproducibility, and operational checks.

### What Changed at a High Level
- Architecture moved from a narrower baseline loop to a modular orchestration model with analyst-team capabilities.
- Measurement moved from limited metrics to a broader, hardened evaluation framework with auditability.
- Documentation shifted from sparse technical notes to a full knowledge base covering architecture, trace contracts, audits, and stabilization evidence.

### Leadership Readout
- Scope increase: 89 files changed (73 added, 16 modified).
- Dominant investment area: core orchestration and evaluation stack under src/.
- Risk profile: reduced execution and reporting risk due to stronger verification and diagnostics; complexity risk increased due to larger system surface.

### Decision Implications
- Team onboarding should prioritize architecture + evaluation docs before deep code dives.
- Future roadmap should favor reliability and operability guardrails before adding further capability breadth.
- Release governance should treat evaluation consistency and trace quality as ship gates.

```mermaid
flowchart LR
  P1[Phase 1: Baseline Agent]
  P2[Phase 2: Scaled Platform]

  P1 --> P2

  P2 --> O1[Outcome: Better reliability]
  P2 --> O2[Outcome: Better explainability]
  P2 --> O3[Outcome: Better demo readiness]
  P2 --> O4[Outcome: Better submission readiness]
```

## Scope Definition
This document describes the delta as:
- Base: branch dev/first-run-baseline
- Target: branch ph2
- Diff source used: git diff --name-status dev/first-run-baseline...ph2

If you hand this to a teammate, they can treat this as the practical change log for everything introduced by ph2 over Phase 1 baseline.

## Executive Delta Summary
- Total files changed: 89
- Added: 73
- Modified: 16
- Net code/docs expansion: 34,264 insertions, 202 deletions
- Main impact area: src/data_agent_baseline/langgraph_agent (43 files)

```mermaid
flowchart LR
  P1[Phase 1 Baseline]
  PH2[Phase 2 ph2 Branch]

  P1 -->|+73 files| PH2
  P1 -->|16 files modified| PH2

  PH2 --> A[LangGraph AAT stack]
  PH2 --> B[Eval v2 stack]
  PH2 --> C[Visualization stack]
  PH2 --> D[Trace canonicalization docs]
  PH2 --> E[Audit and stabilization docs]
```

## Change Composition

```mermaid
pie showData
  title File Operation Split (ph2 vs baseline)
  "Added" : 73
  "Modified" : 16
```

```mermaid
pie showData
  title Top-Level Change Distribution
  "src/" : 55
  "Root docs/config/scripts" : 34
```

## Architectural Delta
Phase 2 introduces a major platform shift from a tighter baseline execution path to an expanded multi-module system with analyst-team logic, evaluation hardening, and executive/trace observability.

```mermaid
flowchart TD
  subgraph P1[Phase 1 Baseline Shape]
    P1A[CLI/Runner]
    P1B[Core agent loop]
    P1C[Basic tools]
    P1D[Baseline artifacts]
    P1A --> P1B --> P1C --> P1D
  end

  subgraph P2[Phase 2 Expanded Shape]
    C1[CLI extensions]
    C2[LangGraph orchestration]
    C3[AAT modules]
    C4[Eval v2 + validators]
    C5[Visualization + narratives]
    C6[Canonical trace and audits]
    C7[Submission hardening]
    C1 --> C2 --> C3
    C2 --> C4
    C2 --> C5
    C3 --> C6
    C4 --> C6
    C4 --> C7
  end

  P1 --> P2
```

## Source Code Delta by Area

### 1) New LangGraph/AAT Core (largest delta)
Major new package footprint under [src/data_agent_baseline/langgraph_agent](src/data_agent_baseline/langgraph_agent).

Key additions include:
- Graph/runtime/state: [src/data_agent_baseline/langgraph_agent/graph.py](src/data_agent_baseline/langgraph_agent/graph.py), [src/data_agent_baseline/langgraph_agent/nodes.py](src/data_agent_baseline/langgraph_agent/nodes.py), [src/data_agent_baseline/langgraph_agent/state.py](src/data_agent_baseline/langgraph_agent/state.py), [src/data_agent_baseline/langgraph_agent/runner.py](src/data_agent_baseline/langgraph_agent/runner.py)
- AAT layer: [src/data_agent_baseline/langgraph_agent/aat](src/data_agent_baseline/langgraph_agent/aat)
- Guarding/normalization/scouting: [src/data_agent_baseline/langgraph_agent/numeric_guards.py](src/data_agent_baseline/langgraph_agent/numeric_guards.py), [src/data_agent_baseline/langgraph_agent/post_hoc_guards.py](src/data_agent_baseline/langgraph_agent/post_hoc_guards.py), [src/data_agent_baseline/langgraph_agent/scout.py](src/data_agent_baseline/langgraph_agent/scout.py), [src/data_agent_baseline/langgraph_agent/normalization.py](src/data_agent_baseline/langgraph_agent/normalization.py)
- Prompting/semantic logic: [src/data_agent_baseline/langgraph_agent/prompts.py](src/data_agent_baseline/langgraph_agent/prompts.py), [src/data_agent_baseline/langgraph_agent/semantic.py](src/data_agent_baseline/langgraph_agent/semantic.py)

```mermaid
flowchart LR
  Q[Question] --> EX[Explore]
  EX --> PL[Planner]
  PL --> CP[Critic Plan]
  CP -->|approved| EXE[Execute]
  EXE --> CE[Critic Execute]
  CE -->|needs replan| PL
  CE -->|ok| ANS[Answer]

  EXE --> AATC[AAT Coordinator]
  AATC --> S1[Schema Specialist]
  AATC --> S2[Domain Specialist]
  AATC --> S3[Hypothesis Specialist]
  AATC --> S4[Verifier]
  S1 --> AATC
  S2 --> AATC
  S3 --> AATC
  S4 --> AATC
```

### 2) Evaluation v2 and Validation Stack
New evaluation system and consistency validators were added, including:
- [src/data_agent_baseline/langgraph_agent/eval_v2.py](src/data_agent_baseline/langgraph_agent/eval_v2.py)
- [src/data_agent_baseline/langgraph_agent/eval_v2_metrics.py](src/data_agent_baseline/langgraph_agent/eval_v2_metrics.py)
- [src/data_agent_baseline/langgraph_agent/eval_v2_validator.py](src/data_agent_baseline/langgraph_agent/eval_v2_validator.py)
- [src/data_agent_baseline/langgraph_agent/evaluation_consistency_validator.py](src/data_agent_baseline/langgraph_agent/evaluation_consistency_validator.py)
- [src/data_agent_baseline/langgraph_agent/comprehensive_evaluator.py](src/data_agent_baseline/langgraph_agent/comprehensive_evaluator.py)

```mermaid
flowchart TD
  R[Run artifacts] --> N[Normalization]
  N --> M[Metric computation]
  M --> V[Validator checks]
  V --> H[Health report]
  M --> REP[Markdown/CSV reports]
  V --> REP
  REP --> DEC[Release decision]
```

### 3) Visualization and Executive Reporting
Added visualization package:
- [src/data_agent_baseline/visualization/live_display.py](src/data_agent_baseline/visualization/live_display.py)
- [src/data_agent_baseline/visualization/executive_reporter.py](src/data_agent_baseline/visualization/executive_reporter.py)
- [src/data_agent_baseline/visualization/narratives.py](src/data_agent_baseline/visualization/narratives.py)
- [src/data_agent_baseline/visualization/trace_parser.py](src/data_agent_baseline/visualization/trace_parser.py)

```mermaid
flowchart LR
  T[Trace + Eval artifacts] --> P[Trace parser]
  P --> L[Live display]
  P --> N[Narratives]
  P --> E[Executive reporter]
  L --> OUT[Demo-friendly output]
  N --> OUT
  E --> OUT
```

### 4) Existing Core Files Modified
Notable modifications to core integration points:
- [src/data_agent_baseline/cli.py](src/data_agent_baseline/cli.py)
- [main.py](main.py)
- [src/data_agent_baseline/config.py](src/data_agent_baseline/config.py)
- [src/data_agent_baseline/run/runner.py](src/data_agent_baseline/run/runner.py)
- [src/data_agent_baseline/tools/sqlite.py](src/data_agent_baseline/tools/sqlite.py)
- [src/data_agent_baseline/tools/filesystem.py](src/data_agent_baseline/tools/filesystem.py)
- [src/data_agent_baseline/tools/registry.py](src/data_agent_baseline/tools/registry.py)

```mermaid
flowchart TD
  CLI[CLI surface expansion] --> CFG[Config/runtime flags]
  CFG --> RUN[Runner behavior]
  RUN --> TOOLS[Tooling integration]
  RUN --> TRACE[Trace/eval artifacts]
  TRACE --> EVAL[Eval v2 pipeline]
```

## Documentation Delta (Reviewed Across All Repo MD Files)
Repository markdown files reviewed (excluding virtualenv/cache): 37.

Markdown files changed in ph2 delta: 22 (mostly added).

Main doc clusters added/updated:
- Baseline + canonical trace docs:
  - [BASELINE_ARCHITECTURE.md](BASELINE_ARCHITECTURE.md)
  - [BASELINE_TRACE_SCHEMA.md](BASELINE_TRACE_SCHEMA.md)
  - [CANONICAL_TRACE_FORMAT.md](CANONICAL_TRACE_FORMAT.md)
  - [CANONICAL_TRACE_IMPLEMENTATION.md](CANONICAL_TRACE_IMPLEMENTATION.md)
  - [CANONICAL_TRACE_QUICKREF.md](CANONICAL_TRACE_QUICKREF.md)
- Evaluation and cleanup docs:
  - [EVALUATION.md](EVALUATION.md)
  - [EVAL_CLEANUP_FIXES_SUMMARY.md](EVAL_CLEANUP_FIXES_SUMMARY.md)
  - [PHASE1_EVALUATION_GUIDE.md](PHASE1_EVALUATION_GUIDE.md)
  - [PHASE1_IMPLEMENTATION_SUMMARY.md](PHASE1_IMPLEMENTATION_SUMMARY.md)
- Architecture/audit/verification docs:
  - [Overview.md](Overview.md)
  - [phase2_verification_audit.md](phase2_verification_audit.md)
  - [evaluation_consistency_audit.md](evaluation_consistency_audit.md)
  - [mas_alignment_audit.md](mas_alignment_audit.md)
  - [recovery_diff_audit.md](recovery_diff_audit.md)
  - [stabilization_report.md](stabilization_report.md)
  - [architecture_simplification.md](architecture_simplification.md)
  - [analyst_team_calibration_fixes.md](analyst_team_calibration_fixes.md)
  - [analyst_team_disagreement_layer.md](analyst_team_disagreement_layer.md)
- Visualization/demo/submission docs:
  - [EXECUTIVE_VISUALIZATION_GUIDE.md](EXECUTIVE_VISUALIZATION_GUIDE.md)
  - [PoC_Output.md](PoC_Output.md)
  - [SUBMISSION_CREATION_GUIDE.md](SUBMISSION_CREATION_GUIDE.md)
  - [insights.md](insights.md)

```mermaid
mindmap
  root((Phase 2 docs delta))
    Architecture
      Overview.md
      BASELINE_ARCHITECTURE.md
      architecture_simplification.md
    Trace
      BASELINE_TRACE_SCHEMA.md
      CANONICAL_TRACE_FORMAT.md
      CANONICAL_TRACE_IMPLEMENTATION.md
      CANONICAL_TRACE_QUICKREF.md
    Evaluation
      EVALUATION.md
      PHASE1_EVALUATION_GUIDE.md
      PHASE1_IMPLEMENTATION_SUMMARY.md
      EVAL_CLEANUP_FIXES_SUMMARY.md
    Audit
      phase2_verification_audit.md
      evaluation_consistency_audit.md
      mas_alignment_audit.md
      recovery_diff_audit.md
      stabilization_report.md
    Demo and Ops
      EXECUTIVE_VISUALIZATION_GUIDE.md
      SUBMISSION_CREATION_GUIDE.md
      PoC_Output.md
      insights.md
```

## Operational and Packaging Delta
- Added/updated packaging/runtime files:
  - [.dockerignore](.dockerignore)
  - [Dockerfile](Dockerfile)
  - [Dockerfile.backup](Dockerfile.backup)
  - [Makefile.submit](Makefile.submit)
  - [pyproject.toml](pyproject.toml)
  - [uv.lock](uv.lock)
- Submission helper script changed:
  - [test_submission_locally.sh](test_submission_locally.sh)

```mermaid
flowchart LR
  DEV[Dev environment] --> PKG[pyproject + uv.lock]
  PKG --> IMG[Docker build]
  IMG --> SUB[Submission artifact]
  SUB --> LOCAL[test_submission_locally.sh]
  LOCAL --> READY[Ready for KDD submission]
```

## Suggested Onboarding Path for Teammates
Use this sequence to ramp quickly on ph2 delta:

1. Read high-level architecture and contract:
   - [Overview.md](Overview.md)
   - [architecture_simplification.md](architecture_simplification.md)

2. Understand trace and evaluation model:
   - [CANONICAL_TRACE_FORMAT.md](CANONICAL_TRACE_FORMAT.md)
   - [EVALUATION.md](EVALUATION.md)

3. Inspect implementation hotspots:
   - [src/data_agent_baseline/langgraph_agent/graph.py](src/data_agent_baseline/langgraph_agent/graph.py)
   - [src/data_agent_baseline/langgraph_agent/nodes.py](src/data_agent_baseline/langgraph_agent/nodes.py)
   - [src/data_agent_baseline/langgraph_agent/eval_v2.py](src/data_agent_baseline/langgraph_agent/eval_v2.py)
   - [src/data_agent_baseline/cli.py](src/data_agent_baseline/cli.py)

4. Review audit evidence and hardening:
   - [phase2_verification_audit.md](phase2_verification_audit.md)
   - [evaluation_consistency_audit.md](evaluation_consistency_audit.md)
   - [recovery_diff_audit.md](recovery_diff_audit.md)

```mermaid
gantt
  title Suggested teammate ramp plan
  dateFormat  YYYY-MM-DD
  section Architecture
  Read overview and simplification       :a1, 2026-06-16, 1d
  section Trace and Eval
  Canonical trace + eval docs            :a2, after a1, 1d
  section Code Deep Dive
  Graph and node internals               :a3, after a2, 1d
  Eval v2 and CLI flow                   :a4, after a3, 1d
  section Validation
  Audit docs and regression checkpoints   :a5, after a4, 1d
```

## Practical Delta Checklist (What is new in ph2)
- New orchestrated LangGraph stack with analyst-team modules.
- New evaluation v2 and consistency validator pipeline.
- New visualization and executive reporting package.
- Expanded CLI and runtime integration points.
- Strongly expanded architecture/trace/evaluation/audit documentation corpus.
- Submission and packaging flow updates for reproducible delivery.

## Notes
- This guide is intentionally delta-focused and not a full design spec.
- For complete context, use [Overview.md](Overview.md) as the canonical architecture index and [EVALUATION.md](EVALUATION.md) as the canonical metrics/evaluation index.