File size: 7,055 Bytes
d3d0e0e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
# Executive Visualization - Usage Guide

This guide shows how to use the new executive-friendly visualization features for stakeholder demonstrations.

## 🎯 Overview

The visualization module transforms technical logging into business-friendly reports that highlight:
- **Decision Reasoning**: Why the agent chose a particular approach
- **Execution Journey**: Timeline of attempts and refinements
- **Confidence Metrics**: Quality indicators based on execution patterns
- **Clear Outcomes**: Success/failure with actionable insights

## 📋 Commands Available

### 1. Run Task with Executive Mode (Live Demo)

**Use this for:** Live demonstrations to executives

```bash
# Run with executive-friendly display (modern, clean output)
uv run dabench run-lang-task task_355 --config configs/react_baseline.azure.yaml --display-mode=executive

# Run with technical display (default - detailed logs)
uv run dabench run-lang-task task_355 --config configs/react_baseline.azure.yaml --display-mode=technical
```

**What you see:**
- Executive mode: Clean panels showing strategy, execution timeline, and results
- Technical mode: Original detailed logging (unchanged, default behavior)

### 2. View Executive Report (Post-Run Analysis)

**Use this for:** Analyzing completed tasks, preparing presentations

```bash
# View report for a specific task from a run
uv run dabench view-exec-report task_355 20260601T075638Z

# Or use full path
uv run dabench view-exec-report task_355 /data3/dataFAIR/kdd-dev/public/artifacts/runs/20260601T075638Z
```

**What you see:**
- 📊 Executive Summary with status and timing
- 🧠 Decision Reasoning (the "why" behind the approach)
- ⚙️ Execution Timeline (refinement cycles)
- 📦 Deliverables (output location)

### 3. Run Multiple Tasks with Executive Display

```bash
# Run multiple tasks for comprehensive demo
uv run dabench run-lang-task task_355 task_330 task_418 \
  --config configs/react_baseline.azure.yaml \
  --display-mode=executive
```

## 🎨 Visual Style

The executive mode uses:
- ✅ ❌ Status emojis for quick scanning
- 🔍 📋 ⚙️ Phase indicators
- 🟢 🟡 🟠 🔴 Confidence color coding
- Rounded panels with clear typography
- Business terminology (not technical jargon)

## 📊 Example Output

### Executive Mode Output

```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 Executive Summary: task_355
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────── Overview ────────────╮
│ Status              ✅ Success   │
│ Completion Time     ⏱️  1m 41s   │
│ Refinement Cycles   🔄 5 attempts│
│ Confidence Level    🟠 Moderate  │
╰─────────────────────────────────╯

╭──────────── 🧠 Decision Reasoning ─────────────╮
│ 💡 Strategy: Find records where expense       │
│ description matches 'Water, Veggie tray and   │
│ supplies', then connect to member records to  │
│ retrieve first_name, last_name, and cost.     │
│                                                │
│ 📋 Target Output: first_name, last_name, cost │
│ 📊 Expected Result: single row(s)             │
╰────────────────────────────────────────────────╯

⚙️  Execution & Refinement Timeline
┌─────────┬──────────┬──────────────────────────────┐
│ Attempt │  Status  │ Insight / Action Taken       │
├─────────┼──────────┼──────────────────────────────┤
│   #1    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #2    │ ❌ Refine│ Search criteria too          │
│         │          │ restrictive - broadening     │
│   #3    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #4    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #5    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
└─────────┴──────────┴──────────────────────────────┘

╭────────── 📦 Deliverables ───────────╮
│ 📁 Results File    .../prediction.csv│
│ 📊 Output Columns  first_name,       │
│                    last_name, cost   │
╰──────────────────────────────────────╯
```

## 🔄 Migration Notes

### Backward Compatibility
- **Default behavior unchanged**: All existing commands work exactly as before
- **Opt-in only**: Executive mode is only active when explicitly requested
- **No breaking changes**: Technical mode is still the default

### When to Use Each Mode

| Audience | Recommended Mode | Why |
|----------|-----------------|-----|
| Software Engineers | `technical` | Full debugging details, logs, stack traces |
| Data Scientists | `technical` | Technical validation, performance metrics |
| Senior Executives | `executive` | Business outcomes, decision reasoning |
| Stakeholder Demos | `executive` | Clear narrative, visual appeal |
| Post-Run Analysis | Use `view-exec-report` | Polished presentation of results |

## 🚀 Quick Start for Demo

For your executive demonstration:

```bash
# 1. Run a task with executive display
uv run dabench run-lang-task task_355 \
  --config configs/react_baseline.azure.yaml \
  --display-mode=executive

# 2. Or view a report for an already-completed run
uv run dabench view-exec-report task_355 20260601T075638Z
```

## 📁 Files Created

```
src/data_agent_baseline/visualization/
├── __init__.py              # Module exports
├── trace_parser.py          # Parse trace.json structure
├── executive_reporter.py    # Generate formatted reports
├── narratives.py            # Technical → Business language mapping
└── live_display.py          # Live demo display (future enhancement)
```

## 🔧 Technical Details

- **trace.json parsing**: Extracts decision reasoning, execution attempts, timing
- **Narrative translation**: Maps technical actions to business-friendly descriptions
- **Rich formatting**: Uses Rich library for beautiful terminal output
- **No side effects**: Read-only operations, doesn't modify any data