CLAUDE.md

Claude Context Guide for Demo Wire Project

SOURCE OF TRUTH: This file is the single source of truth for all AI assistant rules. Both Cursor (via .cursorrules pointer) and Claude CLI read from this file. Make all rule changes here.

---

DemoPrep Core Mission

"The goal of the demo builder is to tell a better story based on different personas, such as a line of business leader or data analyst, and to use a framework around the demo to effectively tell that story."

This is emphasized by leadership. The framework influences:

• Objects (tables, models, liveboards)
• Visualizations (charts, KPIs, layouts)
• Data (including outliers that drive the narrative)

See dev_notes/USE_CASE_FLOW.md for the full use case framework documentation.

---

How to Get Database/Schema for a Demo Run — NEVER ASK THE USER

Given a model GUID or liveboard GUID, derive everything from ThoughtSpot — do not ask.

The full resolution chain is already implemented in load_context_from_liveboard() in smart_data_adjuster.py:

1. Export liveboard TML (export_fqn=True) → find model GUID in visualizations[n].answer.tables[0].fqn
2. Export model TML → read model.tables[0].table.{db, schema}
3. Connect to Snowflake via get_snowflake_connection() in snowflake_auth.py (keypair auth, credentials in .env)

When user gives you a model URL like https://xyz.thoughtspot.cloud/#/data/tables/<GUID>:

• The GUID is everything after /data/tables/
• Use the ThoughtSpot API (authenticated deployer) to export the model TML
• Read db and schema directly from the TML
• Connect to Snowflake with get_snowflake_connection() — keypair is already configured

NEVER ask "what's the database name?" or "what's the schema?" — derive it from the model.

For scratch/fix scripts, follow the pattern in chat_interface.py lines ~415-440 (liveboard-first entry point).

---

CRITICAL: NEVER PUSH WITHOUT EXPLICIT INSTRUCTION

Do NOT push to any remote (origin or hf) unless the user explicitly says to push.

• DO NOT push after making code changes, even if they're working
• DO NOT push "to keep things in sync" or as part of a commit workflow
• ONLY push when user says: "push", "push to HF", "push to origin", "push it", etc.
• This applies especially to the hf remote — that is the live production environment

The user may be running a live demo on HF. Pushing without permission can break an active demo.

---

CRITICAL PROCESS MANAGEMENT RULE

NEVER restart the application unless explicitly requested

• DO NOT run commands like kill, lsof -ti:7862 | xargs kill, or restart scripts
• DO NOT execute launch_chat.py, chat_interface.py or similar startup commands
• When making code changes, say "Changes saved to [file] - restart when ready" and STOP
• ONLY restart if user explicitly says: "restart", "rerun the app", "relaunch", "kill and restart"
• If user says "don't stop the process" or "keep it running" - NEVER restart, no matter what

---

Current Sprint

ALWAYS READ FIRST: sprint_2026_03.md (in root)

For overall project status, see PROJECT_STATUS.md in root.

Sprint Doc Workflow - IMPORTANT

This is how we stay organized. Follow this process consistently:

1. At the start of each session, read the current sprint doc to understand what's in progress
2. As you work, update the sprint doc in real-time:
   • Mark tasks [x] complete as you finish them
   • Add brief notes on what was done (e.g., "✅ - _to_snake_case() in deployer")
   • Move completed items to "Done" section
   • Keep "In Progress" and "To Do" current
3. Before ending a session, make sure the sprint doc reflects current state

This creates a living record of progress and makes handoffs easy. Don't let the sprint doc get stale.

Example:

### Done
- [x] Outlier SQL generation core ✅ - `outlier_system.py` created
- [x] snake_case naming ✅ - `_to_snake_case()` in deployer

Previous sprints: sprint_2026_02.md (February/March - closed), sprint_2026_01.md (January - closed)

---

Contact & Communication

• User (boone) is a ThoughtSpot Architect with 20+ years experience
• You can challenge ideas, but respect deep domain knowledge
• When asked to STOP - stop immediately and wait for further instructions
• Start discussions with back-and-forth using numbered questions
• TS = ThoughtSpot (the company where boone works)
• Ask rather than assume when in doubt

---

Debugging and Problem-Solving Protocol

Confidence and Communication

1. Don't act over-confident unless you're extremely sure

   • Check documentation before claiming you know how something works
   • Say "I don't know" or "Let me check" instead of guessing
   • If you're uncertain, say so upfront

2. NEVER claim something is fixed until it's tested

   • ❌ WRONG: "I've fixed the tags issue" (without testing)
   • ✅ RIGHT: "I've updated the code - let's test if tags work now"
   • Show the test results, don't just assume it works

3. When debugging:

   • Check documentation FIRST before blaming external systems
   • State what you're checking and why
   • Share what you found (even if it proves you wrong)

4. It's OK to say:

   • "I don't know - should I research this?"
   • "I'm not certain, but here are 2 possibilities..."
   • "Let me verify this works before saying it's fixed"

---

Code Quality Standards

Quality over speed. Always.

1. Do it right the first time — don't take shortcuts that create tech debt

   • No hacks "to avoid touching every file" — touch every file if that's the right approach
   • No stuffing values into os.environ as a proxy for proper settings management
   • No fallback chains (x or y or z or 'default') that mask missing configuration
   • If a setting is required, fail loudly when it's missing

2. No silent fallbacks — if something is missing, error immediately with a clear message

   • ❌ WRONG: value = os.getenv('KEY', 'some_default')
   • ✅ RIGHT: value = get_admin_setting('KEY') → raises error if empty
   • User identity is required: Never use fallback users like default@user.com or USER_EMAIL env defaults for settings. If authenticated username/email is missing, fail immediately with a clear error.

3. Single source of truth — every piece of data should have one canonical source

   • Admin settings live in Supabase, read via get_admin_setting()
   • User settings live in Supabase, loaded per-user
   • .env is bootstrap only (Supabase connection creds)

4. Don't sacrifice quality for speed — if the right approach takes longer, do it anyway. The goal is clean, maintainable code, not "slop that works."

---

Code Quality Standards

Quality over speed. Always.

1. Do it right the first time - don't take shortcuts that create tech debt

   • No hacks "to avoid touching every file" - touch every file if that's the right approach
   • No stuffing values into os.environ as a proxy for proper settings management
   • No fallback chains (x or y or z or 'default') that mask missing configuration
   • If a setting is required, fail loudly when it's missing

2. No silent fallbacks - if something is missing, error immediately with a clear message

   • WRONG: value = os.getenv('KEY', 'some_default')
   • RIGHT: value = get_admin_setting('KEY') raises error if empty
   • User identity is required: Never use fallback users like default@user.com or USER_EMAIL env defaults for settings. If authenticated username/email is missing, fail immediately with a clear error.

3. Single source of truth - every piece of data should have one canonical source

   • Admin settings live in Supabase, read via get_admin_setting()
   • User settings live in Supabase, loaded per-user
   • .env is bootstrap only (Supabase connection creds)

4. Don't sacrifice quality for speed - if the right approach takes longer, do it anyway. The goal is clean, maintainable code, not slop that works.

---

Key Project Facts

Environment Setup

• Python: Using virtual environment at ./demoprep/bin/activate
• NOT using conda - if you don't see packages, activate the venv first
• Supabase: IS installed and configured - credentials in .env work fine
• ThoughtSpot: Authentication works with demo_builder_user

Quick Commands

# Run the app properly
source ./demoprep/bin/activate && python chat_interface.py

# Check git changes
git diff --stat

# Find processes using port
lsof -i :7860

Common Mistakes to Avoid

1. DO NOT add unnecessary validation checks for .env variables - they are populated and working
2. DO NOT try to install supabase or other packages - they're already in the venv
3. DO NOT change default values to "thoughtspot.com" - that's for customer URLs, not ThoughtSpot's
4. DO NOT assume worksheets are needed - they're deprecated, we use models now
5. ALWAYS use the venv when running Python: source ./demoprep/bin/activate && python

Working Patterns

• Settings save/load through Supabase - this WORKS when using venv
• ThoughtSpot TML IS YAML format (use yaml.dump() not json.dumps())
• Models have replaced worksheets in modern ThoughtSpot
• Liveboards should match "golden demo" style

---

File Organization Rules - CRITICAL

NEVER create files in root without asking first

tests/ - Real, reusable test cases only

• Unit tests for core functions
• Integration tests that could be automated
• Tests you'd run as part of CI/CD

scratch/ - ALL temporary/experimental/debug files

• ALL experimental/debug/check/verify/analyze scripts
• One-off fixes (fix_*.py, adjust_*.py, emergency_*.py)
• Debug scripts (debug_*.py, check_*.py, verify_*.py)
• Analysis tools (analyze_*.py, get_*.py, show_*.py)
• Test files you're experimenting with
• Backup files (.bak, .bak2)
• Export/debug .yml/.json files
• ANY script that's temporary or one-time use
• DO NOT commit without cleanup/review

dev_notes/ - All documentation and presentations

• All .md files (except README.md, CLAUDE.md, PROJECT_STATUS.md in root)
• Presentation materials (.pptx, .html, .txt slides)
• Research documents
• Architecture notes
• Sprint planning documents

Root directory - ONLY essential core application files

• Main entry points (chat_interface.py, launch_chat.py, thoughtspot_deployer.py)
• Core interfaces (chat_interface.py, liveboard_creator.py)
• Utilities (supabase_client.py, schema_utils.py, demo_logger.py, etc.)
• Configuration (.env, requirements.txt)
• README.md, CLAUDE.md, PROJECT_STATUS.md only
• DO NOT create random files here without asking

Simple Decision Tree for New Files

Creating a new file? Ask yourself:

1. Is it a real test that should be automated? → tests/
2. Is it documentation or presentation material? → dev_notes/
3. Is it core application code? → Root (but ASK FIRST!)
4. Everything else? → scratch/

Golden Rule: When in doubt, PUT IT IN SCRATCH

---

Testing Existing Features

CRITICAL: DO NOT create simplified versions of working code

When user says "create a test for X":

• ❌ WRONG: Write new simplified code from scratch
• ✅ RIGHT: Call the existing working function in a test harness

Working Code Paths - DO NOT BYPASS THESE:

Liveboard Creation:
  HYBRID: chat_interface.py → create_liveboard_from_model_mcp() → enhance_mcp_liveboard()

DO NOT use create_visualization_tml() directly - that's internal low-level code

---

Liveboard Creation — HYBRID Method

Single method: HYBRID (MCP + TML post-processing)

Pipeline

1. MCP creates liveboard via agent.thoughtspot.app (bearer auth)
2. enhance_mcp_liveboard() post-processes with groups, KPIs, colors, layout

• Entry: create_liveboard_from_model_mcp() → enhance_mcp_liveboard()

Post-Processing: enhance_mcp_liveboard()

1. Exports the liveboard TML
2. Classifies visualizations by type (KPI, trend, categorical)
3. Adds Groups with proper group_layouts (Golden Demo style)
4. KPI group always at top (y=0, full width)
5. Fixes KPI sparklines and comparisons
6. Applies brand colors (liveboard-level + group-level)
7. Re-imports the enhanced TML

Spotter Viz Story (Post-Liveboard Output)

After liveboard creation, a Spotter Viz Story is generated:

• A conversational sequence of natural language prompts for ThoughtSpot's Spotter Viz agent
• Uses company context, use case, outlier patterns, and liveboard visualizations
• Displayed in the "Spotter Viz Story" tab in the app
• Can be manually entered into Spotter Viz to recreate/refine the liveboard
• Future: will be automated when the Spotter Viz API is published

KPI Requirements

• For sparklines and percent change comparisons:
  • Must include time dimension (date column)
  • Must specify granularity (daily, weekly, monthly, quarterly, yearly)
  • Example: [Total_revenue] [Order_date].monthly
• Post-processing adds sparkline settings automatically

Terminology (Important!)

• Outliers = Interesting data points in existing data
• Data Adjuster = Modifying data values (needs Snowflake views)

Golden Demo Structure

• Location: dev_notes/liveboard_demogold2/🏬 Global Retail Apparel Sales (New).liveboard.tml
• Uses GROUPS (like tabs) NOT text tiles
• Groups organize visualizations by theme
• Brand colors via style_properties (GBC_A-J for groups, TBC_A-J for tiles)
• KPI structure: [sales] [date].weekly [date].'last 8 quarters'

---

Frustration Points (AVOID)

User gets frustrated when you:

• Don't trust that .env variables are set correctly
• Try to reinstall packages that are already installed
• Make changes without understanding existing context
• Break working code by "simplifying" it
• RESTART THE APPLICATION WITHOUT PERMISSION
• Jump ahead and make changes without discussing first

---

Project Context

• Software stored in boone's repo, will be open sourced to TS repo
• This is a living project - update understanding as you learn
• See PROJECT_STATUS.md for overall project state
• See current sprint doc for active work

---

Last Updated: February 4, 2026 This is the source of truth - update rules here, not in .cursorrules