feat: add Spotter Viz as second liveboard creation method

Implements the Liveboard Creation Design Doc:
- Settings dropdown: ["HYBRID", "SPOTTER_VIZ"] replaces old 3-option menu
- SPOTTER_VIZ routing: direct REST API + TML, no MCP dependency
- Reuses LiveboardCreator.create_liveboard_tml() + deploy_liveboard()
- Gets outlier patterns from vertical×function system
- Gets model columns from ThoughtSpot model metadata
- Same enhance_mcp_liveboard() post-processing as HYBRID
- Backward compat: old "TML"/"MCP" settings mapped to "HYBRID"
- create_liveboard_from_model() extended with outliers + model_columns params
- CLAUDE.md updated with two-method system documentation

Co-authored-by: Cursor <cursoragent@cursor.com>

CLAUDE.md

@@ -194,34 +194,49 @@ When user says "create a test for X":
 **Working Code Paths - DO NOT BYPASS THESE:**
 ```
 Liveboard Creation:
-  chat_interface.py → create_liveboard_from_model_mcp() → enhance_mcp_liveboard()
+  HYBRID: chat_interface.py → create_liveboard_from_model_mcp() → enhance_mcp_liveboard()
+  SPOTTER_VIZ: chat_interface.py → create_liveboard_from_model() → enhance_mcp_liveboard()
 
 DO NOT use create_visualization_tml() directly - that's internal low-level code
 ```
 
 ---
 
-## Liveboard Creation
+## Liveboard Creation - Two-Method System
 
-Liveboard creation is a single unified process with two phases:
+**Settings UI:** Admin tab → "Liveboard Creation Method" dropdown
+**Two options:** HYBRID (default) and SPOTTER_VIZ
 
-1. **MCP Creation** - Uses ThoughtSpot's AI (via Model Context Protocol at `agent.thoughtspot.app`) to generate smart visualizations from natural language questions
-2. **TML Post-Processing** - Enhances the AI-created liveboard with groups, KPI sparklines, brand colors, and layout refinement
+| Method | Speed | Dependency | Best For |
+|--------|-------|------------|----------|
+| **HYBRID** | ~60-90s | MCP server | AI-driven question selection |
+| **SPOTTER_VIZ** | ~20-30s | Direct REST API only | Production demos, reliability |
 
-These are implemented as separate functions but are **one process** - do NOT treat them as separate "methods" or offer the user a choice between them.
+### HYBRID Method (MCP + TML Post-Processing)
+1. MCP creates liveboard via `agent.thoughtspot.app` (bearer auth)
+2. TML post-processing enhances with groups, KPIs, colors, layout
+- **Entry:** `create_liveboard_from_model_mcp()` → `enhance_mcp_liveboard()`
 
-### Key Functions (liveboard_creator.py)
-- **`create_liveboard_from_model_mcp()`** - Main entry point. Handles MCP creation.
-- **`enhance_mcp_liveboard()`** - Post-processing. Exports TML, enhances, re-imports.
-- **`LiveboardCreator` class** - TML utilities used during post-processing.
+### SPOTTER_VIZ Method (Direct API + TML)
+1. LiveboardCreator builds complete TML from outlier patterns + AI
+2. Deploys via REST API `/metadata/tml/import`
+3. Same TML post-processing as HYBRID (groups, KPIs, colors, layout)
+- **Entry:** `create_liveboard_from_model()` → `enhance_mcp_liveboard()`
+- No MCP dependency — uses same auth as table/model deployment
 
-### enhance_mcp_liveboard() Details
-1. Exports the MCP-created liveboard TML
+### Shared Post-Processing: enhance_mcp_liveboard()
+Both methods share the same post-processing function:
+1. Exports the liveboard TML
 2. Classifies visualizations by type (KPI, trend, categorical)
-3. Adds Groups (tabs) to organize by type
-4. Fixes KPI client_state_v2 for sparklines and comparisons
-5. Applies brand colors to groups and tiles
-6. Re-imports the enhanced TML
+3. Adds Groups with proper `group_layouts` (Golden Demo style)
+4. KPI group always at top (y=0, full width)
+5. Fixes KPI sparklines and comparisons
+6. Applies brand colors (liveboard-level + group-level)
+7. Re-imports the enhanced TML
+
+### Backward Compatibility
+- Old settings values "TML" or "MCP" are mapped to "HYBRID" in the router
+- `USE_MCP_LIVEBOARD=true` env var still works (maps to HYBRID)
 
 ### KPI Requirements
 - **For sparklines and percent change comparisons:**

chat_interface.py

@@ -4133,9 +4133,9 @@ def create_settings_tab():
             
             liveboard_method = gr.Dropdown(
                 label="Liveboard Creation Method",
-                choices=["HYBRID", "MCP", "TML"],
+                choices=["HYBRID", "SPOTTER_VIZ"],
                 value="HYBRID",
-                info="HYBRID (recommended): MCP + TML polish. MCP: AI-driven. TML: Template-based."
+                info="HYBRID: MCP + TML polish (AI-driven). SPOTTER_VIZ: Direct API + TML (faster, no MCP dependency)."
             )
     
     # Existing Model Section

liveboard_creator.py

@@ -2505,12 +2505,15 @@ def create_liveboard_from_model(
     use_case: str,
     num_visualizations: int = 6,
     liveboard_name: str = None,
-    llm_model: str = None
+    llm_model: str = None,
+    outliers: Optional[List[Dict]] = None,
+    model_columns: Optional[List[Dict]] = None
 ) -> Dict:
     """
-    Convenience function to create and deploy a Liveboard via TML
+    Create and deploy a Liveboard via TML (Spotter Viz path).
     
-    This is the pure TML approach - full control over visualizations via YAML templates.
+    This is the direct API approach — builds complete TML and imports it.
+    Used by both the legacy TML path and the new SPOTTER_VIZ method.
 
     Args:
         ts_client: Authenticated ThoughtSpotDeployer instance
@@ -2521,6 +2524,8 @@ def create_liveboard_from_model(
         num_visualizations: Number of visualizations to create
         liveboard_name: Optional custom name for the liveboard
         llm_model: LLM model to use for AI-driven viz generation
+        outliers: Optional list of outlier pattern dicts for targeted viz generation
+        model_columns: Optional list of column metadata from TS model
 
     Returns:
         Deployment result dictionary with:
@@ -2529,13 +2534,24 @@ def create_liveboard_from_model(
             - liveboard_name: str
             - liveboard_url: str
     """
-    print(f"🚀 Starting TML liveboard creation...", flush=True)
+    print(f"🚀 Starting Spotter Viz liveboard creation...", flush=True)
     print(f"   - Model: {model_name}", flush=True)
     print(f"   - Use case: {use_case}", flush=True)
     print(f"   - Visualizations: {num_visualizations}", flush=True)
+    if outliers:
+        print(f"   - Outlier patterns: {len(outliers)}", flush=True)
+    if model_columns:
+        print(f"   - Model columns: {len(model_columns)}", flush=True)
     
     creator = LiveboardCreator(ts_client, model_id, model_name, llm_model)
-    liveboard_tml = creator.create_liveboard_tml(company_data, use_case, num_visualizations, liveboard_name)
+    
+    # If external model_columns provided, override the auto-fetched ones
+    if model_columns:
+        creator.model_columns = model_columns
+    
+    liveboard_tml = creator.create_liveboard_tml(
+        company_data, use_case, num_visualizations, liveboard_name, outliers
+    )
     
     # Deploy and get result
     result = creator.deploy_liveboard(liveboard_tml)

sprint_2026_02.md

@@ -89,6 +89,16 @@
 ### Done
 
 
+#### Feb 6, 2026 - Spotter Viz Method Added (Design Doc Implementation)
+- [x] **Settings dropdown updated**: `["HYBRID", "SPOTTER_VIZ"]` replaces old 3-option dropdown ✅
+- [x] **SPOTTER_VIZ routing in deployer**: Direct TML path with outliers + model_columns ✅
+- [x] **Backward compatibility**: old "TML"/"MCP" values mapped to "HYBRID" ✅
+- [x] **Shared post-processing**: Both methods use `enhance_mcp_liveboard()` ✅
+- [x] **create_liveboard_from_model() extended**: accepts `outliers` and `model_columns` params ✅
+- [x] **CLAUDE.md updated**: Two-method system documented ✅
+- [ ] **To test**: Run SPOTTER_VIZ end-to-end with Retail Sales
+
+
 #### Feb 6, 2026 - Gradio Compat + Hybrid Liveboard Layout Fix
 - [x] **Gradio 4.44.1 + pydantic 2.12 + httpx 0.28 compatibility** ✅
   - Monkey-patch `url_ok()` for localhost check bypass

thoughtspot_deployer.py

@@ -2123,9 +2123,13 @@ class ThoughtSpotDeployer:
                             use_mcp = os.getenv('USE_MCP_LIVEBOARD', 'false').lower() == 'true'
                             method = 'MCP' if use_mcp else 'HYBRID'
                         
-                        # Normalize method name
+                        # Normalize method name and backward compatibility
                         method = method.upper()
-                        if method not in ['TML', 'MCP', 'HYBRID']:
+                        # Map old values to current options
+                        if method in ('TML', 'MCP'):
+                            log_progress(f"[INFO] Mapping legacy method '{method}' → HYBRID")
+                            method = 'HYBRID'
+                        if method not in ['HYBRID', 'SPOTTER_VIZ']:
                             log_progress(f"[WARN] Unknown liveboard method '{method}', defaulting to HYBRID")
                             method = 'HYBRID'
                         
@@ -2140,22 +2144,93 @@ class ThoughtSpotDeployer:
                                 'use_case': use_case or 'General Analytics'
                             }
                             
-                            if method == 'TML':
-                                # Pure TML approach - full control via YAML templates
-                                from liveboard_creator import create_liveboard_from_model
+                            if method == 'SPOTTER_VIZ':
+                                # Spotter Viz: Direct REST API + TML — no MCP dependency
+                                from liveboard_creator import create_liveboard_from_model, enhance_mcp_liveboard
+                                
+                                # Get actual column names from ThoughtSpot model
+                                model_columns = self.get_model_columns(model_guid)
+                                if not model_columns:
+                                    log_progress(f"   ⚠️ Could not get model columns, falling back to DDL")
+                                    model_columns = []
+                                    for table_name, columns_list in tables.items():
+                                        for col in columns_list:
+                                            model_columns.append(col)
                                 
-                                liveboard_result = create_liveboard_from_model(
-                                    ts_client=self,
-                                    model_id=model_guid,
-                                    model_name=model_name,
-                                    company_data=company_data,
-                                    use_case=use_case or 'General Analytics',
-                                    num_visualizations=8,
-                                    liveboard_name=liveboard_name,
-                                    llm_model=llm_model
-                                )
+                                # Get outlier patterns from the vertical×function system
+                                outlier_dicts = []
+                                try:
+                                    from outlier_system import get_outliers_for_use_case
+                                    from demo_personas import parse_use_case
+                                    uc_vertical, uc_function = parse_use_case(use_case or '')
+                                    if uc_vertical or uc_function:
+                                        outlier_config = get_outliers_for_use_case(
+                                            uc_vertical or "Generic", 
+                                            uc_function or "Generic"
+                                        )
+                                        for op in outlier_config.required:
+                                            outlier_dicts.append({
+                                                'title': op.name,
+                                                'insight': op.viz_talking_point,
+                                                'viz_type': op.viz_type,
+                                                'show_me_query': op.viz_question,
+                                                'kpi_companion': True,
+                                                'spotter_questions': op.spotter_questions,
+                                            })
+                                        for op in outlier_config.optional[:2]:
+                                            outlier_dicts.append({
+                                                'title': op.name,
+                                                'insight': op.viz_talking_point,
+                                                'viz_type': op.viz_type,
+                                                'show_me_query': op.viz_question,
+                                                'kpi_companion': False,
+                                                'spotter_questions': op.spotter_questions,
+                                            })
+                                        if outlier_dicts:
+                                            log_progress(f"   [SPOTTER] Using {len(outlier_dicts)} outlier patterns from {uc_vertical}×{uc_function}")
+                                except Exception as outlier_err:
+                                    log_progress(f"   [SPOTTER] Outlier loading skipped: {outlier_err}")
+                                
+                                log_progress(f"   [SPOTTER] Model: {model_name}, GUID: {model_guid}")
+                                log_progress(f"   [SPOTTER] Using {len(model_columns)} columns from ThoughtSpot model")
+                                log_progress(f"   Step 1/2: Building liveboard via TML...")
+                                
+                                try:
+                                    liveboard_result = create_liveboard_from_model(
+                                        ts_client=self,
+                                        model_id=model_guid,
+                                        model_name=model_name,
+                                        company_data=company_data,
+                                        use_case=use_case or 'General Analytics',
+                                        num_visualizations=8,
+                                        liveboard_name=liveboard_name,
+                                        llm_model=llm_model,
+                                        outliers=outlier_dicts if outlier_dicts else None,
+                                        model_columns=model_columns
+                                    )
+                                except Exception as spotter_error:
+                                    import traceback
+                                    error_trace = traceback.format_exc()
+                                    log_progress(f"   [SPOTTER ERROR] {type(spotter_error).__name__}: {str(spotter_error)}")
+                                    liveboard_result = {'success': False, 'error': str(spotter_error), 'traceback': error_trace}
+                                
+                                # Spotter Viz: Add TML enhancement (same as Hybrid post-processing)
+                                if liveboard_result.get('success') and liveboard_result.get('liveboard_guid'):
+                                    log_progress(f"   Step 2/2: Enhancing with TML post-processing...")
+                                    enhance_result = enhance_mcp_liveboard(
+                                        liveboard_guid=liveboard_result['liveboard_guid'],
+                                        company_data=company_data,
+                                        ts_client=self,
+                                        add_groups=True,
+                                        fix_kpis=True,
+                                        apply_brand_colors=True
+                                    )
+                                    if enhance_result.get('success'):
+                                        log_progress(f"   [OK] Enhancement applied: {', '.join(enhance_result.get('enhancements', []))}")
+                                    else:
+                                        log_progress(f"   [WARN] Enhancement partial: {enhance_result.get('message', '')[:80]}")
                                 
-                            elif method in ('MCP', 'HYBRID'):
+                            elif method == 'HYBRID':
                                 # MCP and HYBRID both use MCP for creation
                                 # HYBRID adds TML post-processing enhancement
                                 from liveboard_creator import create_liveboard_from_model_mcp, enhance_mcp_liveboard