fix: improve specialized dataset dimensions

DEMOPREP_SELLING_STATUS.md

@@ -0,0 +1,123 @@
+# DemoPrep Status Brief
+
+## Purpose
+
+DemoPrep is a demo-generation application built to help sales and solution teams create relevant, live analytics demos quickly. Given a company and a use case, the application researches the business context, builds a realistic analytics dataset, deploys it to Snowflake, creates a ThoughtSpot model, and generates a liveboard that can be used in customer conversations.
+
+The goal is simple: reduce the time and friction required to create credible, company-specific demos, while improving demo quality and consistency across the field.
+
+## What We Built
+
+DemoPrep now supports an end-to-end workflow from company input to live ThoughtSpot assets:
+
+1. **Company and use-case intake**
+   Users provide a target company, select a vertical / line / function, or enter a custom business context.
+
+2. **Research and context generation**
+   The system gathers business context and uses it to shape the demo narrative, metrics, dimensions, and liveboard story.
+
+3. **Dataset-first generation**
+   We moved away from purely schema-first synthetic data generation. The current approach builds coherent business datasets first, then derives the schema and DDL from that dataset. This has materially improved realism and reduced impossible metric combinations.
+
+4. **Snowflake deployment**
+   DemoPrep creates the schema and loads populated tables into Snowflake.
+
+5. **ThoughtSpot deployment**
+   The application creates the connection, tables, joins, model, and liveboard. The liveboard path uses MCP creation followed by TML post-processing for layout and polish.
+
+6. **Quality test harness**
+   We built an end-to-end Playwright quality suite that submits demo requests, waits for the pipeline, resolves the exact ThoughtSpot model/liveboard, samples Snowflake data, and grades both data quality and liveboard quality with an AI rubric.
+
+## Current Status
+
+The application is now consistently completing end-to-end runs in local testing, with meaningful improvements in data quality.
+
+Recent local six-pack result:
+
+| Test | Data | Liveboard | Total |
+|------|------|-----------|-------|
+| EY Professional Services | 76 | 64 | 79.0 / B |
+| Datadog SaaS Sales | 70 | 73 | 78.2 / B |
+| Nike Retail Sales | 80 | 73 | 83.2 / B |
+| Delta Airline Sales | 65 | 74 | 76.0 / B |
+| Wells Fargo Banking Marketing | 78 | 69 | 81.2 / B |
+| Starbucks Store Operations | 72 | 58 | 75.5 / B |
+
+Average: **78.9 / B**
+
+This is a major improvement from earlier runs where data quality was frequently in the 40s and 50s, with common issues such as returns exceeding sales, load factors over 100%, marketing funnel metrics out of order, and generic data that did not match the target company or use case.
+
+## What Improved
+
+The biggest improvement is the move toward a more coherent data-generation approach:
+
+- **Dataset-first architecture**: We now generate business-realistic datasets before deriving DDL.
+- **Scenario families**: The system maps requests into known scenario families such as SaaS sales, retail sales, banking marketing, airline operations, store operations, sports venue engagement, and professional services.
+- **Custom use case handling**: Custom prompts now influence the dataset structure more strongly. For example, EY now generates professional services data instead of incorrectly producing sports and venue data.
+- **Better test identity**: The test runner now uses page URLs and exact run context instead of schema-prefix guessing.
+- **More reliable diagnostics**: Logging has been improved so run identity, model selection, stage completion, and errors are easier to trace.
+- **MCP contingency path**: If the ThoughtSpot MCP service is unavailable, DemoPrep can create a clearly marked backup liveboard path rather than silently hiding the failure.
+
+## Why This Matters Commercially
+
+Demo quality directly affects seller confidence and customer credibility.
+
+Today, high-quality custom demos require manual work from people who understand the account, the industry, the data model, and ThoughtSpot. That creates a bottleneck. DemoPrep reduces that bottleneck by turning a company and business problem into a working analytics experience.
+
+The commercial value is:
+
+- **Faster demo creation**: Move from days or weeks of prep toward same-day demo generation.
+- **Better account relevance**: Generate demos that reflect the prospect’s company, industry, metrics, and business questions.
+- **Repeatable quality**: Create a structured path for demos instead of relying on ad hoc manual builds.
+- **Field leverage**: Give more sellers and solution engineers access to credible demos without requiring every demo to be hand-built.
+- **Higher confidence in AI demos**: The quality harness creates a feedback loop so we can measure and improve the demos instead of relying on subjective inspection.
+
+## Remaining Gaps
+
+The application is not finished, but the core value is now visible.
+
+Current gaps:
+
+- **Liveboard quality trails data quality**: Data scores are now often B-level, while liveboards still need better narrative layout, stronger KPI selection, and more consistent visual variety.
+- **Some scenarios are still generic**: Certain industry/function combinations need deeper specialization to feel truly customer-specific.
+- **Custom extraction needs hardening**: We improved this significantly, but custom prompts still need more testing across unusual industries and ambiguous company contexts.
+- **Operational resilience**: ThoughtSpot deployment and MCP availability remain external dependencies. We now handle some failures better, but this should continue to be hardened.
+- **Packaging and ownership**: To sell or scale this internally, we need clearer ownership, deployment expectations, support model, and roadmap.
+
+## Call To Action
+
+We are asking for alignment on whether DemoPrep should move from a sprint prototype into a supported internal product motion.
+
+Specific asks:
+
+1. **Executive sponsorship**
+   Confirm that rapid AI-generated demo creation is a priority worth operationalizing.
+
+2. **Pilot group**
+   Identify a small group of sellers and solution engineers to use DemoPrep on real opportunities and provide structured feedback.
+
+3. **Quality bar**
+   Agree on what “demo-ready” means. Proposed starting bar: pipeline completion, data score 75+, liveboard score 70+, and no critical business-logic violations.
+
+4. **Scenario prioritization**
+   Pick the highest-value vertical/function combinations to harden first based on pipeline impact.
+
+5. **Production path**
+   Decide whether this should remain a specialist tool, become an internal field application, or eventually become a customer-facing/productized capability.
+
+## Recommended Follow-Up
+
+After the meeting, the recommended next steps are:
+
+- Run a structured pilot with 5-10 field users.
+- Select 10 priority demo scenarios and harden them to a consistent quality bar.
+- Improve liveboard generation and post-processing so visual storytelling catches up to dataset quality.
+- Add a dashboard for quality trends, failure reasons, scenario coverage, and user feedback.
+- Define ownership for deployment, monitoring, support, and roadmap.
+- Prepare a short internal demo showing before/after quality improvements and a live end-to-end generation.
+
+## Bottom Line
+
+DemoPrep is now showing the shape of a valuable internal sales enablement platform. It can generate complete ThoughtSpot demos from company and use-case input, and recent work has materially improved data quality and pipeline reliability.
+
+The next decision is whether we want to treat this as an experiment, or invest enough to make it a reliable tool for the field.

demoprep_app/dataset/generators/template_generator.py

@@ -183,6 +183,41 @@ class TemplateDatasetGenerator(ScenarioDatasetGenerator):
                 "TEST_DRIVES": self._whole(rng, vehicles, max(vehicles + 1, vehicles * 5), 1.0),
             }
 
+        if profile == "automotive_supplier_sales":
+            quoted_units = self._whole(rng, 2_500, 420_000, mult)
+            award_rate = rng.uniform(0.18, 0.72)
+            awarded_units = max(0, int(quoted_units * award_rate))
+            asp = round(rng.uniform(42, 1_850), 2)
+            quoted_revenue = round(quoted_units * asp, 2)
+            awarded_revenue = round(awarded_units * asp, 2)
+            material_cost = round(awarded_revenue * rng.uniform(0.46, 0.68), 2)
+            manufacturing_cost = round(awarded_revenue * rng.uniform(0.12, 0.26), 2)
+            tooling_recovery = round(awarded_revenue * rng.uniform(0.01, 0.08), 2)
+            warranty_reserve = round(awarded_revenue * rng.uniform(0.004, 0.035), 2)
+            margin = round(awarded_revenue + tooling_recovery - material_cost - manufacturing_cost - warranty_reserve, 2)
+            backlog = round(awarded_revenue * rng.uniform(0.45, 2.8), 2)
+            on_time_launch = round(rng.uniform(72, 98), 2)
+            ppm = round(rng.uniform(8, 240), 2)
+            return {
+                "RFQS_RECEIVED": rng.randint(1, 18),
+                "QUOTED_UNITS": quoted_units,
+                "AWARDED_UNITS": awarded_units,
+                "QUOTE_WIN_RATE_PCT": round(award_rate * 100, 2),
+                "AVG_SELLING_PRICE_USD": asp,
+                "QUOTED_REVENUE_USD": quoted_revenue,
+                "AWARDED_REVENUE_USD": awarded_revenue,
+                "BACKLOG_USD": backlog,
+                "MATERIAL_COST_USD": material_cost,
+                "MANUFACTURING_COST_USD": manufacturing_cost,
+                "TOOLING_RECOVERY_USD": tooling_recovery,
+                "WARRANTY_RESERVE_USD": warranty_reserve,
+                "PROGRAM_MARGIN_USD": margin,
+                "PROGRAM_MARGIN_PCT": self._pct(margin, awarded_revenue),
+                "ON_TIME_LAUNCH_PCT": on_time_launch,
+                "DEFECT_PPM": ppm,
+                "ENGINEERING_CHANGE_ORDERS": rng.randint(0, 14),
+            }
+
         if profile == "apparel_marketing":
             impressions = self._whole(rng, 25_000, 1_200_000, mult)
             clicks = self._bounded_child(rng, impressions, 0.012, 0.075)
@@ -690,18 +725,230 @@ def _dimension_values(name: str, company_name: str, context: str = "") -> list[s
     context_low = (context or "").lower()
     shipping_context = any(term in context_low for term in ("shipping", "shipment", "parcel", "freight", "delivery"))
     life_sciences_context = any(term in context_low for term in ("life sciences", "pharma", "medical device", "therapy", "biotech", "healthcare"))
+    medical_device_context = any(
+        term in context_low or term in (company_name or "").lower()
+        for term in (
+            "medical device",
+            "surgical",
+            "robotic",
+            "robotics",
+            "intuitive surgical",
+            "da vinci",
+            "ion endoluminal",
+            "manufacturing operations",
+            "inventory",
+            "warehouse",
+        )
+    )
+    higher_ed_context = any(
+        term in context_low or term in (company_name or "").lower()
+        for term in (
+            "university",
+            "college",
+            "higher education",
+            "student",
+            "students",
+            "enrollment",
+            "admissions",
+            "retention",
+            "campus",
+            "wake forest",
+        )
+    )
     service_values = (
         ["Ground", "Express", "Freight", "International Priority", "Last Mile", "Returns", "E-Commerce Fulfillment", "Healthcare Logistics"]
         if shipping_context
         else ["API Gateway", "Web App", "Data Platform", "Identity", "Payments", "Reporting", "Observability", "Collaboration"]
     )
-    product_values = (
-        ["Oncology Therapy", "Immunology Therapy", "Surgical Device", "Orthopedic Implant", "Diagnostic Kit", "Cardiovascular Device", "Vision Care", "Consumer Health"]
-        if life_sciences_context
-        else [f"{company} Core", f"{company} Plus", f"{company} Enterprise", f"{company} Analytics", f"{company} Services", f"{company} Mobile"]
+    if medical_device_context:
+        product_values = [
+            "da Vinci 5 Surgical System",
+            "da Vinci Xi Surgical System",
+            "Ion Endoluminal System",
+            "da Vinci SP Surgical System",
+            "EndoWrist Stapler",
+            "SureForm Stapler",
+            "Vessel Sealer Extend",
+            "SynchroSeal Instrument",
+            "Monopolar Curved Scissors",
+            "Maryland Bipolar Forceps",
+            "Fenestrated Bipolar Forceps",
+            "Large Needle Driver",
+            "ProGrasp Forceps",
+            "Tip-Up Fenestrated Grasper",
+            "Cadiere Forceps",
+            "Mega SutureCut Needle Driver",
+            "Harmonic ACE Curved Shears",
+            "Permanent Cautery Hook",
+            "Endoscope Plus",
+            "High Definition Vision Cart",
+            "Surgeon Console",
+            "Patient Cart",
+            "Integrated Table Motion Kit",
+            "Firefly Fluorescence Imaging Kit",
+            "Ion Vision Probe",
+            "Ion Flexision Biopsy Needle",
+            "Bronchoscopy Catheter Kit",
+            "Single-Port Access Kit",
+            "General Surgery Instrument Tray",
+            "Urology Procedure Kit",
+            "Gynecology Procedure Kit",
+            "Thoracic Procedure Kit",
+            "Colorectal Procedure Kit",
+            "Sterile Draping Kit",
+            "Instrument Reprocessing Tray",
+            "Service Parts Kit",
+        ]
+    elif life_sciences_context:
+        product_values = ["Oncology Therapy", "Immunology Therapy", "Surgical Device", "Orthopedic Implant", "Diagnostic Kit", "Cardiovascular Device", "Vision Care", "Consumer Health"]
+    else:
+        product_values = [f"{company} Core", f"{company} Plus", f"{company} Enterprise", f"{company} Analytics", f"{company} Services", f"{company} Mobile"]
+    region_values = (
+        ["North America", "Europe", "Japan", "Asia Pacific", "Latin America", "Field Service Network", "Manufacturing Sites", "Regulatory Hold"]
+        if medical_device_context
+        else ["North Carolina", "Southeast", "Mid-Atlantic", "Northeast", "Midwest", "West Coast", "International", "Online / Hybrid"]
+        if higher_ed_context
+        else ["North America", "EMEA", "APAC", "Latin America", "Public Sector", "Enterprise East", "Enterprise West", "Digital"]
+    )
+    warehouse_values = (
+        [
+            "Sunnyvale Final Assembly",
+            "Sunnyvale Service Depot",
+            "Mexicali Manufacturing Center",
+            "Auburn Hills Logistics Hub",
+            "Atlanta Field Service Depot",
+            "Dallas Distribution Center",
+            "Memphis Fulfillment Center",
+            "Raleigh Clinical Supply Hub",
+            "Boston Northeast Service Depot",
+            "Chicago Central Distribution Center",
+            "Phoenix Southwest Depot",
+            "Los Angeles West Coast Hub",
+            "Seattle Field Service Depot",
+            "Toronto Canada Distribution Center",
+            "Amsterdam EMEA Distribution Center",
+            "Freiburg Instrument Repair Center",
+            "Dublin Regulatory Hold Center",
+            "Singapore APAC Logistics Hub",
+            "Tokyo Japan Service Depot",
+            "Seoul Korea Distribution Center",
+            "Sydney Australia Service Depot",
+            "Shanghai Regional Logistics Hub",
+            "Sao Paulo Latin America Depot",
+            "Dubai Emerging Markets Hub",
+        ]
+        if medical_device_context
+        else ["Northeast DC", "Southeast DC", "Midwest DC", "Southwest DC", "West DC", "Import Hub"]
+    )
+    supplier_values = (
+        [
+            "Precision Motion Components",
+            "Endoscopic Optics Group",
+            "Sterile Packaging Partners",
+            "Robotic Arm Assemblies",
+            "Surgical Instrument Metals",
+            "Vision Sensor Technologies",
+            "Medical Cable Systems",
+            "Servo Motor Works",
+            "Printed Circuit Medical",
+            "Titanium Machining Partners",
+            "Clinical Cart Fabrication",
+            "Single-Use Components",
+            "Fluorescence Imaging Modules",
+            "Catheter Assembly Partners",
+            "Biopsy Needle Fabrication",
+            "Hospital Integration Hardware",
+            "Regulated Plastics Molding",
+            "Cleanroom Packaging Services",
+            "Optical Lens Manufacturing",
+            "Electromechanical Subsystems",
+            "Quality Inspection Services",
+            "Sterilization Services",
+            "Field Service Parts Supply",
+            "Global Freight Medical",
+        ]
+        if medical_device_context
+        else ["Preferred Supply", "Global Components", "Prime Source", "North Coast", "Vertex Manufacturing", "Reliable Partners"]
+    )
+    program_values = (
+        [
+            "School of Business",
+            "Computer Science",
+            "Health & Exercise Science",
+            "Data Analytics",
+            "Education",
+            "Engineering Sciences",
+            "Economics",
+            "Biology",
+            "Psychology",
+            "Communication",
+            "Politics & International Affairs",
+            "Accountancy",
+            "Finance",
+            "Biomedical Sciences",
+            "Law",
+            "Divinity",
+            "Sustainability",
+            "Liberal Studies",
+        ]
+        if higher_ed_context
+        else ["Business", "Computer Science", "Nursing", "Data Analytics", "Education", "Engineering"]
+    )
+    campus_values = (
+        [
+            "Reynolda Campus",
+            "Wake Downtown",
+            "Innovation Quarter",
+            "School of Business",
+            "School of Medicine",
+            "School of Law",
+            "Charlotte Center",
+            "Online Programs",
+            "Graduate School",
+            "Divinity School",
+            "Study Abroad",
+            "Athletics Cohort",
+            "Downtown Winston-Salem",
+            "Regional Partnerships",
+            "Executive Education",
+            "First-Year Experience",
+            "Pre-College Programs",
+            "Summer Session",
+        ]
+        if higher_ed_context
+        else None
+    )
+    student_segment_values = (
+        [
+            "First-Year Undergraduate",
+            "Transfer Students",
+            "North Carolina Residents",
+            "Out-of-State Domestic",
+            "International Students",
+            "First-Generation Students",
+            "Pell Eligible Students",
+            "Honors Applicants",
+            "Graduate Professional",
+            "Online Graduate",
+            "Adult Learners",
+            "Legacy Families",
+            "Student Athletes",
+            "STEM Applicants",
+            "Business Applicants",
+            "Health Sciences Applicants",
+            "Rural NC Applicants",
+            "Urban Market Applicants",
+        ]
+        if higher_ed_context
+        else None
+    )
+    term_values = (
+        ["Fall Early Decision I", "Fall Early Decision II", "Regular Decision", "Spring Transfer", "Summer Session", "Graduate Fall", "Graduate Spring", "Online Rolling"]
+        if higher_ed_context
+        else None
     )
     values = {
-        "REGIONS": ["North America", "EMEA", "APAC", "Latin America", "Public Sector", "Enterprise East", "Enterprise West", "Digital"],
+        "REGIONS": region_values,
         "PRODUCTS": product_values,
         "CUSTOMERS": ["Enterprise", "Mid-Market", "SMB", "Strategic", "Commercial", "Public Sector"],
         "ACCOUNTS": ["Apex Systems", "Northstar Health", "Summit Retail", "Vertex Bank", "Beacon Logistics", "Cloudline Media"],
@@ -722,8 +969,8 @@ def _dimension_values(name: str, company_name: str, context: str = "") -> list[s
         "BRANDS": ["Luxury", "Premium", "Select Service", "Extended Stay", "Resort", "Convention", "Lifestyle", "Boutique"],
         "PRODUCT_CATEGORIES": ["Denim", "Outerwear", "Sportswear", "Dresses", "Footwear", "Accessories", "Kids", "Underwear"],
         "STORES": [f"{company} Flagship", f"{company} Core", f"{company} Outlet", f"{company} Digital", f"{company} Urban", f"{company} Suburban"],
-        "WAREHOUSES": ["Northeast DC", "Southeast DC", "Midwest DC", "Southwest DC", "West DC", "Import Hub"],
-        "SUPPLIERS": ["Preferred Supply", "Global Components", "Prime Source", "North Coast", "Vertex Manufacturing", "Reliable Partners"],
+        "WAREHOUSES": warehouse_values,
+        "SUPPLIERS": supplier_values,
         "ROUTES": ["Northeast Corridor", "West Coast Loop", "Central Express", "Transatlantic", "Pacific Shuttle", "Southern Line"],
         "CARRIERS": ["Rapid Freight", "Prime Carrier", "ExpressLine", "Metro Delivery", "AirBridge", "Last Mile Co"],
         "AIRPORTS": ["JFK", "LAX", "ORD", "DFW", "ATL", "SFO"],
@@ -734,7 +981,10 @@ def _dimension_values(name: str, company_name: str, context: str = "") -> list[s
         "SERVICE_LINES": service_lines,
         "DEPARTMENTS": ["Engineering", "Sales", "Operations", "Finance", "Support", "Marketing"],
         "MATTERS": ["Contract Review", "Employment Claim", "Litigation", "Compliance Audit", "IP Filing", "Privacy Review"],
-        "PROGRAMS": ["Business", "Computer Science", "Nursing", "Data Analytics", "Education", "Engineering"],
+        "PROGRAMS": program_values,
+        "CAMPUSES": campus_values or ["Campus 1", "Campus 2", "Campus 3", "Campus 4", "Campus 5", "Campus 6"],
+        "STUDENT_SEGMENTS": student_segment_values or ["Student Segment 1", "Student Segment 2", "Student Segment 3", "Student Segment 4", "Student Segment 5", "Student Segment 6"],
+        "TERMS": term_values or ["Term 1", "Term 2", "Term 3", "Term 4", "Term 5", "Term 6"],
         "TEAMS": context_values or [f"{company} Home Team", f"{company} Visiting Team", f"{company} Women's Team", f"{company} Development Team", f"{company} League Event", f"{company} Exhibition"],
         "VENUES": context_values or [company, f"{company} Arena", f"{company} Theater", f"{company} Premium Club", f"{company} Event Hall", f"{company} Fan Zone"],
         "EVENTS": context_values or [f"{company} Regular Season", f"{company} Premium Hospitality", f"{company} Group Sales Night", f"{company} Concert", f"{company} Theater Event", f"{company} Family Show"],

demoprep_app/pipeline/dataset_first.py

@@ -21,8 +21,14 @@ class DatasetFirstBuild:
     ddl: str
 
 
-def infer_scenario_type(use_case: str, vertical: str | None = None, function: str | None = None) -> str | None:
-    text = f"{vertical or ''} {function or ''} {use_case or ''}".lower()
+def infer_scenario_type(
+    use_case: str,
+    vertical: str | None = None,
+    function: str | None = None,
+    company_name: str | None = None,
+    company_url: str | None = None,
+) -> str | None:
+    text = f"{company_name or ''} {company_url or ''} {vertical or ''} {function or ''} {use_case or ''}".lower()
     fn = (function or "").strip().lower()
 
     if fn == "hr":
@@ -72,6 +78,8 @@ def infer_scenario_type(use_case: str, vertical: str | None = None, function: st
         return "sales_pipeline"
     if any(tok in text for tok in ("warehousing", "warehouse", "supply chain", "fulfillment", "distribution center")):
         return "inventory_supply_chain"
+    if any(tok in text for tok in ("borgwarner", "automotive supplier", "auto supplier", "oem supplier", "tier 1", "tier-one", "component", "components", "powertrain", "drivetrain", "propulsion system")) and "sales" in text:
+        return "automotive_supplier_sales"
     if any(tok in text for tok in ("automotive", "vehicle", "dealer", "dealership")) and "sales" in text:
         return "automotive_sales"
     if any(tok in text for tok in ("healthcare provider", "healthcare providers", "healthcare payer", "healthcare payers", "hospital", "clinic")) and "sales" in text:
@@ -101,7 +109,7 @@ def build_dataset_first_demo(
     llm_model: str | None = None,
     use_llm_contract: bool = False,
 ) -> DatasetFirstBuild | None:
-    preliminary_type = infer_scenario_type(use_case, vertical, function)
+    preliminary_type = infer_scenario_type(use_case, vertical, function, company_name, company_url)
     extraction = extract_dataset_scenario(
         company_name=company_name,
         company_url=company_url,
@@ -117,6 +125,7 @@ def build_dataset_first_demo(
         "retail_sales",
         "shipping_sales",
         "automotive_sales",
+        "automotive_supplier_sales",
         "hotel_finance",
         "apparel_marketing",
         "sales_pipeline",

demoprep_app/scenario/families.py

@@ -192,6 +192,20 @@ SCENARIO_FAMILIES: dict[str, ScenarioFamilyTemplate] = {
             "How do retail, fleet, and lease channels compare?",
         ),
     ),
+    "automotive_supplier_sales": ScenarioFamilyTemplate(
+        "automotive_supplier_sales",
+        "SUPPLIER_PROGRAM_SALES",
+        "oem-customer-component-program-month",
+        "AWARDED_REVENUE_USD",
+        ("OEM_CUSTOMERS", "COMPONENT_PROGRAMS", "VEHICLE_PLATFORMS", "PLANTS", "REGIONS"),
+        ("RFQ", "Quoted", "Nominated", "SOP", "Ramp", "Production", "End Of Production"),
+        "automotive_supplier_sales",
+        (
+            "Which OEM customers and component programs drive awarded revenue?",
+            "Where are tooling, warranty, or material costs pressuring margin?",
+            "Which vehicle platforms have the strongest backlog and quote conversion?",
+        ),
+    ),
     "logistics_route_operations": ScenarioFamilyTemplate(
         "logistics_route_operations",
         "LOGISTICS_ROUTES",

docs/thoughtspot_quality_reporting_source.md

@@ -0,0 +1,29 @@
+# ThoughtSpot Quality Reporting Source
+
+The DemoPrep quality reporting liveboard is:
+
+- URL: `https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/85ccf9c9-2487-40b6-9f97-f9cf5207436b`
+- Liveboard name: `demoprep testing`
+- ThoughtSpot model: `demoprep_test_data`
+- Model GUID: `db310fd7-b382-49d2-8398-b2f378d179f6`
+- Physical table: Postgres `public.ts_quality_results`
+- ThoughtSpot table GUID: `c12ee8a3-2b5d-4597-a873-eddb9ffaddc5`
+
+Important: this reporting liveboard does **not** read from `session_logs`.
+`session_logs` is useful for pipeline-stage diagnostics and run lineage, but deleting
+bad rows from `session_logs` will not remove them from the quality reporting charts.
+
+The liveboard's score charts use `ts_quality_results`, including:
+
+- `Weekly Average Score Trend`: `average [Total Score] [Run Timestamp].weekly`
+- `Weekly Average Data Score Trend`: `average [Data Score] [Run Timestamp].weekly`
+- `Daily Data Score Trend`: `average [Data Score] [Run Timestamp].daily [Run Timestamp] = 'last 3 weeks'`
+
+Cleanup workflow:
+
+1. Inspect the liveboard TML if the source is uncertain.
+2. Query `ts_quality_results` for the affected date window.
+3. Delete or filter rows in `ts_quality_results`, not only `session_logs`, when the
+   reporting liveboard needs to change.
+4. Use `session_logs` only to validate run provenance, user, stage completion, and
+   failure causes.

tests/e2e_quality.py

@@ -44,6 +44,7 @@ load_dotenv(Path(__file__).parent.parent / ".env")
 BASE_URL      = os.getenv("TEST_TARGET_URL", "")  # may be overridden by --url flag at runtime
 TEST_USER     = os.getenv("TEST_USER")
 TEST_PASSWORD = os.getenv("TEST_PASSWORD")
+TEST_NEW_PASSWORD = os.getenv("TEST_NEW_PASSWORD", "")
 
 CONFIG_FILE = Path(__file__).parent / "quality_config.yaml"
 RESULTS_DIR = Path(__file__).parent / "quality_results"
@@ -358,10 +359,61 @@ def _do_login(page: Page):
     page.fill('input[type=text]', TEST_USER)
     page.fill('input[type=password]', TEST_PASSWORD)
     page.click('button:has-text("Login")')
+    _wait_for_visible_app_or_auth_control(page, include_login=False, timeout=90000)
+    _handle_forced_password_change(page)
     page.wait_for_selector('button[role=tab]', timeout=90000)
     page.wait_for_timeout(3000)
 
 
+def _wait_for_visible_app_or_auth_control(page: Page, *, include_login: bool, timeout: int):
+    """Wait until a visible app tab or auth control is present."""
+    page.wait_for_function(
+        """
+        ({ includeLogin }) => {
+            const visible = (el) => !!(
+                el &&
+                (el.offsetWidth || el.offsetHeight || el.getClientRects().length)
+            );
+            const hasVisibleTab = Array.from(document.querySelectorAll('button[role="tab"]'))
+                .some(visible);
+            if (hasVisibleTab) return true;
+
+            const buttons = Array.from(document.querySelectorAll('button'))
+                .filter(visible)
+                .map((button) => (button.textContent || '').trim());
+            if (buttons.some((text) => text.includes('Change Password'))) return true;
+            if (includeLogin && buttons.some((text) => text.includes('Login'))) return true;
+            return false;
+        }
+        """,
+        arg={"includeLogin": include_login},
+        timeout=timeout,
+    )
+
+
+def _handle_forced_password_change(page: Page):
+    """Handle or explicitly fail on the app's temporary-password gate."""
+    try:
+        gate = page.get_by_text("Change Password Required", exact=False)
+        if not gate.is_visible(timeout=1500):
+            return
+    except Exception:
+        return
+
+    if not TEST_NEW_PASSWORD:
+        raise RuntimeError(
+            "Test user is blocked by the temporary-password gate. "
+            "Clear must_change_password for TEST_USER or set TEST_NEW_PASSWORD "
+            "so the harness can complete the required password change."
+        )
+
+    page.locator('input[placeholder="Enter the password you just used to sign in"]').first.fill(TEST_PASSWORD)
+    page.locator('input[placeholder="At least 8 characters"]').first.fill(TEST_NEW_PASSWORD)
+    page.locator('input[placeholder="Repeat new password"]').first.fill(TEST_NEW_PASSWORD)
+    page.click('button:has-text("Change Password")', timeout=5000)
+    page.wait_for_selector('button[role=tab]', timeout=30000)
+
+
 def _navigate_and_ensure_logged_in(page: Page, max_wait_secs: int = 300):
     """
     Navigate to BASE_URL and ensure we're on the logged-in app.
@@ -375,10 +427,7 @@ def _navigate_and_ensure_logged_in(page: Page, max_wait_secs: int = 300):
         try:
             page.goto(BASE_URL, timeout=90000)
             # Wait for either the logged-in app (tabs) or the login form
-            page.wait_for_selector(
-                'button[role=tab], input[type=password]',
-                timeout=60000,
-            )
+            _wait_for_visible_app_or_auth_control(page, include_login=True, timeout=60000)
             break
         except Exception as nav_err:
             remaining = int(deadline - time.time())
@@ -391,11 +440,12 @@ def _navigate_and_ensure_logged_in(page: Page, max_wait_secs: int = 300):
 
     # If we landed on the login page (session expired or space rebuilt), re-login
     try:
-        if page.locator('input[type=password]').is_visible(timeout=2000):
+        if page.locator('button:has-text("Login")').is_visible(timeout=2000):
             print("  🔑 Session expired — re-logging in...")
             _do_login(page)
     except Exception:
         pass  # Already on the app — no login needed
+    _handle_forced_password_change(page)
 
 
 def submit_job(page: Page, test_case: dict):
@@ -1653,6 +1703,30 @@ def save_summary_md(run: dict, json_path: Path, env_name: str = "") -> Path:
     return md_path
 
 
+def print_handoff_block(run: dict, json_path: Path, md_path: Path, env_name: str = ""):
+    latest_name = f"latest_{env_name}_summary.md" if env_name else "latest_summary.md"
+    latest_path = RESULTS_DIR / latest_name
+    timestamp = run.get("timestamp", "")
+    run_id = run.get("run_id", "")
+    target = run.get("target_url", "")
+    avg = run.get("avg_score", "")
+    grade = run.get("overall_grade", "")
+
+    print("\n📌 Agent handoff")
+    print(f"  Run ID:    {run_id}")
+    print(f"  Timestamp: {timestamp}")
+    print(f"  Target:    {target}")
+    print(f"  Results:   {json_path}")
+    print(f"  Summary:   {md_path}")
+    print(f"  Latest:    {latest_path}")
+    print(
+        "  Paste this: "
+        f"DemoPrep quality run {run_id} ({timestamp}) "
+        f"avg={avg}/{grade} target={target} "
+        f"results={json_path} summary={md_path}"
+    )
+
+
 # ---------------------------------------------------------------------------
 # Main
 # ---------------------------------------------------------------------------
@@ -1664,9 +1738,11 @@ def run_quality_suite(max_tests: int = None, env_name: str = "", suite_override:
     suite  = suite_override or build_test_suite(config)
     if max_tests:
         suite = suite[:max_tests]
+    run_id  = str(uuid.uuid4())[:8]
 
     print(f"\n{'='*62}")
     print(f"  DemoPrep Quality Run — {datetime.now().strftime('%Y-%m-%d %H:%M')}")
+    print(f"  Run ID: {run_id}")
     print(f"  Target: {BASE_URL}")
     print(f"  {len(suite)} tests  |  "
           f"{sum(1 for t in suite if t['type']=='fixed')} fixed  "
@@ -1679,7 +1755,6 @@ def run_quality_suite(max_tests: int = None, env_name: str = "", suite_override:
         label = {"fixed": "🔒", "random": "🎲", "ai_generated": "🤖", "custom": "✏️"}[tc["type"]]
         print(f"  [{i}] {label} {tc['name']}")
 
-    run_id  = str(uuid.uuid4())[:8]
     results = []
 
     with sync_playwright() as p:
@@ -1744,7 +1819,8 @@ def run_quality_suite(max_tests: int = None, env_name: str = "", suite_override:
         "test_count": len(results), "tests": results,
     }
     path = save_results(run)
-    save_summary_md(run, path, env_name=env_name)
+    md_path = save_summary_md(run, path, env_name=env_name)
+    print_handoff_block(run, path, md_path, env_name=env_name)
 
     # --- Summary table ---
     try:

tests/test_dataset_first_builders.py

@@ -259,6 +259,51 @@ def test_grocery_finance_routes_to_cpg_financials_not_sales_fact():
     assert "LTV_USD" not in fact_columns
 
 
+def test_automotive_supplier_sales_routes_to_supplier_programs():
+    build = build_dataset_first_demo(
+        company_name="BorgWarner",
+        company_url="https://borgwarner.com",
+        use_case="Automotive Sales",
+        vertical="Manufacturing",
+        function="Sales",
+        row_count_guidance=100,
+    )
+
+    assert build is not None
+    assert build.scenario.scenario_type == "automotive_supplier_sales"
+    table_map = build.dataset.table_map()
+    assert "SUPPLIER_PROGRAM_SALES" in table_map
+    assert "VEHICLE_SALES" not in table_map
+    fact_columns = {column.name for column in table_map["SUPPLIER_PROGRAM_SALES"].columns}
+    assert {
+        "RFQS_RECEIVED",
+        "QUOTED_REVENUE_USD",
+        "AWARDED_REVENUE_USD",
+        "BACKLOG_USD",
+        "TOOLING_RECOVERY_USD",
+        "WARRANTY_RESERVE_USD",
+        "PROGRAM_MARGIN_USD",
+        "DEFECT_PPM",
+    } <= fact_columns
+    assert "DEALER_HOLDBACK_USD" not in fact_columns
+    assert "TEST_DRIVES" not in fact_columns
+
+
+def test_vehicle_oem_sales_stays_on_automotive_sales():
+    build = build_dataset_first_demo(
+        company_name="Stellantis",
+        company_url="https://stellantis.com",
+        use_case="Automotive Sales",
+        vertical="Manufacturing",
+        function="Sales",
+        row_count_guidance=100,
+    )
+
+    assert build is not None
+    assert build.scenario.scenario_type == "automotive_sales"
+    assert "VEHICLE_SALES" in build.dataset.table_map()
+
+
 def test_function_specific_routing_overrides_industry_terms():
     cases = [
         ("Financial Services", "Banking", "HR", "workforce_hr"),
@@ -306,6 +351,71 @@ def test_it_operations_has_two_year_hourly_calendar():
     assert max(row["INCIDENTS"] for row in fact.rows[:100]) <= 10
 
 
+def test_medical_device_inventory_dimensions_are_not_numbered_fillers():
+    build = build_dataset_first_demo(
+        company_name="Intuitive Surgical",
+        company_url="https://www.intuitive.com",
+        use_case="Medical device manufacturing operations inventory and warehouse supply chain",
+        vertical="Manufacturing",
+        function="Operations",
+        row_count_guidance=100,
+    )
+
+    assert build is not None
+    assert build.scenario.scenario_type == "inventory_supply_chain"
+
+    table_map = build.dataset.table_map()
+    products = [row["PRODUCT_NAME"] for row in table_map["PRODUCTS"].rows]
+    warehouses = [row["WAREHOUSE_NAME"] for row in table_map["WAREHOUSES"].rows]
+    suppliers = [row["SUPPLIER_NAME"] for row in table_map["SUPPLIERS"].rows]
+    regions = [row["REGION_NAME"] for row in table_map["REGIONS"].rows]
+
+    assert len(products) == 36
+    assert len(warehouses) == 24
+    assert len(suppliers) == 24
+    assert "da Vinci 5 Surgical System" in products
+    assert "Sunnyvale Final Assembly" in warehouses
+    assert "Precision Motion Components" in suppliers
+    assert "Manufacturing Sites" in regions
+
+    generated_names = products + warehouses + suppliers
+    assert not any("Product 29" in name or "Warehouse 21" in name or "Supplier 19" in name for name in generated_names)
+    assert not any(name.rsplit(" ", 1)[-1].isdigit() for name in generated_names)
+
+
+def test_higher_education_enrollment_dimensions_are_not_numbered_fillers():
+    build = build_dataset_first_demo(
+        company_name="Wake Forest University",
+        company_url="https://www.wfu.edu",
+        use_case="Student Success Enrollment",
+        vertical="Education",
+        function="Operations",
+        row_count_guidance=100,
+    )
+
+    assert build is not None
+    assert build.scenario.scenario_type == "education_enrollment"
+
+    table_map = build.dataset.table_map()
+    programs = [row["PROGRAM_NAME"] for row in table_map["PROGRAMS"].rows]
+    campuses = [row["CAMPUS_NAME"] for row in table_map["CAMPUSES"].rows]
+    segments = [row["STUDENT_SEGMENT_NAME"] for row in table_map["STUDENT_SEGMENTS"].rows]
+    terms = [row["TERM_NAME"] for row in table_map["TERMS"].rows]
+
+    assert len(programs) == 18
+    assert len(campuses) == 18
+    assert len(segments) == 18
+    assert len(terms) == 8
+    assert "School of Business" in programs
+    assert "Reynolda Campus" in campuses
+    assert "First-Generation Students" in segments
+    assert "Fall Early Decision I" in terms
+
+    generated_names = programs + campuses + segments + terms
+    assert not any("Program 17" in name or "Campus 18" in name or "Student Segment 14" in name or "Term 07" in name for name in generated_names)
+    assert not any(name.rsplit(" ", 1)[-1].isdigit() for name in generated_names)
+
+
 def test_new_quality_pool_domains_are_not_generic_finance_or_pipeline():
     cases = [
         ("J.B. Hunt", "jbhunt.com", "Transportation & Logistics", "Trucking Finance", "Finance", "trucking_finance", "TRUCKING_FINANCIALS", {"LOADS", "REVENUE_PER_LOADED_MILE_USD", "EMPTY_MILE_PCT"}),