Spaces:
Running
Running
chore: README, e2e grader, share-permission fix, sprint + quality results
Browse files- README: chat_interface.py is the full app (App tab primary), app.py entry point; updated structure/config
- tests/e2e_quality.py: date-aware grader, COMPLETENESS rubric rewrite, --ts-environment override, hard-fail on missing dropdown label
- thoughtspot_deployer.py: share-permission 400 fix β notify_on_share=false + empty message to satisfy the GraphQL $message contract
- sprint_2026_04.md: progress updates
- tests/quality_results: run artifacts incl. 2026-07-28 82.9/B run
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- README.md +70 -65
- sprint_2026_04.md +2 -1
- tests/e2e_quality.py +12 -3
- tests/quality_results/2026-07-23_21-14-12_quality_run.md +42 -0
- tests/quality_results/2026-07-24_01-51-18_quality_run.md +27 -0
- tests/quality_results/2026-07-24_05-49-30_quality_run.md +43 -0
- tests/quality_results/2026-07-24_16-37-19_quality_run.md +27 -0
- tests/quality_results/2026-07-27_12-47-19_quality_run.md +27 -0
- tests/quality_results/2026-07-27_19-23-28_quality_run.md +27 -0
- tests/quality_results/2026-07-28_02-18-45_quality_run.md +43 -0
- tests/quality_results/latest_test_summary.md +36 -20
- thoughtspot_deployer.py +8 -1
README.md
CHANGED
|
@@ -11,48 +11,55 @@ license: mit
|
|
| 11 |
python_version: "3.11"
|
| 12 |
---
|
| 13 |
|
| 14 |
-
#
|
| 15 |
|
| 16 |
-
A
|
| 17 |
|
| 18 |
-
> **Deployed on Hugging Face Spaces**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
## π Features
|
| 21 |
|
| 22 |
-
- **AI-
|
| 23 |
-
- **
|
| 24 |
-
- **
|
| 25 |
-
- **ThoughtSpot
|
| 26 |
-
- **Interactive UI**
|
| 27 |
|
| 28 |
## π οΈ Tech Stack
|
| 29 |
|
| 30 |
- **Frontend**: Gradio (Python web UI)
|
| 31 |
-
- **Backend**: Python 3.
|
| 32 |
-
- **Database**: Snowflake
|
| 33 |
-
- **Analytics**: ThoughtSpot
|
| 34 |
-
- **AI**: OpenAI
|
| 35 |
-
- **
|
| 36 |
|
| 37 |
## π Prerequisites
|
| 38 |
|
| 39 |
-
- Python 3.
|
| 40 |
-
- Snowflake account
|
| 41 |
- ThoughtSpot Cloud account
|
| 42 |
-
-
|
|
|
|
| 43 |
|
| 44 |
## π Quick Start
|
| 45 |
|
| 46 |
1. **Clone the repository**
|
| 47 |
```bash
|
| 48 |
-
git clone
|
| 49 |
-
cd
|
| 50 |
```
|
| 51 |
|
| 52 |
-
2. **Set up virtual environment**
|
| 53 |
```bash
|
| 54 |
-
python -m venv
|
| 55 |
-
source
|
| 56 |
```
|
| 57 |
|
| 58 |
3. **Install dependencies**
|
|
@@ -60,82 +67,80 @@ A powerful Gradio-based application that automatically generates and deploys com
|
|
| 60 |
pip install -r requirements.txt
|
| 61 |
```
|
| 62 |
|
| 63 |
-
4. **Configure environment variables**
|
| 64 |
```bash
|
| 65 |
cp .env.example .env
|
| 66 |
-
#
|
| 67 |
```
|
| 68 |
|
| 69 |
5. **Run the application**
|
| 70 |
```bash
|
| 71 |
-
python
|
| 72 |
```
|
|
|
|
| 73 |
|
| 74 |
6. **Open your browser**
|
| 75 |
-
Navigate to `http://localhost:7860`
|
| 76 |
|
| 77 |
## βοΈ Configuration
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
```env
|
| 82 |
-
#
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
SNOWFLAKE_WAREHOUSE=your_warehouse
|
| 90 |
-
SNOWFLAKE_DATABASE=your_database
|
| 91 |
-
SNOWFLAKE_SCHEMA=your_schema
|
| 92 |
-
|
| 93 |
-
# ThoughtSpot
|
| 94 |
-
THOUGHTSPOT_URL=your_thoughtspot_url
|
| 95 |
-
THOUGHTSPOT_USERNAME=your_username
|
| 96 |
-
THOUGHTSPOT_PASSWORD=your_password
|
| 97 |
|
| 98 |
# Slack deployment notifications (optional, outbound-only)
|
| 99 |
-
SLACK_BOT_TOKEN=xoxb-
|
| 100 |
SLACK_DEPLOYMENT_CHANNEL_ID=C0123456789
|
| 101 |
```
|
| 102 |
|
| 103 |
-
Slack notifications use the Slack Web API to post deployment status
|
| 104 |
-
DemoPrep into one approved channel. This path is outbound-only: no Socket Mode,
|
| 105 |
-
event subscriptions, slash commands, or public Slack request URL are required.
|
| 106 |
-
The Slack app only needs the `chat:write` bot scope, and the bot must be invited
|
| 107 |
-
to the target channel.
|
| 108 |
|
| 109 |
## π― Usage
|
| 110 |
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
|
| 117 |
## π Project Structure
|
| 118 |
|
| 119 |
```
|
| 120 |
-
|
| 121 |
-
βββ
|
| 122 |
-
βββ
|
| 123 |
-
βββ thoughtspot_deployer.py # ThoughtSpot
|
| 124 |
-
βββ
|
| 125 |
-
βββ
|
| 126 |
-
βββ
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
βββ requirements.txt # Python dependencies
|
| 128 |
βββ docs/ # Documentation
|
| 129 |
-
βββ tests/ #
|
| 130 |
βββ results/ # Generated demo results
|
| 131 |
```
|
| 132 |
|
| 133 |
## π§ͺ Testing
|
| 134 |
|
| 135 |
-
Run the test suite:
|
| 136 |
-
|
| 137 |
```bash
|
|
|
|
| 138 |
python -m pytest tests/
|
|
|
|
|
|
|
|
|
|
| 139 |
```
|
| 140 |
|
| 141 |
## π€ Contributing
|
|
@@ -154,16 +159,16 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
|
|
| 154 |
|
| 155 |
- ThoughtSpot for the analytics platform
|
| 156 |
- Snowflake for the data warehouse
|
| 157 |
-
- OpenAI for the AI capabilities
|
| 158 |
- Gradio for the web interface
|
| 159 |
|
| 160 |
## π Development Notes
|
| 161 |
|
| 162 |
-
|
| 163 |
|
| 164 |
## π Support
|
| 165 |
|
| 166 |
-
For support,
|
| 167 |
|
| 168 |
---
|
| 169 |
|
|
|
|
| 11 |
python_version: "3.11"
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# DemoPrep β AI-Powered ThoughtSpot Demo Builder
|
| 15 |
|
| 16 |
+
A Gradio application that generates and deploys complete ThoughtSpot demo environments end to end β company research, an LLM-authored data blueprint, deterministic data generation, a Snowflake schema + data load, and a ThoughtSpot semantic model and liveboard.
|
| 17 |
|
| 18 |
+
> **Deployed on Hugging Face Spaces.** `app.py` is the entry point. See [DEPLOYMENT.md](DEPLOYMENT.md) for setup.
|
| 19 |
+
|
| 20 |
+
## π₯οΈ The interface
|
| 21 |
+
|
| 22 |
+
The app is a **tabbed Gradio UI**, and the **App tab is the primary way to build a demo**: fill in a short form and press **GO**, and the full pipeline runs automatically with live progress.
|
| 23 |
+
|
| 24 |
+
> **Naming note:** the whole application lives in **`chat_interface.py`** β that name is *historical*. It is **not** just a chat interface; that single file hosts every tab (App, Chat, Admin, Settings, Run History, β¦), and the **App tab is the main path**. A **Chat tab** offers an alternative conversational flow that runs the same pipeline. `app.py` simply configures and launches this app.
|
| 25 |
|
| 26 |
## π Features
|
| 27 |
|
| 28 |
+
- **AI-powered research** β researches the company and industry with an LLM
|
| 29 |
+
- **Single blueprint pipeline** β one path, no keyword routing or fallback: research β LLM-authored `DemoBlueprint` β deterministic engine β validation β derived DDL β load
|
| 30 |
+
- **Realistic data** β bounded/related measures (a part never exceeds its whole, rates stay β€ 1, prices are stable per entity) plus planted, discoverable "Demo to Win" insights the validator proves are visible
|
| 31 |
+
- **ThoughtSpot integration** β deploys the connection, tables, semantic model, and an enhanced liveboard
|
| 32 |
+
- **Interactive UI** β tabbed Gradio app with real-time progress
|
| 33 |
|
| 34 |
## π οΈ Tech Stack
|
| 35 |
|
| 36 |
- **Frontend**: Gradio (Python web UI)
|
| 37 |
+
- **Backend**: Python 3.11
|
| 38 |
+
- **Database**: Snowflake (keypair auth)
|
| 39 |
+
- **Analytics**: ThoughtSpot (per-environment trusted auth)
|
| 40 |
+
- **AI**: multiple LLM providers (OpenAI and Anthropic/Claude), routed via `llm_config.py`
|
| 41 |
+
- **Settings**: Supabase (admin + per-user settings)
|
| 42 |
|
| 43 |
## π Prerequisites
|
| 44 |
|
| 45 |
+
- Python 3.11
|
| 46 |
+
- Snowflake account (keypair configured)
|
| 47 |
- ThoughtSpot Cloud account
|
| 48 |
+
- An LLM API key (OpenAI and/or Anthropic)
|
| 49 |
+
- Supabase project (settings storage)
|
| 50 |
|
| 51 |
## π Quick Start
|
| 52 |
|
| 53 |
1. **Clone the repository**
|
| 54 |
```bash
|
| 55 |
+
git clone <repo-url>
|
| 56 |
+
cd demoprep
|
| 57 |
```
|
| 58 |
|
| 59 |
+
2. **Set up a virtual environment**
|
| 60 |
```bash
|
| 61 |
+
python -m venv .venv
|
| 62 |
+
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
| 63 |
```
|
| 64 |
|
| 65 |
3. **Install dependencies**
|
|
|
|
| 67 |
pip install -r requirements.txt
|
| 68 |
```
|
| 69 |
|
| 70 |
+
4. **Configure bootstrap environment variables** (see Configuration)
|
| 71 |
```bash
|
| 72 |
cp .env.example .env
|
| 73 |
+
# edit .env
|
| 74 |
```
|
| 75 |
|
| 76 |
5. **Run the application**
|
| 77 |
```bash
|
| 78 |
+
python app.py
|
| 79 |
```
|
| 80 |
+
`app.py` launches the full Gradio app defined in `chat_interface.py`.
|
| 81 |
|
| 82 |
6. **Open your browser**
|
| 83 |
+
Navigate to `http://localhost:7860` and use the **App** tab.
|
| 84 |
|
| 85 |
## βοΈ Configuration
|
| 86 |
|
| 87 |
+
`.env` holds only the **bootstrap** secrets needed to start the app and reach Supabase and the LLM providers. All other credentials β the Snowflake keypair and the ThoughtSpot **per-environment trusted-auth keys** β live in **Supabase admin settings** and are loaded at runtime.
|
| 88 |
|
| 89 |
```env
|
| 90 |
+
# Supabase (settings storage)
|
| 91 |
+
SUPABASE_URL=...
|
| 92 |
+
SUPABASE_ANON_KEY=...
|
| 93 |
+
|
| 94 |
+
# LLM providers
|
| 95 |
+
OPENAI_API_KEY=...
|
| 96 |
+
GOOGLE_API_KEY=... # optional (Gemini)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
|
| 98 |
# Slack deployment notifications (optional, outbound-only)
|
| 99 |
+
SLACK_BOT_TOKEN=xoxb-...
|
| 100 |
SLACK_DEPLOYMENT_CHANNEL_ID=C0123456789
|
| 101 |
```
|
| 102 |
|
| 103 |
+
Slack notifications use the Slack Web API to post deployment status into one approved channel. This path is outbound-only: no Socket Mode, event subscriptions, slash commands, or public request URL. The Slack app needs only the `chat:write` bot scope and must be invited to the target channel.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
## π― Usage
|
| 106 |
|
| 107 |
+
The **App tab** is the primary interface β fill the form, press **GO**, and the pipeline runs end to end:
|
| 108 |
+
|
| 109 |
+
1. **Fill the form** β vertical / line of business / function (or a custom use case), the company URL, and the target **ThoughtSpot environment**
|
| 110 |
+
2. **Press GO** β research β blueprint β data generation β validation β Snowflake load β ThoughtSpot model + liveboard
|
| 111 |
+
3. **Review** β model and liveboard links, a Demo Pack, and a Spotter Viz story appear on completion
|
| 112 |
|
| 113 |
## π Project Structure
|
| 114 |
|
| 115 |
```
|
| 116 |
+
demoprep/
|
| 117 |
+
βββ app.py # Entry point β launches the Gradio app on :7860
|
| 118 |
+
βββ chat_interface.py # The full Gradio app (App/Chat/Admin/Settings/β¦) β historical name
|
| 119 |
+
βββ thoughtspot_deployer.py # ThoughtSpot deploy: connection, tables, model, liveboard
|
| 120 |
+
βββ liveboard_creator.py # MCP liveboard creation + TML post-processing (enhance_mcp_liveboard)
|
| 121 |
+
βββ snowflake_auth.py # Snowflake keypair authentication
|
| 122 |
+
βββ llm_config.py # LLM provider/model routing (single source of truth)
|
| 123 |
+
βββ demo_personas.py # Vertical Γ Function use-case configs
|
| 124 |
+
βββ demoprep_app/ # The demo-generation pipeline package
|
| 125 |
+
β βββ pipeline/build_demo.py # the single build entry point
|
| 126 |
+
β βββ scenario/ # blueprint contract + LLM authoring + directives
|
| 127 |
+
β βββ dataset/ # deterministic engine + validator
|
| 128 |
+
β βββ ddl/ # Snowflake DDL derived from the dataset
|
| 129 |
+
β βββ integrations/snowflake/ # Snowflake row loader
|
| 130 |
βββ requirements.txt # Python dependencies
|
| 131 |
βββ docs/ # Documentation
|
| 132 |
+
βββ tests/ # Unit tests + the e2e quality harness (e2e_quality.py)
|
| 133 |
βββ results/ # Generated demo results
|
| 134 |
```
|
| 135 |
|
| 136 |
## π§ͺ Testing
|
| 137 |
|
|
|
|
|
|
|
| 138 |
```bash
|
| 139 |
+
# Unit tests (pytest collects test_*.py)
|
| 140 |
python -m pytest tests/
|
| 141 |
+
|
| 142 |
+
# End-to-end quality harness β drives a running app via the browser and grades the output
|
| 143 |
+
python tests/e2e_quality.py --env-name test
|
| 144 |
```
|
| 145 |
|
| 146 |
## π€ Contributing
|
|
|
|
| 159 |
|
| 160 |
- ThoughtSpot for the analytics platform
|
| 161 |
- Snowflake for the data warehouse
|
| 162 |
+
- OpenAI and Anthropic for the AI capabilities
|
| 163 |
- Gradio for the web interface
|
| 164 |
|
| 165 |
## π Development Notes
|
| 166 |
|
| 167 |
+
Sprint planning and working notes live in `dev_notes/` (gitignored); architecture and handoff docs are in `docs/`.
|
| 168 |
|
| 169 |
## π Support
|
| 170 |
|
| 171 |
+
For support, create an issue in this repository.
|
| 172 |
|
| 173 |
---
|
| 174 |
|
sprint_2026_04.md
CHANGED
|
@@ -267,6 +267,7 @@ should tell. KPI targets, growth trends, and anomaly patterns live in the matrix
|
|
| 267 |
- [x] **AI Viz Titles** β `_humanize_viz_titles()` in `liveboard_creator.py`; one LLM call renames all raw TS column-name titles to business-readable labels; runs as Step 6.5 in `enhance_mcp_liveboard()` before TML re-import β
|
| 268 |
- [x] **KPI conversion fix** β Step 3.5 no longer promotes "by X" dimensional breakdowns to KPIs (was putting "Averagesellingprice by Category Weekly" in the Key Metrics group) β
|
| 269 |
- [x] **Chart variety** β multi-dim breakdowns ("by X and Y") β STACKED_COLUMN; single-dim categorical breakdowns now catches LINE charts (MCP often generates LINE for categoricals, blocking donut conversion) β
|
|
|
|
| 270 |
|
| 271 |
### Shipped at end of Sprint 3 / mini sprint (Apr 28-29)
|
| 272 |
|
|
@@ -286,4 +287,4 @@ should tell. KPI targets, growth trends, and anomaly patterns live in the matrix
|
|
| 286 |
|
| 287 |
---
|
| 288 |
|
| 289 |
-
*Last updated:
|
|
|
|
| 267 |
- [x] **AI Viz Titles** β `_humanize_viz_titles()` in `liveboard_creator.py`; one LLM call renames all raw TS column-name titles to business-readable labels; runs as Step 6.5 in `enhance_mcp_liveboard()` before TML re-import β
|
| 268 |
- [x] **KPI conversion fix** β Step 3.5 no longer promotes "by X" dimensional breakdowns to KPIs (was putting "Averagesellingprice by Category Weekly" in the Key Metrics group) β
|
| 269 |
- [x] **Chart variety** β multi-dim breakdowns ("by X and Y") β STACKED_COLUMN; single-dim categorical breakdowns now catches LINE charts (MCP often generates LINE for categoricals, blocking donut conversion) β
|
| 270 |
+
- [x] **Share fix** β `share_objects()` in `thoughtspot_deployer.py` was failing with `400 "Variable \"$message\" of required type \"String!\" was not provided"` on both model + liveboard auto-share (surfaced on Nike run, se-thoughtspot staging). The REST 2.0 `/security/metadata/share` endpoint proxies to a GraphQL mutation that requires a non-null `message`; payload now sends `message: ""` + `notify_on_share: False` β
|
| 271 |
|
| 272 |
### Shipped at end of Sprint 3 / mini sprint (Apr 28-29)
|
| 273 |
|
|
|
|
| 287 |
|
| 288 |
---
|
| 289 |
|
| 290 |
+
*Last updated: July 23, 2026 β share fix (`$message` required-variable 400 on model + liveboard auto-share)*
|
tests/e2e_quality.py
CHANGED
|
@@ -72,7 +72,7 @@ RUN_SETTINGS = {
|
|
| 72 |
"share_with": "mike.boone@thoughtspot.com",
|
| 73 |
"geo_scope": "USA Only",
|
| 74 |
"ai_model": "claude-sonnet-4-6", # model used for this test run
|
| 75 |
-
"ts_environment": "
|
| 76 |
}
|
| 77 |
|
| 78 |
|
|
@@ -949,11 +949,14 @@ def grade_data_quality(company: str, vertical: str, line: str, function: str,
|
|
| 949 |
"failure at this volume, but it should reduce realism/story quality.\n"
|
| 950 |
)
|
| 951 |
|
|
|
|
| 952 |
prompt = f"""You are grading a ThoughtSpot demo dataset.
|
| 953 |
|
| 954 |
Company: {company}
|
| 955 |
Vertical: {vertical} / {line}
|
| 956 |
Analytics function: {function}
|
|
|
|
|
|
|
| 957 |
|
| 958 |
The goal is a compelling demo with realistic data, outliers that drive a narrative,
|
| 959 |
and a schema that supports the key KPIs for this use case.
|
|
@@ -972,11 +975,17 @@ Score based on what you can observe.
|
|
| 972 |
1. REALISM (20 pts): Values look like real {company} data at realistic scale and ranges.
|
| 973 |
2. STORY POTENTIAL (30 pts): Outliers, trends, or anomalies exist that anchor a demo narrative.
|
| 974 |
3. TIME COVERAGE (20 pts): 12β24 months of history with meaningful trends over time.
|
|
|
|
|
|
|
| 975 |
4. SCHEMA FITNESS (15 pts): Star schema design supports the key KPIs for {line} {function}.
|
| 976 |
-
5. COMPLETENESS (15 pts):
|
|
|
|
|
|
|
|
|
|
|
|
|
| 977 |
RULE: If the row counts above show any key table at 0 rows, score COMPLETENESS = 0 for
|
| 978 |
that criteria. If the fact table is 0 rows, also deduct heavily from STORY POTENTIAL.
|
| 979 |
-
PENALTY: If there are
|
| 980 |
such as "North Corridor Route 31", "Customer 17", or "Product 42", penalize realism.
|
| 981 |
If this pattern is repeated or widespread, the data should fail.
|
| 982 |
|
|
|
|
| 72 |
"share_with": "mike.boone@thoughtspot.com",
|
| 73 |
"geo_scope": "USA Only",
|
| 74 |
"ai_model": "claude-sonnet-4-6", # model used for this test run
|
| 75 |
+
"ts_environment": "sebe - se", # se-cloud having issues; run on sebe
|
| 76 |
}
|
| 77 |
|
| 78 |
|
|
|
|
| 949 |
"failure at this volume, but it should reduce realism/story quality.\n"
|
| 950 |
)
|
| 951 |
|
| 952 |
+
today = datetime.now().strftime("%Y-%m-%d")
|
| 953 |
prompt = f"""You are grading a ThoughtSpot demo dataset.
|
| 954 |
|
| 955 |
Company: {company}
|
| 956 |
Vertical: {vertical} / {line}
|
| 957 |
Analytics function: {function}
|
| 958 |
+
Today's date is {today}. Treat any date on or before today as HISTORICAL β do NOT
|
| 959 |
+
penalize current-year or recent dates as "future-dated"; the demo is built to run today.
|
| 960 |
|
| 961 |
The goal is a compelling demo with realistic data, outliers that drive a narrative,
|
| 962 |
and a schema that supports the key KPIs for this use case.
|
|
|
|
| 975 |
1. REALISM (20 pts): Values look like real {company} data at realistic scale and ranges.
|
| 976 |
2. STORY POTENTIAL (30 pts): Outliers, trends, or anomalies exist that anchor a demo narrative.
|
| 977 |
3. TIME COVERAGE (20 pts): 12β24 months of history with meaningful trends over time.
|
| 978 |
+
Judge coverage relative to today's date above β recent/current-year data is historical,
|
| 979 |
+
not "future"; only genuinely implausible far-future dates should count against this.
|
| 980 |
4. SCHEMA FITNESS (15 pts): Star schema design supports the key KPIs for {line} {function}.
|
| 981 |
+
5. COMPLETENESS (15 pts): Fact tables are well-populated (thousands of rows) with variation
|
| 982 |
+
across dimensions. Dimensions have REALISTIC cardinality for what they represent β a
|
| 983 |
+
handful of values for a naturally-small dimension (channel, region, tier, segment) is
|
| 984 |
+
CORRECT and must NOT be penalized; entity dimensions (products, customers, accounts,
|
| 985 |
+
stores) may have many. Do NOT require any fixed member count.
|
| 986 |
RULE: If the row counts above show any key table at 0 rows, score COMPLETENESS = 0 for
|
| 987 |
that criteria. If the fact table is 0 rows, also deduct heavily from STORY POTENTIAL.
|
| 988 |
+
PENALTY: If there are generated-looking dimension labels with numeric suffixes
|
| 989 |
such as "North Corridor Route 31", "Customer 17", or "Product 42", penalize realism.
|
| 990 |
If this pattern is repeated or widespread, the data should fail.
|
| 991 |
|
tests/quality_results/2026-07-23_21-14-12_quality_run.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-23T21:14
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 56.4/100 Grade: D
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Gap | Retail & Consumer Goods / Fashio | n/a | n/a | 16.0/F β | β° timeout |
|
| 7 |
+
| Accenture | Custom | 62 | 72 | 74.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/50ebee01-e68c-47ed-bf7c-485a575ffe2d) | β
solid |
|
| 8 |
+
| Dynatrace | Technology / Software as a Servi | 62 | 72 | 74.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/43ea010a-ea4a-4ae1-8924-119dab5e451f) | β
solid |
|
| 9 |
+
| Chipotle | Travel & Hospitality / Restauran | 58 | 72 | 72.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/1feb215c-a613-4fce-920a-e5fcccd0e7a3) | β
solid |
|
| 10 |
+
| Wells Fargo | Financial Services / Banking / M | 52 | 72 | 69.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/46247799-e75e-41de-a1cb-de4dd370337d) | β
solid |
|
| 11 |
+
| Johnson & Johnson | Healthcare & Life Sciences / Lif | 42 | 72 | 64.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/180815ea-9124-41f0-a267-aa0db787c710) | β
solid |
|
| 12 |
+
| Nike | Retail & Consumer Goods / Fashio | 62 | 72 | 74.0/C [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/742a52c0-a92e-4975-8354-069840d928ba) | β
solid |
|
| 13 |
+
| PwC | Custom | n/a | n/a | 8.0/F β | β° timeout |
|
| 14 |
+
|
| 15 |
+
## Issues
|
| 16 |
+
|
| 17 |
+
- **Gap**: TIMEOUT β last event: unhandled pipeline exception
|
| 18 |
+
- **PwC**: TIMEOUT β last event: run failed
|
| 19 |
+
|
| 20 |
+
## Data Quality Weaknesses
|
| 21 |
+
|
| 22 |
+
**Accenture** (data=62/100):
|
| 23 |
+
- Critical data integrity failure: BIDS_WON exceeds BIDS_SUBMITTED in multiple rows, making win rates mathematically impos
|
| 24 |
+
- TOTAL_VALUE_WON_USD exceeds TOTAL_VALUE_SUBMITTED_USD in some rows, which is logically impossible
|
| 25 |
+
**Dynatrace** (data=62/100):
|
| 26 |
+
- Critical data integrity flaw: OPTIMIZED_SPEND_USD frequently exceeds CLOUD_SPEND_USD by large multiples (up to 12x), whi
|
| 27 |
+
- REALIZED_SAVINGS_USD is deeply negative in many rows (e.g., -$206,079, -$49,927), which contradicts the metric's name an
|
| 28 |
+
**Chipotle** (data=58/100):
|
| 29 |
+
- BOUNCE_RATE exceeds 1.0 in multiple rows (1.03, 1.75, 1.50) β a mathematically impossible value that would immediately d
|
| 30 |
+
- CLICK_TO_ORDER_RATE of 1.83 in row 4 is impossible (orders cannot exceed clicks); DIGITAL_ORDERS_INITIATED=1,650 vs CLIC
|
| 31 |
+
**Wells Fargo** (data=52/100):
|
| 32 |
+
- Critical data integrity errors: FILL_RATE >1.0, APPROVAL_RATE >1.0, and BOUNCE_RATE >1.0 are mathematically impossible a
|
| 33 |
+
- BOUNCED_SESSIONS exceeding LANDING_PAGE_SESSIONS in at least one row is an impossible combination
|
| 34 |
+
**Johnson & Johnson** (data=42/100):
|
| 35 |
+
- Critical completeness failure: DIM_ACCOUNT, DIM_PRODUCT, and DIM_SALES_REP each have only 10 rows β well below the 20+ m
|
| 36 |
+
- With only 10 reps, 10 products, and 10 accounts, the 3,333 fact rows are highly repetitive combinations that undermine a
|
| 37 |
+
**Nike** (data=62/100):
|
| 38 |
+
- All dimension tables fall well below the 20-member threshold required for completeness scoring: DIM_PRODUCT=12, DIM_CHAN
|
| 39 |
+
- COGS_PER_UNIT values are inconsistent and sometimes implausibly low (e.g., $13.61β$21.18 for what should be a $124β$156
|
| 40 |
+
|
| 41 |
+
---
|
| 42 |
+
*JSON: 2026-07-23_21-14-12_quality_run.json*
|
tests/quality_results/2026-07-24_01-51-18_quality_run.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-24T01:51
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 12.5/100 Grade: F
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| PNC Financial | Financial Services / Banking / H | n/a | n/a | 8.0/F β | β° timeout |
|
| 7 |
+
| Deloitte | Custom | n/a | n/a | 8.0/F β | β° timeout |
|
| 8 |
+
| Nike | Retail & Consumer Goods / Fashio | n/a | n/a | 20.0/F β | β° timeout |
|
| 9 |
+
| FedEx | Transportation & Logistics / Shi | n/a | n/a | 8.0/F β | β° timeout |
|
| 10 |
+
| General Mills | Retail & Consumer Goods / Grocer | n/a | n/a | 20.0/F β | β° timeout |
|
| 11 |
+
| Wells Fargo | Financial Services / Banking / M | n/a | n/a | 20.0/F β | β° timeout |
|
| 12 |
+
| J.B. Hunt | Transportation & Logistics / Tru | n/a | n/a | 8.0/F β | β° timeout |
|
| 13 |
+
| EY | Custom | n/a | n/a | 8.0/F β | β° timeout |
|
| 14 |
+
|
| 15 |
+
## Issues
|
| 16 |
+
|
| 17 |
+
- **PNC Financial**: TIMEOUT β last event: run failed
|
| 18 |
+
- **Deloitte**: TIMEOUT β last event: run failed
|
| 19 |
+
- **Nike**: TIMEOUT β last event: run failed
|
| 20 |
+
- **FedEx**: TIMEOUT β last event: run failed
|
| 21 |
+
- **General Mills**: TIMEOUT β last event: run failed
|
| 22 |
+
- **Wells Fargo**: TIMEOUT β last event: run failed
|
| 23 |
+
- **J.B. Hunt**: TIMEOUT β last event: run failed
|
| 24 |
+
- **EY**: TIMEOUT β last event: run failed
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
*JSON: 2026-07-24_01-51-18_quality_run.json*
|
tests/quality_results/2026-07-24_05-49-30_quality_run.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-24T05:49
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 81.9/100 Grade: B
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Landstar | Transportation & Logistics / Tru | 72 | 74 | 79.5/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/0c4997a8-693e-4b77-953c-c74c3c331fab) | π great |
|
| 7 |
+
| Lear | Manufacturing / Automotive / Sal | 72 | 72 | 79.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/00b625ea-be9c-457f-a1b0-880a5a3ba4d4) | π great |
|
| 8 |
+
| Wells Fargo | Financial Services / Banking / M | 78 | 72 | 82.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/b10e50e0-a4ec-4fbd-aac2-cae593db3c66) | π great |
|
| 9 |
+
| Nike | Retail & Consumer Goods / Fashio | 82 | 72 | 84.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/833be44d-b9ca-43ae-822b-fb96155cfe48) | π great |
|
| 10 |
+
| Deloitte | Custom | 82 | 72 | 84.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/5338db4d-2b4a-415c-a4d2-10c92ee36c25) | π great |
|
| 11 |
+
| Hyatt | Travel & Hospitality / Hotels / | 82 | 72 | 84.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/01365018-1def-4e1a-a8c3-c46c2581bec2) | π great |
|
| 12 |
+
| PwC | Custom | 82 | 72 | 84.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/cde36441-ff38-49e4-b13f-a45d046d917e) | π great |
|
| 13 |
+
| Edward Jones | Financial Services / Asset & Wea | 72 | 72 | 79.0/B [lb](https://se-thoughtspot-cloud.thoughtspot.cloud/#/pinboard/9b17aa08-27cc-4c0b-87fe-4ff2f30cc12f) | π great |
|
| 14 |
+
|
| 15 |
+
## Data Quality Weaknesses
|
| 16 |
+
|
| 17 |
+
**Landstar** (data=72/100):
|
| 18 |
+
- Critical data integrity failure: SPOT_LOADS + CONTRACT_LOADS frequently do not sum to TOTAL_LOADS_AGENT (e.g., 203+480=6
|
| 19 |
+
- CAPACITY_COVERAGE_RATE exceeds 1.0 in multiple rows (1.14, 1.26, 1.18, 1.29) without clear explanation β this metric sho
|
| 20 |
+
**Lear** (data=72/100):
|
| 21 |
+
- All three fact tables have exactly 3,333 rows β identical counts strongly suggest templated/synthetic generation rather
|
| 22 |
+
- FACT_DELIVERY_QUALITY model joins only include manufacturing region, missing OEM customer and vehicle platform joins tha
|
| 23 |
+
**Wells Fargo** (data=78/100):
|
| 24 |
+
- FULL_FUNNEL_CONVERSION_RATE is 0.00 across all sampled rows β this appears to be a broken or miscalculated metric, which
|
| 25 |
+
- Time coverage extends only to late 2025 (roughly 15 months from earliest visible date), falling short of the preferred 2
|
| 26 |
+
**Nike** (data=82/100):
|
| 27 |
+
- Only 24 products in the PRODUCT dimension limits assortment breadth stories and makes product-level drill-downs feel con
|
| 28 |
+
- Only 6 promotional events in the PROMO table is thin for demonstrating promotional effectiveness analysis across a full
|
| 29 |
+
**Deloitte** (data=82/100):
|
| 30 |
+
- All three fact tables have exactly 3,333 rows β the identical count across independent fact tables looks artificially ge
|
| 31 |
+
- Sample data only shows dimension keys (not label values), so actual quality of dimension member names (e.g., whether pra
|
| 32 |
+
**Hyatt** (data=82/100):
|
| 33 |
+
- Only 18 properties in the PROPERTY dimension is thin for a global hotel company demo β limits regional/brand/tier breakd
|
| 34 |
+
- Only 12 corporate accounts constrains the AR Collections narrative β cannot demonstrate meaningful segmentation across l
|
| 35 |
+
**PwC** (data=82/100):
|
| 36 |
+
- STANDARD_FEE_USD values repeat across engagements (e.g., $1,021,325.29 appears multiple times), suggesting fees are set
|
| 37 |
+
- CLIENT_ACCOUNT dimension at only 15 rows is sparse for a firm-wide PwC advisory demo; limits client-level storytelling
|
| 38 |
+
**Edward Jones** (data=72/100):
|
| 39 |
+
- All three fact tables have exactly 3,333 rows β identical counts are an obvious generation artifact that undermines real
|
| 40 |
+
- FACT_ADVISOR_PRODUCTIVITY joining to DIM_REVENUE_TYPE_KEY is semantically questionable; advisor productivity is typicall
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
*JSON: 2026-07-24_05-49-30_quality_run.json*
|
tests/quality_results/2026-07-24_16-37-19_quality_run.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-24T16:37
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 4.2/100 Grade: F
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Wipro | Technology / IT Services / Marke | n/a | n/a | 2.0/F β | β° timeout |
|
| 7 |
+
| Shake Shack | Travel & Hospitality / Restauran | n/a | n/a | 2.0/F β | β° timeout |
|
| 8 |
+
| Nike | Retail & Consumer Goods / Fashio | n/a | n/a | 8.0/F β | β° timeout |
|
| 9 |
+
| Wells Fargo | Financial Services / Banking / M | n/a | n/a | 8.0/F β | β° timeout |
|
| 10 |
+
| Landstar | Transportation & Logistics / Tru | n/a | n/a | 8.0/F β | β° timeout |
|
| 11 |
+
| UPS | Transportation & Logistics / Shi | n/a | n/a | 2.0/F β | β° timeout |
|
| 12 |
+
| McKinsey | Custom | n/a | n/a | 2.0/F β | β° timeout |
|
| 13 |
+
| Accenture | Custom | n/a | n/a | 2.0/F β | β° timeout |
|
| 14 |
+
|
| 15 |
+
## Issues
|
| 16 |
+
|
| 17 |
+
- **Wipro**: TIMEOUT β last event: run failed
|
| 18 |
+
- **Shake Shack**: TIMEOUT β last event: run failed
|
| 19 |
+
- **Nike**: TIMEOUT β last event: run failed
|
| 20 |
+
- **Wells Fargo**: TIMEOUT β last event: run failed
|
| 21 |
+
- **Landstar**: TIMEOUT β last event: run failed
|
| 22 |
+
- **UPS**: TIMEOUT β last event: run failed
|
| 23 |
+
- **McKinsey**: TIMEOUT β last event: run failed
|
| 24 |
+
- **Accenture**: TIMEOUT β last event: run failed
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
*JSON: 2026-07-24_16-37-19_quality_run.json*
|
tests/quality_results/2026-07-27_12-47-19_quality_run.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-27T12:47
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 20.0/100 Grade: F
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| NetSuite | Technology / Software as a Servi | n/a | n/a | 20.0/F β | β° timeout |
|
| 7 |
+
| General Mills | Retail & Consumer Goods / Grocer | n/a | n/a | 20.0/F β | β° timeout |
|
| 8 |
+
| PwC | Custom | n/a | n/a | 20.0/F β | β° timeout |
|
| 9 |
+
| Zendesk | Technology / Software as a Servi | n/a | n/a | 20.0/F β | β° timeout |
|
| 10 |
+
| Yum Brands | Travel & Hospitality / Restauran | n/a | n/a | 20.0/F β | β° timeout |
|
| 11 |
+
| Nike | Retail & Consumer Goods / Fashio | n/a | n/a | 20.0/F β | β° timeout |
|
| 12 |
+
| Wells Fargo | Financial Services / Banking / M | n/a | n/a | 20.0/F β | β° timeout |
|
| 13 |
+
| Accenture | Custom | n/a | n/a | 20.0/F β | β° timeout |
|
| 14 |
+
|
| 15 |
+
## Issues
|
| 16 |
+
|
| 17 |
+
- **NetSuite**: TIMEOUT β last event: run failed
|
| 18 |
+
- **General Mills**: TIMEOUT β last event: run failed
|
| 19 |
+
- **PwC**: TIMEOUT β last event: run failed
|
| 20 |
+
- **Zendesk**: TIMEOUT β last event: run failed
|
| 21 |
+
- **Yum Brands**: TIMEOUT β last event: run failed
|
| 22 |
+
- **Nike**: TIMEOUT β last event: run failed
|
| 23 |
+
- **Wells Fargo**: TIMEOUT β last event: run failed
|
| 24 |
+
- **Accenture**: TIMEOUT β last event: run failed
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
*JSON: 2026-07-27_12-47-19_quality_run.json*
|
tests/quality_results/2026-07-27_19-23-28_quality_run.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-27T19:23
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 18.5/100 Grade: F
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Nike | Retail & Consumer Goods / Fashio | n/a | n/a | 20.0/F β | β° timeout |
|
| 7 |
+
| Darden | Travel & Hospitality / Restauran | n/a | n/a | 20.0/F β | β° timeout |
|
| 8 |
+
| XPO Logistics | Transportation & Logistics / Tru | n/a | n/a | 20.0/F β | β° timeout |
|
| 9 |
+
| FedEx | Transportation & Logistics / Shi | n/a | n/a | 20.0/F β | β° timeout |
|
| 10 |
+
| McKinsey | Custom | n/a | n/a | 20.0/F β | β° timeout |
|
| 11 |
+
| EY | Custom | n/a | n/a | 20.0/F β | β° timeout |
|
| 12 |
+
| Wells Fargo | Financial Services / Banking / M | n/a | n/a | 20.0/F β | β° timeout |
|
| 13 |
+
| Splunk | Technology / Software as a Servi | n/a | n/a | 8.0/F β | β° timeout |
|
| 14 |
+
|
| 15 |
+
## Issues
|
| 16 |
+
|
| 17 |
+
- **Nike**: TIMEOUT β last event: run failed
|
| 18 |
+
- **Darden**: TIMEOUT β last event: run failed
|
| 19 |
+
- **XPO Logistics**: TIMEOUT β last event: run failed
|
| 20 |
+
- **FedEx**: TIMEOUT β last event: run failed
|
| 21 |
+
- **McKinsey**: TIMEOUT β last event: run failed
|
| 22 |
+
- **EY**: TIMEOUT β last event: run failed
|
| 23 |
+
- **Wells Fargo**: TIMEOUT β last event: run failed
|
| 24 |
+
- **Splunk**: TIMEOUT β last event: run failed
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
*JSON: 2026-07-27_19-23-28_quality_run.json*
|
tests/quality_results/2026-07-28_02-18-45_quality_run.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-28T02:18
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 82.9/100 Grade: B
|
| 3 |
+
|
| 4 |
+
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
+
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Deloitte | Custom | 78 | 78 | 83.5/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/ec141083-908e-42db-902c-3e54a3126062) | π great |
|
| 7 |
+
| Nike | Retail & Consumer Goods / Fashio | 78 | 74 | 82.5/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/a8d011b3-c221-4898-bd1c-e35018862598) | π great |
|
| 8 |
+
| Wells Fargo | Financial Services / Banking / M | 72 | 72 | 79.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/2d211fc5-8008-4fe6-9dba-85d411656623) | π great |
|
| 9 |
+
| Splunk | Technology / Software as a Servi | 78 | 72 | 82.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/0d329d55-758c-4d1d-9ac8-5e84fc4fb9c8) | π great |
|
| 10 |
+
| McKinsey | Custom | 84 | 72 | 85.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/c709d276-d37e-4ff6-8743-6359f8e5f437) | π great |
|
| 11 |
+
| Zendesk | Technology / Software as a Servi | 82 | 72 | 84.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/e3b2e2a4-6b87-4f2f-bd76-2be87c0814e6) | π great |
|
| 12 |
+
| Wipro | Technology / IT Services / Marke | 84 | 72 | 85.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/cbab7edc-f92a-4f30-a4d6-a9eeff433ef6) | π great |
|
| 13 |
+
| LG | Retail & Consumer Goods / Consum | 78 | 72 | 82.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/d6805e4f-c372-4660-a4c8-40750bf1fd1e) | π great |
|
| 14 |
+
|
| 15 |
+
## Data Quality Weaknesses
|
| 16 |
+
|
| 17 |
+
**Deloitte** (data=78/100):
|
| 18 |
+
- Only 12 engagement dimension records driving 3,333 fact rows creates excessive repetition per engagement (~278 rows per
|
| 19 |
+
- FACT_PIPELINE_PROPOSALS sample data was not visible in the provided output, making it impossible to validate pipeline wi
|
| 20 |
+
**Nike** (data=78/100):
|
| 21 |
+
- Only 23 products is thin for a Nike demo β limits cross-product comparison depth and makes the catalog feel narrow
|
| 22 |
+
- FACT_INVENTORY_SELLTHROUGH joins only to CHANNEL in the model despite having REGION_KEY in the table, blocking regional
|
| 23 |
+
**Wells Fargo** (data=72/100):
|
| 24 |
+
- Row 2 has a critical internal inconsistency: 3,921 accounts activated with only $5,575 total first-year revenue ($1.42/a
|
| 25 |
+
- Most recent sample date visible is March 2026, leaving a ~4-month gap to today (July 2026); unclear if current-quarter d
|
| 26 |
+
**Splunk** (data=78/100):
|
| 27 |
+
- Only 12 customer accounts is a very thin customer dimension β limits cross-account segmentation and makes patterns feel
|
| 28 |
+
- The extremely high DOWNTIME_MINUTES of 357.40 in one row (vs. typical 16β86) may read as a data error rather than a narr
|
| 29 |
+
**McKinsey** (data=84/100):
|
| 30 |
+
- STANDARD_BILLING_RATE_USD appears to be a fixed per-partner attribute repeated in the fact table rather than varying by
|
| 31 |
+
- All three fact tables have exactly 3,333 rows β the identical count across tables feels mechanical and slightly artifici
|
| 32 |
+
**Zendesk** (data=82/100):
|
| 33 |
+
- PIPELINE_COVERAGE_RATIO is identical at exactly 3.23 across all visible rows β eliminates any meaningful trend or narrat
|
| 34 |
+
- FACT_SALES table does not appear to join to PRODUCT or REGION dimensions based on the schema shown, limiting product and
|
| 35 |
+
**Wipro** (data=84/100):
|
| 36 |
+
- Only 10 campaigns for a global IT services firm feels thin and limits drill-down variety
|
| 37 |
+
- No account-level or named-account dimension despite ABM being a listed campaign type
|
| 38 |
+
**LG** (data=78/100):
|
| 39 |
+
- Zero-value rows (LISTED_PRICE=0, CHANNEL_SELLING_PRICE=0, ASP_VS_COMPETITOR_INDEX=0) in FACT_CHANNEL_ASP_TRACKER appear
|
| 40 |
+
- COMPETITOR_INDEX_PRICE of $0.00 for product 6 across multiple rows zeros out the ASP vs. competitor metric for that SKU,
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
*JSON: 2026-07-28_02-18-45_quality_run.json*
|
tests/quality_results/latest_test_summary.md
CHANGED
|
@@ -1,27 +1,43 @@
|
|
| 1 |
-
# DemoPrep Quality Run β 2026-
|
| 2 |
-
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:**
|
| 3 |
|
| 4 |
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
|---------|----------|------|----|-------|------|
|
| 6 |
-
|
|
| 7 |
-
|
|
| 8 |
-
|
|
| 9 |
-
|
|
| 10 |
-
|
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
|
| 15 |
-
##
|
| 16 |
|
| 17 |
-
|
| 18 |
-
-
|
| 19 |
-
-
|
| 20 |
-
|
| 21 |
-
-
|
| 22 |
-
-
|
| 23 |
-
|
| 24 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
---
|
| 27 |
-
*JSON: 2026-
|
|
|
|
| 1 |
+
# DemoPrep Quality Run β 2026-07-28T02:18
|
| 2 |
+
**Target:** https://thoughtspot-dp-test-demoprep.hf.space | **Avg:** 82.9/100 Grade: B
|
| 3 |
|
| 4 |
| Company | Use Case | Data | LB | Total | Note |
|
| 5 |
|---------|----------|------|----|-------|------|
|
| 6 |
+
| Deloitte | Custom | 78 | 78 | 83.5/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/ec141083-908e-42db-902c-3e54a3126062) | π great |
|
| 7 |
+
| Nike | Retail & Consumer Goods / Fashio | 78 | 74 | 82.5/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/a8d011b3-c221-4898-bd1c-e35018862598) | π great |
|
| 8 |
+
| Wells Fargo | Financial Services / Banking / M | 72 | 72 | 79.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/2d211fc5-8008-4fe6-9dba-85d411656623) | π great |
|
| 9 |
+
| Splunk | Technology / Software as a Servi | 78 | 72 | 82.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/0d329d55-758c-4d1d-9ac8-5e84fc4fb9c8) | π great |
|
| 10 |
+
| McKinsey | Custom | 84 | 72 | 85.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/c709d276-d37e-4ff6-8743-6359f8e5f437) | π great |
|
| 11 |
+
| Zendesk | Technology / Software as a Servi | 82 | 72 | 84.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/e3b2e2a4-6b87-4f2f-bd76-2be87c0814e6) | π great |
|
| 12 |
+
| Wipro | Technology / IT Services / Marke | 84 | 72 | 85.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/cbab7edc-f92a-4f30-a4d6-a9eeff433ef6) | π great |
|
| 13 |
+
| LG | Retail & Consumer Goods / Consum | 78 | 72 | 82.0/B [lb](https://sebe.thoughtspotstaging.cloud/#/pinboard/d6805e4f-c372-4660-a4c8-40750bf1fd1e) | π great |
|
| 14 |
|
| 15 |
+
## Data Quality Weaknesses
|
| 16 |
|
| 17 |
+
**Deloitte** (data=78/100):
|
| 18 |
+
- Only 12 engagement dimension records driving 3,333 fact rows creates excessive repetition per engagement (~278 rows per
|
| 19 |
+
- FACT_PIPELINE_PROPOSALS sample data was not visible in the provided output, making it impossible to validate pipeline wi
|
| 20 |
+
**Nike** (data=78/100):
|
| 21 |
+
- Only 23 products is thin for a Nike demo β limits cross-product comparison depth and makes the catalog feel narrow
|
| 22 |
+
- FACT_INVENTORY_SELLTHROUGH joins only to CHANNEL in the model despite having REGION_KEY in the table, blocking regional
|
| 23 |
+
**Wells Fargo** (data=72/100):
|
| 24 |
+
- Row 2 has a critical internal inconsistency: 3,921 accounts activated with only $5,575 total first-year revenue ($1.42/a
|
| 25 |
+
- Most recent sample date visible is March 2026, leaving a ~4-month gap to today (July 2026); unclear if current-quarter d
|
| 26 |
+
**Splunk** (data=78/100):
|
| 27 |
+
- Only 12 customer accounts is a very thin customer dimension β limits cross-account segmentation and makes patterns feel
|
| 28 |
+
- The extremely high DOWNTIME_MINUTES of 357.40 in one row (vs. typical 16β86) may read as a data error rather than a narr
|
| 29 |
+
**McKinsey** (data=84/100):
|
| 30 |
+
- STANDARD_BILLING_RATE_USD appears to be a fixed per-partner attribute repeated in the fact table rather than varying by
|
| 31 |
+
- All three fact tables have exactly 3,333 rows β the identical count across tables feels mechanical and slightly artifici
|
| 32 |
+
**Zendesk** (data=82/100):
|
| 33 |
+
- PIPELINE_COVERAGE_RATIO is identical at exactly 3.23 across all visible rows β eliminates any meaningful trend or narrat
|
| 34 |
+
- FACT_SALES table does not appear to join to PRODUCT or REGION dimensions based on the schema shown, limiting product and
|
| 35 |
+
**Wipro** (data=84/100):
|
| 36 |
+
- Only 10 campaigns for a global IT services firm feels thin and limits drill-down variety
|
| 37 |
+
- No account-level or named-account dimension despite ABM being a listed campaign type
|
| 38 |
+
**LG** (data=78/100):
|
| 39 |
+
- Zero-value rows (LISTED_PRICE=0, CHANNEL_SELLING_PRICE=0, ASP_VS_COMPETITOR_INDEX=0) in FACT_CHANNEL_ASP_TRACKER appear
|
| 40 |
+
- COMPETITOR_INDEX_PRICE of $0.00 for product 6 across multiple rows zeros out the ASP vs. competitor metric for that SKU,
|
| 41 |
|
| 42 |
---
|
| 43 |
+
*JSON: 2026-07-28_02-18-45_quality_run.json*
|
thoughtspot_deployer.py
CHANGED
|
@@ -2484,7 +2484,14 @@ class ThoughtSpotDeployer:
|
|
| 2484 |
"metadata": [
|
| 2485 |
{"identifier": guid, "type": object_type}
|
| 2486 |
for guid in object_guids
|
| 2487 |
-
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2488 |
}
|
| 2489 |
)
|
| 2490 |
if response.status_code in [200, 204]:
|
|
|
|
| 2484 |
"metadata": [
|
| 2485 |
{"identifier": guid, "type": object_type}
|
| 2486 |
for guid in object_guids
|
| 2487 |
+
],
|
| 2488 |
+
# The REST endpoint proxies to a GraphQL mutation that declares
|
| 2489 |
+
# $message as a non-null String! β omitting it makes the backend
|
| 2490 |
+
# reject the request ("Variable \"$message\" ... was not provided").
|
| 2491 |
+
# An empty string satisfies the contract; notify is off so nothing
|
| 2492 |
+
# is emailed to the recipient.
|
| 2493 |
+
"notify_on_share": False,
|
| 2494 |
+
"message": ""
|
| 2495 |
}
|
| 2496 |
)
|
| 2497 |
if response.status_code in [200, 204]:
|