1. Negation + Cloud/Platform Combos (tests polarity + bridge) Show me the worst 5 performing queries for targets not on cloud Average anomaly score for all targets except those running on Linux or AIX Give me the top 3 worst SQL queries from targets that are not cloud-hosted and not demo Worst anomalies excluding any target on Linux or Windows 2. Vague / Multi-Condition Traps (tests holistic metadata detection) Show the worst performing queries on all cloud targets that are not Oracle Average anomaly score for targets that are on cloud but stopped *******Top 3 slowest queries from non-Linux targets that are still active and not demo ***********Performance issues for targets that are either on cloud or in us-east-1 but not demo 3. Referential / Follow-up Style (tests memory + bridge persistence) Show me the worst queries from the ones that are on cloud (after asking about all targets first) *********What about the expensive ones that are not on Linux? (follow-up style) Give me the average anomaly score for those targets we talked about that are not running on cloud 4. Aggregation + Complex Bridge Cases Give me the average anomaly score grouped by target for all non-cloud databases that use Oracle or PostgreSQL Show the total number of severe anomalies across targets that are on cloud but not in us-east-1 *******Minimum and maximum anomaly scores for targets not running on Linux and not demo 5. Evil Router-Trap + Digest/Plan Combos (tests intent override) Show me the worst 3 performing queries for all targets on cloud and also show their execution plans What do the actual SQL queries look like for the worst anomalies on targets not running on cloud? Dig up the execution plans for the top 3 bottlenecks on cloud-hosted targets that are not demo 18. Bonus Nuclear Test (maximum brittleness) Give me the worst performing SQL queries for targets that are cloud but not Oracle and not stopped and not in us-east-1 derby_target_cases = [ ("List all targets running on Linux.", "SELECT * FROM target WHERE hlc_server_platform = 'Linux'"), ("How many MySQL databases are there?", "SELECT count(*) FROM target WHERE hlc_database_type = 'MySQL'"), ("Show all cloud-hosted targets.", "SELECT * FROM target WHERE is_on_cloud = 'true'"), ("List all targets with status Running.", "SELECT name FROM target WHERE db_target_status = 'Running'"), ("Show all Oracle targets.", "SELECT * FROM target WHERE hlc_database_type = 'Oracle'"), ("How many targets are deployed on cloud?", "SELECT count(*) FROM target WHERE is_on_cloud = 'true'"), ("List demo targets.", "SELECT * FROM target WHERE is_demo_target = 'true'"), ] derby_digest_cases = [ ("Show the SQL query text for digest 'NULL--abc123'.", "SELECT digest_text FROM performance_schema WHERE digest = 'NULL--abc123'"), ("Show the plain text execution plan for the digest 'myschema--def456'.", "SELECT T2.sql_plain_text_plan FROM performance_schema AS T1 JOIN sql_plan AS T2 ON T1.sql_plan_id = T2.id WHERE T1.digest = 'myschema--def456'"), ("Get the execution plan for digest 'NULL--999abc'.", "SELECT T2.sql_plan FROM performance_schema AS T1 JOIN sql_plan AS T2 ON T1.sql_plan_id = T2.id WHERE T1.digest = 'NULL--999abc'"), ] derby_template_cases = [ ("How many templates are system defaults?", "SELECT count(*) FROM template WHERE is_system_default = 'true'"), ("List all dynamic templates.", "SELECT * FROM template WHERE is_dynamic = 'true'"), ] derby_report_cases = [ ("List all scheduled reports.", "SELECT * FROM report WHERE is_scheduled = 'true'"), ("Show all reports that send PDF emails.", "SELECT * FROM report WHERE is_email_pdf_report_enabled = 'true'"), ("How many reports run daily?", "SELECT count(*) FROM report WHERE scheduler_params_period = 'DAILY'"), ("List reports scheduled on Monday.", "SELECT * FROM report WHERE scheduler_params_day_of_week = 'MONDAY'"), ] derby_integration_cases = [ ("List all active integrations.", "SELECT * FROM integration WHERE enabled = 'true'"), ("Show all disabled integrations.", "SELECT * FROM integration WHERE enabled = 'false'"), ("How many integrations are enabled?", "SELECT count(*) FROM integration WHERE enabled = 'true'"), ] derby_collector_cases = [ ("List all active collectors.", "SELECT * FROM dbactdc_instance WHERE status = 'Active'"), ("Show all remote collector instances.", "SELECT * FROM dbactdc_instance WHERE is_remote = 'true'"), ("How many collectors are currently inactive?", "SELECT count(*) FROM dbactdc_instance WHERE status = 'Inactive'"), ] derby_pricing_cases = [ ("Show all AWS cloud pricing entries.", "SELECT * FROM cloud_database_pricing WHERE cloud_database_provider = 'AWS'"), (******************"What is the average credit cost across all cloud providers?", "SELECT avg(credit_cost) FROM cloud_database_pricing"), ] influx_cases = [ ("Show all performance bottlenecks.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true'"), ("What are the anomalies detected for each target?", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true'"), ("Show all spikes excluding demo data.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true'"), ("What is the average anomaly score for each target?", "SELECT target, avg(data_value_double) FROM sys_target_alerts WHERE is_demo_alert != 'true' GROUP BY target"), ("****************What is the minimum anomaly value for each target?", "SELECT target, min(data_value_double) FROM sys_target_alerts WHERE is_demo_alert != 'true' GROUP BY target"), ("Show the total anomaly count per target.", "SELECT target, count(*) FROM sys_target_alerts WHERE is_demo_alert != 'true' GROUP BY target"), ("Find the top 3 worst performing queries.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 3"), ("Show the top 10 highest anomaly scores.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 10"), ("Find the bottom 5 least severe alerts.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double ASC LIMIT 5"), ("What capture sources are used for alerts?", "SELECT target, capture, data_value_double FROM sys_target_alerts WHERE is_demo_alert != 'true'"), ("Show capture methods for all performance issues.", "SELECT target, capture, data_value_double FROM sys_target_alerts WHERE is_demo_alert != 'true'"), ("Which SQL digests have the worst performance bottlenecks?", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 5"), ("Show top sql issues.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 5"), ] coworker_cases = [ ("What is the current status of all my targets?", "SELECT name, db_target_status FROM target"), ("Give me 'MySQL_QUICK_1711_1' target information", "SELECT * FROM target WHERE name = 'MySQL_QUICK_1711_1'"), ("Give me the complete profile for the all target", "SELECT * FROM target"), ("Give me anomalies for all targets", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true'"), (***********"How many targets we have?", "SELECT count(*) FROM target"), ("What are the top sql issue on 'DB_PROD_015' target", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE target = 'MySQL_QUICK_1711_1' AND is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 5") ] multi_step_cases = [ "What is the actual SQL query text for the top 2 worst performing queries?", "Identify the single worst spike", "Get the SQL query texts causing anomalies.", "Show me the execution plans for the top 5 performance bottlenecks.", "What SQL is running on the most anomalous target?", "Show me the actual queries behind the top 3 spikes.", "********Which queries are causing the worst bottlenecks and what do they look like?", "Get execution plans for the top 2 anomalies.", "********I need to see the query text for the biggest bottleneck we have.", "Dig up the execution plans for the 4 most severe issues.", "********What does the actual SQL look like for the worst spike?" ] complex_conversational_cases = [ # Conversational Status & Platform ("Hey, can you just show me all the databases that are currently down?", "SELECT name FROM target WHERE db_target_status = 'Stopped'"), ("*******Which of my Linux targets are running Oracle?", "SELECT * FROM target WHERE hlc_server_platform = 'Linux' AND hlc_database_type = 'Oracle'"), # Implicit Math & Aggregation ("I need to know how much we are paying for AWS, just give me the average.", "SELECT avg(credit_cost) FROM cloud_database_pricing WHERE cloud_database_provider = 'AWS'"), ("Just count how many remote agents we have active right now.", "SELECT count(*) FROM dbactdc_instance WHERE is_remote = 'true' AND status = 'Active'"), # Complex Cross-Filtering ("Get me the scheduled reports that send out PDF emails.", "SELECT * FROM report WHERE is_scheduled = 'true' AND is_email_pdf_report_enabled = 'true'"), ("Show me the dynamic templates used for FinOps.", "SELECT * FROM template WHERE is_dynamic = 'true' AND template_type = 'FinOps'"), # Conversational Influx & Multi-Step Triggers ("Are there any severe issues on my systems right now?", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 5"), ("*********Give me the exact SQL queries that are responsible for the top 3 highest spikes.", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 3") ] grounded_lookup_cases = [ ("I need the full configuration profile for the DB_DEV_007 database.", "SELECT * FROM target WHERE name = 'DB_DEV_007'"), ("Is Cassandra_Cluster currently running or stopped?", "SELECT name, db_target_status FROM target WHERE name = 'Cassandra_Cluster'"), ("Show me all the PostgreSQL databases we are hosting on AIX machines.", "SELECT * FROM target WHERE hlc_database_type = 'PostgreSQL' AND hlc_server_platform = 'AIX'"), ("List all targets running on Windows that use SQLServer.", "SELECT * FROM target WHERE hlc_server_platform = 'Windows' AND hlc_database_type = 'SQLServer'"), ("List all targets deployed in the eu-central-1 region.", "SELECT * FROM target WHERE cloud_region = 'eu-central-1'"), ("Which of our databases are hosted on-prem?", "SELECT * FROM target WHERE cloud_region = 'on-prem'"), ("Give me the credit costs for Oracle databases located in us-east-1.", "SELECT name, hlc_credit_cost FROM target WHERE hlc_database_type = 'Oracle' AND cloud_region = 'us-east-1'"), ("Are there any performance bottlenecks for DB_DEV_013?", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE target = 'DB_DEV_013' AND is_demo_alert != 'true'") ] # ============================================================================= # NEW TRICKY / STRESS-TEST CASES (added here for fragility testing) # ============================================================================= tricky_stress_cases = [ # 1. Vague listing (rescue logic stress) ("Give me an overview of everything we monitor", "SELECT name, hlc_database_type, db_target_status FROM target"), ("Show me the current picture of all reports", "SELECT name, title, scheduler_params_period FROM report"), ("What does the collector landscape look like right now?", "SELECT name, status, platform FROM dbactdc_instance"), # 2. Multi-condition + lexicon conflicts ("Show me all Linux Oracle targets that are currently stopped", "SELECT * FROM target WHERE hlc_server_platform = 'Linux' AND hlc_database_type = 'Oracle' AND db_target_status = 'Stopped'"), ("List collectors that are monitoring Oracle databases on AWS", "SELECT * FROM dbactdc_instance WHERE is_remote = 'true'"), # Note: indirect, may need target join in real system ("How many active integrations do we have for targets running on-prem?", "SELECT count(*) FROM integration WHERE enabled = 'true'"), # Indirect — should not crash # 3. Negation + tricky phrasing ("Show me all performance issues for stopped targets", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' AND db_target_status != 'Stopped'"), ("Give me anomalies that are NOT coming from Cassandra_Cluster", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE target != 'Cassandra_Cluster' AND is_demo_alert != 'true'"), ("List every target that is NOT running on Linux or AIX", "SELECT * FROM target WHERE hlc_server_platform NOT IN ('Linux', 'AIX')"), # # 4. Ambiguous / conversational / follow-up style # ("Tell me about the ones that are down right now", "SELECT name, db_target_status FROM target WHERE db_target_status = 'Stopped'"), # ("What about the expensive ones?", "SELECT name, hlc_credit_cost FROM target WHERE hlc_credit_cost > 0 ORDER BY hlc_credit_cost DESC LIMIT 5"), # ("Show me the worst ones, but only the real ones, not the demo stuff", "SELECT target, metric, data_value_double, time FROM sys_target_alerts WHERE is_demo_alert != 'true' ORDER BY data_value_double DESC LIMIT 5"), # Bonus evil ones ("**************Give me the credit costs for all Oracle databases hosted on Linux in us-east-1", "SELECT name, hlc_credit_cost FROM target WHERE hlc_database_type = 'Oracle' AND hlc_server_platform = 'Linux' AND cloud_region = 'us-east-1'"), ("List all dynamic FinOps templates used by active collectors", "SELECT * FROM template WHERE is_dynamic = 'true' AND template_type = 'FinOps'"), # No direct link — should not hallucinate join ] 1. is_global_query — will fail to wipe memory: "Show me everything you're monitoring" "Give me the full picture of all databases" "What does the entire system look like?" "Across every target, what's the average anomaly score?" "Pull data from all of them" 2. is_anomaly_like bridge trigger — will fire incorrectly on non-Influx queries: "What is the worst credit cost across all providers?" — "worst" triggers bridge unnecessarily "Show me the top performing templates" — "performing" triggers bridge "Which targets have the worst configuration?" — "worst" triggers bridge, no Influx intent "Performance of my report scheduler" — "performance" triggers bridge "Give me the most severe credit overspend" — "severe" triggers bridge on Derby pricing query And will fail to trigger bridge when it should: "Dig up issues from targets not on Linux" — "issues" not in trigger list? Actually it is. But: "Find slow databases not in us-east-1" — "slow" not in list "Which targets are struggling?" — no trigger words at all 3. col_first_op — will corrupt mixed polarity queries: "Worst queries for targets on Linux or Windows but not Oracle or MySQL" — Linux/Windows are positive (=), Oracle/MySQL are negative (!=). col_first_op for hlc_database_type would be set by whichever comes first "Anomalies for MySQL targets not in us-east-1 or eu-central-1" — MySQL is positive, regions are negative, different columns so safe here — but: "Performance issues for targets on Linux but not Linux servers in eu-central-1" — same column, mixed intent 4. asked_for_cost — "average" too broad, fires on wrong queries: "Average anomaly score for cloud targets" — "average" matches, pulls cloud_database_pricing into an Influx query "What is the average credit score of my targets?" — "average" + "credit" both match, but this is a target query not a pricing query "Show me the average number of running targets" — "average" matches inappropriately 5. follow_up_pattern — will miss: "And what about those on Windows?" — "And what about" not matched "From that set, which ones are stopped?" — "from that set" not matched "Among them, show me the worst performers" — "among them" not matched "Now filter by Oracle only" — no follow-up words at all but clearly a refinement "Narrow it down to just cloud targets" — completely missed Will fire incorrectly: "What about the pricing table?" — "what about" triggers memory injection from previous query even though it's a fresh topic "How about showing me all collectors?" — "how about" triggers memory injection 1. Target <-> Pricing (target + cloud_database_pricing) Prompt: "Show me the target names and their cloud pricing credit costs for the AWS provider." What to watch for: It must generate JOIN cloud_database_pricing ON target.cloud_region = cloud_database_pricing.region. 2. Target <-> Collector (target + dbactdc_instance) Prompt: "List the target databases that are managed by remote collector agents." What to watch for: It must detect "remote" as dbactdc_instance.is_remote = 'true' and generate JOIN dbactdc_instance ON target.hlc_server_name = dbactdc_instance.name. 3. Reports <-> Templates (report + report_template) Prompt: "Which scheduled reports are currently using FinOps templates?" What to watch for: It must map "scheduled" to report.is_scheduled and "FinOps" to report_template.template_type, generating a clean join on report_id. 4. Templates <-> Charts (template + charts_template_element) Prompt: "Show me the dynamic dashboard templates and the metrics used in their chart elements." What to watch for: It should join template to charts_template_element on template_id and select template.name alongside charts_template_element.metric. 5. The 3-Table Deep Dive (report -> report_template -> report_template_element_alert) Prompt: "List the report titles that have element alerts with an alert value greater than 90." What to watch for: This is the ultimate stress test. It requires a 3-table chain. The model must traverse from report through report_template to reach report_template_element_alert to filter the alert_value.