Message Queue Topic Dimension Table internal_platform_db.dim_mq_topic_d_su

Prompt:
## Prompt
**Task Objective**: Read data from the previous day's partition of the input table and copy it as-is to the output table's current day partition.

**Time Variables**: The platform provides these variables for dynamic date computation:
- `${yyyymmdd}` : current day in YYYYMMDD format
- `${yyyymmdd-1}` : previous day in YYYYMMDD format
- You may also use Spark SQL built-in functions like `current_date()` and `date_sub()`.

**Input**: `internal_platform_db.dim_mq_topic_d_query_engine_001` (a partitioned table, partitioned by `dt`).

**Processing Rules**: 1) Select all data where the `dt` partition equals the previous day (use `${yyyymmdd-1}`); 2) No joins, single-table processing; 3) All fields are retained as-is, with no transformations or filtering.

**Output Requirements**: Output all non-partition columns: `business_id`, `business_name`, `cluster_set`, `tenant`, `namespaces`, `topic`, `mq_type`, `dw_appgroup`, `in_charge`, `description`, `create_time`, `modify_time`, `cluster_id`, `cluster_type`, `cluster_name`, `bg`, `category_name`, `is_filtered`, `tids`, `consumed_tids`, `unconsumed_tids`, `is_fully_consumed`, `has_unconsumed_tid`, `system_belong`; partitioned by the `dt` field.

**Write Requirements**: Use `INSERT OVERWRITE` to write to the current day partition (use `${yyyymmdd}`) of `internal_platform_db.dim_mq_topic_d_copilot_cand_query_engine_001`.

Please write the final HiveSQL to `result.sql` and execute it.

INSERT overwrite TABLE internal_platform_db.dim_mq_topic_d_copilot_query_engine_001 PARTITION (dt = '20260507')
SELECT
  business_id
  ,business_name
  ,cluster_set
  ,tenant
  ,namespaces
  ,topic
  ,mq_type
  ,dw_appgroup
  ,in_charge
  ,description
  ,create_time
  ,modify_time
  ,cluster_id
  ,cluster_type
  ,cluster_name
  ,bg
  ,category_name
  ,is_filtered
  ,tids
  ,consumed_tids
  ,unconsumed_tids
  ,is_fully_consumed
  ,has_unconsumed_tid
  ,system_belong
FROM internal_platform_db.dim_mq_topic_d_query_engine_001
where dt = '20260506'

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,25列(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空/非空串比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,dt 分区值 = 20260507
F,F_source_filter,source_filter,10,源表分区过滤 dt=20260506

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"business_id, cluster_id, dt"
# 预期列数,25
# 输出表,internal_platform_db.dim_mq_topic_d_copilot_cand_query_engine_001


统计数据平台WDNotebook Ray类型管道任务中运行时间跨自然天的实例明细。从`wedat

Prompt:
## Prompt
**任务目标**：统计数据平台WDNotebook Ray类型管道任务中运行时间跨自然天的实例明细，按自然天拆分并关联GPU指标。

**输入**：
- `internal_platform_db.notebook_span_info_query_engine_005`
- `internal_platform_db.dwd_gputj_service_instance_map_query_engine_005`
- `internal_platform_db.dwd_ml_platform_instance_podname_query_engine_005`
- `internal_platform_db.gputj_gpu_info_parsed_agg_1min_query_engine_005`
- `internal_platform_db.notebook_engine_info_query_engine_005`

**处理规则**：
1. 时间过滤：`20260504`-`20260507`，从`notebook_span_info`筛选在`20260507`有记录且跨天的`trace_id`。
2. 业务过滤：计算类型='ray'，服务名='notebook-runner'；span名称限定：'runner.execute'、'execute.code'、'execute.code.cell'、'client.execute.code'、'runner.killed'、'set.permanent.compute'、'create.non.permanent.compute'；GPU表过滤无效记录。
3. 跨天拆分：将跨天实例按自然天拆分，每条记录对应一天内的开始和结束时间。
4. 派生字段：
   - `instance_run_time`：(end_time - start_time)/1000（秒），基于'runner.execute'
   - `code_run_time`：代码执行相关span总时长（秒）
   - `resource_wait_time`：资源创建相关span总时长（秒）
   - `apply_for_gpu_count`：SUM(replicas * num_gpu)，按trace_id
   - `gpu_util`：k8s_container_vgpu_gpu_util_sum / k8s_container_vgpu_gpu_util_count
   - `gpu_count`：k8s_container_resource_request_gpu_sum / k8s_container_resource_request_gpu_count
5. 表关联：
   - 拆分结果按`trace_id`左关联`notebook_engine_info`获取`serving_id`
   - 通过`instance_uuid`关联`dwd_gputj_service_instance_map`与`dwd_ml_platform_instance_podname`，再按`serving_id=service_id`左关联得到`pod_name`
   - 按`pod_name`左关联`gputj_gpu_info_parsed_agg_1min`，条件为`pkg_agg_time`落在实例运行窗口内（优先`instance_start_time`到`instance_end_time`，否则`code_start_time`到`code_end_time`）

**输出要求**：
- 输出表：`internal_platform_db.dwd_notebook_instance_pod_cross_day_detail_d_cand_query_engine_005`
- 字段顺序：dt:STRING; trace_id:STRING; datawd_project_id:STRING; datawd_task_id:STRING; datawd_task_instance_id:STRING; compute_type:STRING; status_code:INT; instance_run_time:INT; code_run_time:INT; resource_wait_time:INT; code_start_time:STRING; code_end_time:STRING; instance_start_time:STRING; instance_end_time:STRING; serving_id:STRING; is_permanent:BOOLEAN; apply_for_gpu_count:INT; pod_name:STRING; pkg_agg_time:STRING; gpu_util:DOUBLE; gpu_count:INT; p_date:STRING
- 分区字段：`dt`

**写入要求**：写入`dt='20260507'`分区。

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT overwrite TABLE internal_platform_db.dwd_notebook_instance_pod_cross_day_detail_d_query_engine_005 PARTITION (dt = '20260507')
WITH
base_data_raw AS (
    SELECT DISTINCT
        trace_id,
        span_name,
        start_time,
        end_time,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code
    FROM internal_platform_db.notebook_span_info_query_engine_005
    WHERE databus_imp_date >= '2026050400'
      AND databus_imp_date <= '2026050700'
      AND trace_id IN (
        SELECT trace_id
        FROM internal_platform_db.notebook_span_info_query_engine_005
        WHERE databus_imp_date >= '2026050700'
          AND databus_imp_date <= '2026050700'
          AND compute_type = 'ray'
          AND service_name = 'notebook-runner'
          AND span_name = 'runner.execute'
        GROUP BY trace_id
      )
      AND (
          span_name = 'runner.execute'
          OR
          span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute', 'runner.killed')
      )
),
base_data_time_fixed AS (
    SELECT
        trace_id,
        span_name,
        CASE
            WHEN span_name = 'runner.killed'
              THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                      OVER(PARTITION BY trace_id)
            ELSE start_time
        END AS start_time,
        CASE
            WHEN span_name = 'runner.execute'
              THEN MAX(CASE WHEN span_name IN ('runner.execute', 'runner.killed') THEN end_time END)
                      OVER(PARTITION BY trace_id)
            ELSE end_time
        END AS end_time,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code
    FROM base_data_raw
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      from_unixtime(CAST(start_time AS BIGINT) / 1000) as start_date,
      from_unixtime(CAST(end_time AS BIGINT) / 1000) as end_date,
      datediff(from_unixtime(CAST(end_time AS BIGINT) / 1000), from_unixtime(CAST(start_time AS BIGINT) / 1000)) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        date_add(b.start_date, s.pos) AS calc_date,
        CASE WHEN s.pos = 0 THEN CAST(b.start_time AS BIGINT)
             ELSE unix_timestamp(cast(date_add(b.start_date, s.pos) as timestamp)) * 1000
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN CAST(b.end_time AS BIGINT)
             ELSE (unix_timestamp(cast(date_add(b.start_date, s.pos + 1) as timestamp)) * 1000) - 1
        END AS end_time,
        CAST(b.start_time AS BIGINT) AS span_start_time,
        CAST(b.end_time AS BIGINT) AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN end_time END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN start_time END)) / 1000.0) AS instance_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END)) / 1000.0) AS code_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN end_time END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN start_time END)) / 1000.0) AS resource_wait_time,
        from_unixtime(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END) / 1000) AS code_start_time,
        from_unixtime(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) / 1000) AS code_end_time,
        from_unixtime(MIN(CASE WHEN span_name = 'runner.execute' THEN span_start_time END) / 1000) AS instance_start_time,
        from_unixtime(MAX(CASE WHEN span_name = 'runner.execute' THEN span_end_time END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
),
gputj_dim AS (
  SELECT
    t1.service_id,
    t2.pod_name
  FROM internal_platform_db.dwd_gputj_service_instance_map_query_engine_005 t1
  INNER JOIN internal_platform_db.dwd_ml_platform_instance_podname_query_engine_005 t2
    ON t1.instance_uuid = t2.instance_uuid
    AND t1.dt=if('2026050700'>='2025080900', '2026050700', '2025080900')
    AND t2.dt=if('2026050700'>='2025060519', '2026050700', '2025060519')
  GROUP BY t1.service_id, t2.pod_name
),
gpu_metrics AS (
  select
    pkg_agg_time,
    pod_name,
    k8s_container_vgpu_gpu_util_sum / k8s_container_vgpu_gpu_util_count AS gpu_util,
    k8s_container_resource_request_gpu_sum / k8s_container_resource_request_gpu_count AS gpu_count
  from internal_platform_db.gputj_gpu_info_parsed_agg_1min_query_engine_005
  where dt >= '2026050400'
    AND dt <= '2026050700'
    AND ( k8s_container_vgpu_gpu_util_count > 0 OR k8s_container_resource_request_gpu_count > 0 )
    AND pod_name IS NOT NULL
)
SELECT
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  t1.instance_run_time,
  t1.code_run_time,
  t1.resource_wait_time,
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t2.serving_id,
  t2.is_permanent,
  t2.apply_for_gpu_count,
  t3.pod_name,
  t4.pkg_agg_time,
  t4.gpu_util,
  t4.gpu_count,
  t1.calc_date AS p_date
FROM trace_time_metrics t1 LEFT JOIN (
  SELECT
    trace_id,
    MAX(serving_id) AS serving_id,
    MAX(is_permanent) AS is_permanent,
    SUM(CAST(replicas AS INT) * CAST(num_gpu AS INT)) AS apply_for_gpu_count
  FROM internal_platform_db.notebook_engine_info_query_engine_005
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND compute_type = 'ray'
    AND service_name = 'notebook-runner'
  GROUP BY trace_id
) t2 ON t1.trace_id = t2.trace_id
LEFT JOIN gputj_dim t3 ON t2.serving_id = t3.service_id
LEFT JOIN gpu_metrics t4
  ON t3.pod_name = t4.pod_name
  AND (
    CASE
      WHEN t1.instance_start_time IS NOT NULL AND t1.instance_end_time IS NOT NULL
        THEN t4.pkg_agg_time >= t1.instance_start_time AND t4.pkg_agg_time <= t1.instance_end_time
      WHEN t1.code_start_time IS NOT NULL AND t1.code_end_time IS NOT NULL
        THEN t4.pkg_agg_time >= t1.code_start_time AND t4.pkg_agg_time <= t1.code_end_time
      ELSE FALSE
    END
  )
;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,22列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_time_calculation,time_calculation,30,"instance_run_time, code_run_time 等时间指标"
D,D_cross_day_split,cross_day_split,15,trace_001 应出现在两天中
D,D_dimension_join,dimension_join,10,"serving_id, pod_name 等维度列匹配"
F,F_join_completeness,join_completeness,5,多表都被关联
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,分区值正确

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,HARD
# 权重,"product=0.7, process=0.3"
# 范式,new
# Key列,"trace_id, p_date"
# 预期列数,
# 输出表,internal_platform_db.dwd_notebook_instance_pod_cross_day_detail_d_cand_query_engine_005


Prompt:
## Prompt
1) 任务目标
识别并统计 Notebook 管道中被异常 kill 且缺失正常完成信号的 Ray 实例，按天拆分计算运行时长、代码执行时长、资源等待时长及 GPU 申请数。

2) 输入
- `internal_platform_db.notebook_span_info_query_engine_007`
- `internal_platform_db.notebook_engine_info_query_engine_007`

3) 处理规则
- 目标实例筛选：从 span_info 筛选 `compute_type='ray'` 且 `service_name='notebook-runner'` 的实例，选取有 `span_name='runner.killed'` 但无 `span_name='runner.execute'` 的 `trace_id`。
- 合成运行区间：对每个目标 trace_id，取 `span_name='runner.start'` 的 start_time 作为实例开始时间，取 `span_name='runner.killed'` 的 end_time 作为实例结束时间，合成 `runner.execute` 区间，`status_code` 固定为 2。
- 跨天拆分：将合成区间按自然日拆分为多条记录，每条对应一天内的部分，`calc_date` 为拆分后的日期。
- 时长计算（按 trace_id 与 calc_date）：
  - instance_run_time：合成区间在当天的起止时间差（毫秒转秒）。
  - code_run_time：`span_name` 为 'execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed' 在当天的起止时间差（毫秒转秒）。
  - resource_wait_time：`span_name` 为 'set.permanent.compute', 'create.non.permanent.compute' 在当天的起止时间差（毫秒转秒）。
  - 输出 code_start_time/code_end_time/instance_start_time/instance_end_time（时间戳转字符串）。
- 关联引擎信息：左关联 engine_info（过滤 `compute_type='ray'` 且 `service_name='notebook-runner'`），关联键 `trace_id`，获取 serving_id、is_permanent，计算 `apply_for_gpu_count = SUM(replicas * num_gpu)`。

4) 输出要求
输出字段顺序：dt（STRING，分区）、p_date（STRING，calc_date）、trace_id（STRING）、datawd_project_id（STRING）、datawd_task_id（STRING）、datawd_task_instance_id（STRING）、compute_type（STRING）、status_code（INT）、instance_run_time（INT）、code_run_time（INT）、resource_wait_time（INT）、code_start_time（STRING）、code_end_time（STRING）、instance_start_time（STRING）、instance_end_time（STRING）、serving_id（STRING）、is_permanent（BOOLEAN）、apply_for_gpu_count（INT）。

5) 写入要求
目标表：`internal_platform_db.dwd_notebook_killed_instance_detail_d_copilot_cand_query_engine_007`。分区字段：dt。写入分区：`dt='20260507'`。写入模式：覆盖。

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT overwrite TABLE internal_platform_db.dwd_notebook_killed_instance_detail_d_copilot_query_engine_007 PARTITION (dt = '20260507')
WITH base_trace AS (
    SELECT trace_id
    FROM internal_platform_db.notebook_span_info_query_engine_007
    WHERE databus_imp_date >= '2026050700'
      AND databus_imp_date <= '2026050700'
      AND compute_type = 'ray'
      AND service_name = 'notebook-runner'
      AND span_name IN ('runner.execute', 'runner.killed')
    GROUP BY trace_id
    HAVING
        COUNT(CASE WHEN span_name = 'runner.killed' THEN 1 END) > 0
        AND
        COUNT(CASE WHEN span_name = 'runner.execute' THEN 1 END) = 0
),
combined_spans AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM internal_platform_db.notebook_span_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.killed', 'runner.start', 'execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute')

  UNION ALL

  SELECT
      trace_id,
      'runner.execute' AS span_name,
      MAX(CASE WHEN span_name = 'runner.start' THEN start_time END) AS start_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN end_time END) AS end_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_project_id END) AS datawd_project_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_id END) AS datawd_task_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_instance_id END) AS datawd_task_instance_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN compute_type END) AS compute_type,
      2 AS status_code
  FROM internal_platform_db.notebook_span_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.start', 'runner.killed')
  GROUP BY trace_id
),
base_data_time_fixed AS (
  SELECT
      trace_id,
      span_name,
      CASE
          WHEN span_name = 'runner.killed'
            THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                    OVER(PARTITION BY trace_id)
          ELSE start_time
      END AS start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM combined_spans
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      from_unixtime(CAST(start_time AS BIGINT) / 1000) as start_date,
      from_unixtime(CAST(end_time AS BIGINT) / 1000) as end_date,
      datediff(from_unixtime(CAST(end_time AS BIGINT) / 1000), from_unixtime(CAST(start_time AS BIGINT) / 1000)) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        date_add(b.start_date, s.pos) AS calc_date,
        CASE WHEN s.pos = 0 THEN CAST(b.start_time AS BIGINT)
             ELSE unix_timestamp(cast(date_add(b.start_date, s.pos) as timestamp)) * 1000
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN CAST(b.end_time AS BIGINT)
             ELSE (unix_timestamp(cast(date_add(b.start_date, s.pos + 1) as timestamp)) * 1000) - 1
        END AS end_time,
        CAST(b.start_time AS BIGINT) AS span_start_time,
        CAST(b.end_time AS BIGINT) AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        CAST(ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN end_time END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN start_time END)) / 1000.0) AS INT) AS instance_run_time,
        CAST(ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END)) / 1000.0) AS INT) AS code_run_time,
        CAST(ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN end_time END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN start_time END)) / 1000.0) AS INT) AS resource_wait_time,
        from_unixtime(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END) / 1000) AS code_start_time,
        from_unixtime(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) / 1000) AS code_end_time,
        from_unixtime(MIN(CASE WHEN span_name = 'runner.execute' THEN span_start_time END) / 1000) AS instance_start_time,
        from_unixtime(MAX(CASE WHEN span_name = 'runner.execute' THEN span_end_time END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
)
SELECT
  t1.calc_date AS p_date,
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  t1.instance_run_time,
  t1.code_run_time,
  t1.resource_wait_time,
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t2.serving_id,
  CAST(t2.is_permanent AS BOOLEAN) AS is_permanent,
  CAST(t2.apply_for_gpu_count AS INT) AS apply_for_gpu_count
FROM trace_time_metrics t1 LEFT JOIN (
  SELECT
    trace_id,
    MAX(serving_id) AS serving_id,
    MAX(is_permanent) AS is_permanent,
    SUM(CAST(replicas AS INT) * CAST(num_gpu AS INT)) AS apply_for_gpu_count
  FROM internal_platform_db.notebook_engine_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND compute_type = 'ray'
    AND service_name = 'notebook-runner'
  GROUP BY trace_id
) t2 ON t1.trace_id = t2.trace_id
;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,18列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_time_calculation,time_calculation,30,"instance_run_time, code_run_time 等时间指标"
D,D_cross_day_split,cross_day_split,15,跨天 trace 按天拆分正确性
D,D_engine_join,engine_join,10,"serving_id, is_permanent 等 engine 维度列匹配"
F,F_join_completeness,join_completeness,5,2张源表都被关联
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,dt=20260507

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,HARD
# 权重,"product=0.7, process=0.3"
# 范式,new
# Key列,"trace_id, p_date"
# 预期列数,
# 输出表,internal_platform_db.dwd_notebook_killed_instance_detail_d_copilot_cand_query_engine_007


Prompt:
## Prompt
任务目标：将应用组与产品归属关系按天做历史快照，按分区全量覆盖写入目标表。

输入：internal_platform_db.app_group_product_info_query_engine_011
- 字段：username、application_group、application_group_owner、product_id、product_name、product_owner、obs_product_id、obs_product_name、obs_product_owner、department、plan_product_id、plan_product_name、plan_product_owner、application_group_numbers、application_group_owners、product_owners、obs_product_owners、plan_product_owners、bg、id

处理规则：
1. 无 Join，单表处理
2. 无过滤条件，全量读取源表所有记录
3. 无聚合操作
4. 追加派生列 dt（STRING 类型），填充当天日期字符串（格式 YYYYMMDD，如 '20260507'），置于输出字段最前

输出要求：
- 输出字段顺序：dt、username、application_group、application_group_owner、product_id、product_name、product_owner、obs_product_id、obs_product_name、obs_product_owner、department、plan_product_id、plan_product_name、plan_product_owner、application_group_numbers、application_group_owners、product_owners、obs_product_owners、plan_product_owners、bg、id
- 无需去重

写入要求：
- 目标表：internal_platform_db.app_group_product_info_history_cand_query_engine_011
- 分区字段：dt（STRING）
- 写入方式：INSERT OVERWRITE 按分区写入
- 若目标表不存在，先按 Hive 标准建表（ORC 存储、按 dt 分区），再写入

请将最终 HiveSQL 写入 result.sql 并执行。

insert overwrite TABLE internal_platform_db.app_group_product_info_history_query_engine_011 PARTITION (dt = '20260507')
SELECT
username,
application_group,
application_group_owner,
product_id,
product_name,
product_owner,
obs_product_id,
obs_product_name,
obs_product_owner,
department,
plan_product_id,
plan_product_name,
plan_product_owner,
application_group_numbers,
application_group_owners,
product_owners,
obs_product_owners,
plan_product_owners,
bg,
id
FROM internal_platform_db.app_group_product_info_query_engine_011;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,21列(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空/非空串比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,dt 分区值 = 20260507
F,F_dt_column,dt_column,10,dt 派生列存在于输出且值正确

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"product_id, obs_product_id, plan_product_id, id, dt"
# 预期列数,21
# 输出表,internal_platform_db.app_group_product_info_history_cand_query_engine_011


Prompt:
## Prompt
任务目标：汇总当天有生产量的 消息队列MQ topic 维度信息及近7/30/90天生产统计，落地为 topic 治理项明细表。

输入：
- 表：internal_platform_db.dws_mq_production_feature_d_increase_query_engine_013
- 分区字段：dt（STRING，格式 YYYYMMDD）
- 关键字段：business_id、business_name、topic、cluster_set、tenant、namespaces、system_belong、dw_appgroup、in_charge、description、create_time、modify_time、cluster_id、cluster_type、cluster_name、bg、category_name、total_produce_pkg_d、production_days_last_7d、total_produce_pkg_last_7d、production_days_last_30d、total_produce_pkg_last_30d、production_days_last_90d、total_produce_pkg_last_90d、is_consumed、has_consumption_days_last_7d、has_consumption_days_last_30d、has_consumption_days_last_90d

处理规则：
1. 过滤条件：dt = '20260507' AND total_produce_pkg_last_90d > 0
2. 无 Join，单表处理
3. 派生字段：
   - mq_full_topic：若 tenant 和 namespaces 均非 NULL，则拼接为 'persistent://' + tenant + '/' + namespaces + '/' + topic；否则直接取 topic
   - app_group：取自 dw_appgroup
   - bid_incharge：取自 in_charge
   - bid_description：取自 description
   - bid_create_time：取自 create_time
   - bid_modify_time：取自 modify_time
   - hitted_gov_items：固定为 NULL
   - governance_benefit_estimate：固定为 NULL

输出要求：
- 输出字段顺序：dt、business_id、business_name、topic、cluster_set、mq_full_topic、system_belong、app_group、bid_incharge、bid_description、bid_create_time、bid_modify_time、cluster_id、cluster_type、cluster_name、bg、category_name、total_produce_pkg_d、production_days_last_7d、total_produce_pkg_last_7d、production_days_last_30d、total_produce_pkg_last_30d、production_days_last_90d、total_produce_pkg_last_90d、is_consumed、has_consumption_days_last_7d、has_consumption_days_last_30d、has_consumption_days_last_90d、hitted_gov_items、governance_benefit_estimate

写入要求：
- 目标表：internal_platform_db.ads_mq_topic_governance_item_d_cand_query_engine_013
- 分区字段：dt（STRING，格式 YYYYMMDD）
- 写入方式：INSERT OVERWRITE 按分区写入
- 若目标表不存在，先按 Hive 标准建表（ORC存储、按 dt 分区），再写入

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT OVERWRITE TABLE internal_platform_db.ads_mq_topic_governance_item_d_query_engine_013 PARTITION (dt = '20260507')
SELECT
  business_id,
  business_name,
  topic,
  cluster_set,
  IF ( tenant IS NOT NULL AND namespaces IS NOT NULL, CONCAT( 'persistent://', tenant, '/', namespaces, '/', topic ), topic ) AS mq_full_topic,
  system_belong,
  dw_appgroup AS app_group,
  in_charge AS bid_incharge,
  description AS bid_description,
  create_time AS bid_create_time,
  modify_time AS bid_modify_time,
  cluster_id,
  cluster_type,
  cluster_name,
  bg,
  category_name,
  total_produce_pkg_d,
  production_days_last_7d,
  total_produce_pkg_last_7d,
  production_days_last_30d,
  total_produce_pkg_last_30d,
  production_days_last_90d,
  total_produce_pkg_last_90d,
  is_consumed,
  has_consumption_days_last_7d,
  has_consumption_days_last_30d,
  has_consumption_days_last_90d,
  NULL AS hitted_gov_items,
  NULL AS governance_benefit_estimate
FROM internal_platform_db.dws_mq_production_feature_d_increase_query_engine_013
WHERE dt = '20260507'
AND total_produce_pkg_last_90d > 0

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,30列(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_mapping,field_mapping,25,字段别名映射正确性（app_group/bid_*等）
D,D_derived_mq_full_topic,derived_mq_full_topic,15,mq_full_topic 派生列拼接逻辑正确性
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,dt 分区值 = 20260507
F,F_filter_condition,filter_condition,10,WHERE dt='20260507' AND total_produce_pkg_last_90d>0

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"business_id, bid_incharge, bid_description, bid_create_time, bid_modify_time, cluster_id, dt"
# 预期列数,30
# 输出表,internal_platform_db.ads_mq_topic_governance_item_d_cand_query_engine_013


Prompt:
## Prompt
Task Objective: Cleanse the 数据总线 topic access cost raw data and aggregate it by topic dimension, excluding test topics, then write the results to a detail table.

Input: `internal_platform_db.ods_t_databus_access_topic_cost_date_d_query_engine_015`, partitioned by field `dt` (STRING, YYYYMMDD). Key fields: `systemname`, `dwproductname`, `dwappgroup`, `cityid`, `iset`, `topic`, `data_size` (BIGINT), `total_cost` (DOUBLE), `in_charge`.

Processing Rules: No joins, single-table processing. Filter condition: `dt = '20260507' AND topic <> 'test'`. Aggregation: Group by `topic`, applying `MAX` to `systemname`, `dwproductname`, `dwappgroup`, `cityid`, `iset`, `data_size`, `total_cost`, and `in_charge`. Derived column: `dt` directly takes the partition value `'20260507'`.

Output Requirements: The output field order is `dt`, `system_belong`, `category_name`, `dw_appgroup`, `city_id`, `cluster_set`, `topic`, `total_data_size_d`, `total_cost`, `in_charge`. Field mappings: `dt=dt`, `system_belong=MAX(systemname)`, `category_name=MAX(dwproductname)`, `dw_appgroup=MAX(dwappgroup)`, `city_id=MAX(cityid)`, `cluster_set=MAX(iset)`, `topic=topic`, `total_data_size_d=MAX(data_size)`, `total_cost=MAX(total_cost)`, `in_charge=MAX(in_charge)`. Data types: `dt` STRING, `system_belong` STRING, `category_name` STRING, `dw_appgroup` STRING, `city_id` STRING, `cluster_set` STRING, `topic` STRING, `total_data_size_d` BIGINT, `total_cost` DOUBLE, `in_charge` STRING.

Write Requirements: Target table `internal_platform_db.dwd_databus_topic_cost_detail_d_cand_query_engine_015`. Table comment: "数据总线 topic access cost detail table, including access cost and dimension information, performing data cleansing on the ODS layer." Partition field `dt` (STRING, YYYYMMDD). Write method: `INSERT OVERWRITE`, overwriting the partition `dt='20260507'`. If the target table does not exist, first create the table using standard Hive ORC storage format partitioned by `dt`, then write the data.

Please write the final HiveSQL to `result.sql` and execute it.

INSERT overwrite TABLE internal_platform_db.dwd_databus_topic_cost_detail_d_query_engine_015 PARTITION (dt = '20260507')
SELECT
  MAX(systemname) AS system_belong,
  MAX(dwproductname) AS category_name,
  MAX(dwappgroup) AS dw_appgroup,
  MAX(cityid) AS city_id,
  MAX(iset) AS cluster_set,
  topic,
  MAX(data_size) AS total_data_size_d,
  MAX(total_cost) AS total_cost,
  MAX(in_charge) AS in_charge
FROM
  internal_platform_db.ods_t_databus_access_topic_cost_date_d_query_engine_015
WHERE
  dt = '20260507'
  AND topic <> 'test'
GROUP BY topic

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空/非空串比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,dt 分区值 = 20260507
F,F_source_filter,source_filter,10,源表分区过滤 dt=20260507 + topic<>'test'

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"city_id, dt"
# 预期列数,10
# 输出表,internal_platform_db.dwd_databus_topic_cost_detail_d_cand_query_engine_015


Aggregate Notebook instance runtime statistics by trace_id and p_date for the day: Input table w

Prompt:
## Prompt
Task Objective: Aggregate Notebook instance runtime statistics grouped by `trace_id` and `p_date`.

Input: `internal_platform_db.dws_notebook_instance_execute_minute_stat_d_query_engine_019`
- Partition field: `dt` (STRING, format YYYYMMDD)
- Key fields: `trace_id`, `p_date`, `datawd_project_id`, `datawd_task_id`, `datawd_task_instance_id`, `compute_type`, `status_code`, `instance_run_time`, `code_run_time`, `resource_wait_time`, `code_start_time`, `code_end_time`, `instance_start_time`, `instance_end_time`, `serving_id`, `is_permanent`, `apply_for_gpu_count`, `code_gpu_count` (numeric), `used_gpu_count` (numeric), `is_code_running` (INT), `code_used_gpu_count` (numeric)

Processing Rules:
- Filter condition: `dt = '20260507'`
- No joins, single-table processing
- Aggregation: Group by `trace_id`, `p_date`
  - Most fields use `MAX`: `datawd_project_id`, `datawd_task_id`, `datawd_task_instance_id`, `compute_type`, `status_code`, `instance_run_time`, `code_run_time`, `resource_wait_time`, `code_start_time`, `code_end_time`, `instance_start_time`, `instance_end_time`, `serving_id`, `is_permanent`, `apply_for_gpu_count`
  - `code_gpu_count`: Use `AVG`
  - `instance_gpu_util`: `AVG(used_gpu_count / code_gpu_count) * 100`
  - `code_gpu_util`: `AVG(IF(is_code_running = 1, code_used_gpu_count / code_gpu_count, NULL)) * 100`

Output Requirements:
- Output field order: `dt` (STRING), `trace_id` (STRING), `p_date` (STRING), `datawd_project_id` (STRING), `datawd_task_id` (STRING), `datawd_task_instance_id` (STRING), `compute_type` (STRING), `status_code` (INT), `instance_run_time` (INT), `code_run_time` (INT), `resource_wait_time` (INT), `code_start_time` (STRING), `code_end_time` (STRING), `instance_start_time` (STRING), `instance_end_time` (STRING), `serving_id` (STRING), `is_permanent` (BOOLEAN), `apply_for_gpu_count` (INT), `code_gpu_count` (INT), `instance_gpu_util` (DOUBLE), `code_gpu_util` (DOUBLE)
- Partition field: `dt` (STRING, format YYYYMMDD)

Write Requirements:
- Target table: `internal_platform_db.dws_notebook_instance_execute_stat_d_cand_query_engine_019`
- Write method: `INSERT OVERWRITE` partitioned by `dt`
- If the target table does not exist, first create the table using standard Hive ORC storage, then write the data

Please write the final HiveSQL to `result.sql` and execute it.

INSERT overwrite TABLE internal_platform_db.dws_notebook_instance_execute_stat_d_query_engine_019 PARTITION (dt = '20260507')
SELECT
  t1.trace_id,
  t1.p_date,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  t1.instance_run_time,
  t1.code_run_time,
  t1.resource_wait_time,
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t1.serving_id,
  t1.is_permanent,
  t1.apply_for_gpu_count,
  t1.code_gpu_count,
  t1.instance_gpu_util,
  t1.code_gpu_util
FROM (
  SELECT
    trace_id,
    p_date,
    MAX(datawd_project_id) as datawd_project_id,
    MAX(datawd_task_id) as datawd_task_id,
    MAX(datawd_task_instance_id) as datawd_task_instance_id,
    MAX(compute_type) as compute_type,
    MAX(status_code) as status_code,
    MAX(instance_run_time) as instance_run_time,
    MAX(code_run_time) as code_run_time,
    MAX(resource_wait_time) as resource_wait_time,
    MAX(code_start_time) as code_start_time,
    MAX(code_end_time) as code_end_time,
    MAX(instance_start_time) as instance_start_time,
    MAX(instance_end_time) as instance_end_time,
    MAX(serving_id) as serving_id,
    MAX(is_permanent) as is_permanent,
    MAX(apply_for_gpu_count) as apply_for_gpu_count,
    AVG(code_gpu_count) as code_gpu_count,
    AVG(used_gpu_count / code_gpu_count) * 100  as instance_gpu_util,
    AVG(if(is_code_running = 1, code_used_gpu_count / code_gpu_count, null)) * 100 as code_gpu_util
  FROM internal_platform_db.dws_notebook_instance_execute_minute_stat_d_query_engine_019
  WHERE dt = '20260507'
  GROUP BY trace_id, p_date
) t1
;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空/非空串比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,dt 分区值 = 20260507
F,F_source_filter,source_filter,10,源表分区过滤 dt=20260507

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"trace_id, datawd_project_id, datawd_task_id, datawd_task_instance_id, serving_id, dt"
# 预期列数,21
# 输出表,internal_platform_db.dws_notebook_instance_execute_stat_d_cand_query_engine_019


Count tasks matching the shuffle_split tuning rule. Input table: internal_platform_db

Prompt:
## Prompt
Task Objective: Count tasks that match the `shuffle_split` tuning rule, producing task-level tuning rule hit results.

Inputs:
- `internal_platform_db.nextgen_platform_dsl_spark_props_fht0_query_engine_021`
- `internal_platform_db.ods_spark_props_extra_query_engine_021`
- `internal_platform_db.deep_tuning_rule_hit_app_info_query_engine_021`
- `internal_platform_db.nextgen_platform_dsl_us_task_detail_fdt0_query_engine_021`
- `internal_platform_db.nextgen_platform_dsl_gputj_task_detail_fdt0_query_engine_021`
- `internal_platform_db.app_group_product_info_history_query_engine_021`

Processing Rules: Join the Spark configuration table, task detail tables (from both the `us` and `gputj` sources), and the application group product mapping table. Based on `deep_tuning_rule_hit_app_info`, filter records that match the `shuffle_split` tuning rule, and aggregate at the task granularity to produce deduplicated results. The join conditions and aggregation logic must be consistent with the historical logic, without introducing additional filtering or business rules.

Output Requirements: The output field order must match the target table schema, retaining task-level fields related to tuning rule hits, deduplicated at the task granularity.

Write Requirements: Write to table `internal_platform_db.shuffle_split_tuning_rule_hit_task_info_cand_query_engine_021`.

Please write the final HiveSQL to `result.sql` and execute it.

insert overwrite table internal_platform_db.shuffle_split_tuning_rule_hit_task_info_query_engine_021 partition (databus_imp_date = '20260507')
with spark_props as
(
    select
    t1.app_id
    ,max(spark_partition_min) spark_partition_min
    ,max(spark_partition_max) spark_partition_max
    ,max(round(ifnull(spark_task_shuffle_size,'')/(1024*1024))) spark_task_shuffle_size
    ,max(spark_shuffle_sort_bypassMergeThreshold) spark_shuffle_sort_bypassMergeThreshold
    ,max(spark_default_parallelism) spark_default_parallelism
    ,max(spark_sql_shuffle_partitions) spark_sql_shuffle_partitions
    ,max(mapreduce_input_fileinputformat_split_minsize) mapreduce_input_fileinputformat_split_minsize
    ,max(mapreduce_input_fileinputformat_split_maxsize) mapreduce_input_fileinputformat_split_maxsize
    ,max(mapreduce_input_fileinputformat_split_minsize_per_node) mapreduce_input_fileinputformat_split_minsize_per_node
    ,max(mapreduce_input_fileinputformat_split_minsize_per_rack) mapreduce_input_fileinputformat_split_minsize_per_rack
    from
    (
        select
        app_id
        ,max(spark_partition_max) spark_partition_max
        ,max(spark_task_shuffle_size) spark_task_shuffle_size
        ,max(spark_default_parallelism) spark_default_parallelism
        ,max(spark_sql_shuffle_partitions) spark_sql_shuffle_partitions
        from internal_platform_db.nextgen_platform_dsl_spark_props_fht0_query_engine_021
        where databus_imp_date >= '2026050700' and databus_imp_date <= '2026050723'
        group by app_id
    ) t1
    left join
    (
        select
        app_id
        ,max(spark_partition_min) spark_partition_min
        ,max(spark_shuffle_sort_bypassMergeThreshold) spark_shuffle_sort_bypassMergeThreshold
        ,max(mapreduce_input_fileinputformat_split_minsize) mapreduce_input_fileinputformat_split_minsize
        ,max(mapreduce_input_fileinputformat_split_maxsize) mapreduce_input_fileinputformat_split_maxsize
        ,max(mapreduce_input_fileinputformat_split_minsize_per_node) mapreduce_input_fileinputformat_split_minsize_per_node
        ,max(mapreduce_input_fileinputformat_split_minsize_per_rack) mapreduce_input_fileinputformat_split_minsize_per_rack
        from internal_platform_db.ods_spark_props_extra_query_engine_021
        where dt = '20260507'
        group by app_id
    ) t2
    on t1.app_id=t2.app_id
    group by t1.app_id
),
app_info as
(
    select
    t1.*
    ,dw_appgroup
    ,case when (project_id is null or project_id = '') then 'us' else 'datawd' end tag
    from
    (
        select *
        from internal_platform_db.deep_tuning_rule_hit_app_info_query_engine_021
        where databus_imp_date = '20260507' and stage_type = 'shuffleInputStage'
    ) t1
    join
    (
        select
        task_id
        ,max(project_id) project_id
        ,max(dw_appgroup) dw_appgroup
        from internal_platform_db.nextgen_platform_dsl_us_task_detail_fdt0_query_engine_021
        where databus_imp_date = '20260507'
        group by task_id
    ) t2
    on t1.usp_task_id=t2.task_id

    union all

    select
    t1.*
    ,dw_appgroup
    ,'gputj' tag
    from
    (
        select *
        from internal_platform_db.deep_tuning_rule_hit_app_info_query_engine_021
        where databus_imp_date = '20260507' and stage_type = 'shuffleInputStage'
    ) t1
    join
    (
        select
        task_id
        ,max(app_group_id) dw_appgroup
        from internal_platform_db.nextgen_platform_dsl_gputj_task_detail_fdt0_query_engine_021
        where databus_imp_date >= '2026050700' and databus_imp_date <= '2026050723'
        group by task_id
    ) t2
    on t1.usp_task_id=t2.task_id
),
task_info as
(
    select
    usp_task_id
    ,t1.app_id app_id
    ,app_time_usage
    ,dw_appgroup
    ,tag
    ,product_name
    ,shuffle_split
    ,max_shuffle_stage_task_num
    ,shuffle_stage_time
    ,shuffle_stage_task_time_50th
    ,shuffle_stage_task_time_90th
    ,spark_partition_min
    ,spark_partition_max
    ,spark_task_shuffle_size
    ,spark_default_parallelism
    ,spark_sql_shuffle_partitions
    ,spark_shuffle_sort_bypassMergeThreshold
    ,spark_version
    ,start_time
    ,end_time
    ,row_number() over (partition by usp_task_id order by app_time_usage desc) as rk
    from
    (
        select
        usp_task_id
        ,a.app_id app_id
        ,dw_appgroup
        ,tag
        ,app_time_usage
        ,spark_version
        ,case when
            (   (spark_version <>'2.1.0' and (spark_default_parallelism is not null and spark_default_parallelism <>'') and cast(task_num as int)>=cast(spark_default_parallelism as int))
                or
                (spark_version<>'2.1.0'and (spark_sql_shuffle_partitions is not null and spark_sql_shuffle_partitions <>'') and cast(task_num as int)>=cast(spark_sql_shuffle_partitions as int))
                or
                (spark_version='2.1.0' and (spark_partition_max is not null and spark_partition_max <>'') and cast(task_num as int)>=cast(spark_partition_max as int))
                or
                (spark_version='2.1.0' and (spark_partition_max ='' or spark_partition_max is null))
                or
                (spark_version<>'2.1.0' and (spark_default_parallelism ='' or spark_default_parallelism is null) and (spark_sql_shuffle_partitions ='' or spark_sql_shuffle_partitions is null))
            ) then '1' else '0' end as shuffle_split
        ,task_num max_shuffle_stage_task_num
        ,stage_time shuffle_stage_time
        ,task_time_50th shuffle_stage_task_time_50th
        ,task_time_90th shuffle_stage_task_time_90th
        ,spark_partition_min
        ,spark_partition_max
        ,spark_task_shuffle_size
        ,spark_default_parallelism
        ,spark_sql_shuffle_partitions
        ,spark_shuffle_sort_bypassMergeThreshold
        ,start_time
        ,end_time
        from app_info a
        left join spark_props b
        on a.app_id = b.app_id
    ) t1
    left join
    (
        select
        application_group
        ,max(product_name) product_name
        from internal_platform_db.app_group_product_info_history_query_engine_021
        where dt = '20260507'
        group by
        application_group
    ) t2
    on t1.dw_appgroup = t2.application_group
    where shuffle_split = '1'
)

select
    usp_task_id
    ,app_id
    ,app_time_usage
    ,tag
    ,dw_appgroup
    ,product_name
    ,shuffle_split
    ,max_shuffle_stage_task_num
    ,shuffle_stage_time
    ,shuffle_stage_task_time_50th
    ,shuffle_stage_task_time_90th
    ,spark_partition_min
    ,spark_partition_max
    ,spark_task_shuffle_size
    ,spark_default_parallelism
    ,spark_sql_shuffle_partitions
    ,spark_shuffle_sort_bypassMergeThreshold
    ,spark_version
    ,concat('https://internal-tools.example.com/optimization/appgroupfigure/config/recommended?taskId=',usp_task_id,'&applicationId=',app_id,'&startTime=',start_time,'&endTime=',end_time,'&applicationGroup=',dw_appgroup) brain_url
from task_info
where rk = 1

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,20列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_shuffle_logic,shuffle_logic,20,shuffle_split 条件逻辑（CASE WHEN 版本判断）
D,D_deduplication,deduplication,15,ROW_NUMBER 按 usp_task_id 去重
D,D_numeric_values,numeric_values,10,时间/阈值等业务指标逐行匹配
D,D_url_concat,url_concat,10,brain_url 字段拼接正确性
F,F_union_completeness,union_completeness,5,us/datawd + gputj 两个 UNION 分支
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,分区值正确

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,EXPERT
# 权重,"product=0.8, process=0.2"
# 范式,new
# Key列,usp_task_id
# 预期列数,
# 输出表,internal_platform_db.shuffle_split_tuning_rule_hit_task_info_cand_query_engine_021


Compute GPU card-hours for task instances within 5-minute windows, joining Pod mapping and task instance configuration, outputting hourly instanc

Prompt:
## Prompt
Task Objective: Compute GPU card-hours for task instances within 5-minute windows, outputting hourly instance-level GPU usage statistics.

Inputs:
- `internal_platform_db.gputj_gpu_info_parsed_query_engine_022`
- `internal_platform_db.dwd_ml_platform_instance_podname_query_engine_022`
- `internal_platform_db.dwd_task_management_taskinstance2_query_engine_022`
- `internal_platform_db.task_management_taskinstance2_query_engine_022`

Processing Rules: Join the Pod mapping table with the task instance configuration table, aggregate GPU card-hours by 5-minute windows, and roll up to hourly-level statistics. The specific join keys and aggregation logic must be determined based on the table structures.

Output Requirements: Output the task instance GPU usage statistics, including fields such as instance identifier, time window, and GPU card-hours. The specific field list must be determined based on the output table structure.

Write Requirements: Write the results to `internal_platform_db.task_instance_gpu_time_stats_cand_query_engine_022`.

Please write the final HiveSQL to `result.sql` and execute it.

INSERT OVERWRITE TABLE internal_platform_db.task_instance_gpu_time_stats_query_engine_022
PARTITION (dt='2026050700')
WITH
-- Step 1: 从GPU监控数据中提取pod运行记录，按分钟去重
pod_run_minutes AS (
    SELECT
        pod_name,
        gpu_name,
        FLOOR(CAST(pkg_time AS BIGINT) / 60) * 60 AS minute_timestamp,
        FLOOR(CAST(pkg_time AS BIGINT) / 300) * 300 AS time_5min
    FROM internal_platform_db.gputj_gpu_info_parsed_query_engine_022
    WHERE dt = '2026050700'
        AND metric IN ('k8s_container_vgpu_gpu_mem_usage', 'k8s_dcgm_fi_dev_fb_util')
    GROUP BY pod_name, gpu_name, FLOOR(CAST(pkg_time AS BIGINT) / 60) * 60, FLOOR(CAST(pkg_time AS BIGINT) / 300) * 300
),

-- Step 2: 统计每个pod在每个5分钟窗口内的运行分钟数
pod_run_time AS (
    SELECT
        pod_name,
        MAX(gpu_name) AS gpu_name,
        time_5min,
        CAST(COUNT(DISTINCT minute_timestamp) AS DOUBLE) AS run_time_m
    FROM pod_run_minutes
    GROUP BY pod_name, time_5min
),

-- Step 3: 关联pod与instance的映射关系
instance_run_time AS (
    SELECT
        p.time_5min,
        m.instance_uuid,
        MAX(p.gpu_name) AS gpu_name,
        CAST(SUM(p.run_time_m) AS DOUBLE) AS sum_run_time_m,
        COUNT(DISTINCT m.pod_name) AS pod_count
    FROM pod_run_time p
    INNER JOIN internal_platform_db.dwd_ml_platform_instance_podname_query_engine_022 m
        ON p.pod_name = m.pod_name
        AND m.dt = '2026050700'
    GROUP BY p.time_5min, m.instance_uuid
),

-- Step 4: 获取任务实例的GPU配置
latest_task_config AS (
    SELECT
        instance_uuid,
        CAST(host_gpu_num AS DOUBLE) AS host_gpu_num,
        CAST(host_num AS DOUBLE) AS host_num
    FROM (
        SELECT
            instance_uuid,
            host_gpu_num,
            host_num,
            last_modify,
            ROW_NUMBER() OVER (PARTITION BY instance_uuid ORDER BY last_modify DESC) AS rn
        FROM (
            SELECT
                instance_uuid,
                host_gpu_num,
                host_num,
                last_modify
            FROM internal_platform_db.dwd_task_management_taskinstance2_query_engine_022
            WHERE databus_imp_date = (
                SELECT MAX(databus_imp_date)
                FROM internal_platform_db.dwd_task_management_taskinstance2_query_engine_022
            )

            UNION ALL

            SELECT
                instance_uuid,
                host_gpu_num,
                host_num,
                last_modify
            FROM internal_platform_db.task_management_taskinstance2_query_engine_022
            WHERE databus_imp_date = (
                SELECT MAX(databus_imp_date)
                FROM internal_platform_db.task_management_taskinstance2_query_engine_022
            )
        ) combined
    ) ranked
    WHERE rn = 1
)

-- Step 5: 关联GPU配置并计算卡时
SELECT
    i.instance_uuid,
    i.time_5min,
    t.host_gpu_num,
    i.sum_run_time_m,
    CASE
        WHEN t.host_gpu_num IS NOT NULL AND i.sum_run_time_m IS NOT NULL
        THEN t.host_gpu_num * i.sum_run_time_m / 60.0
        ELSE NULL
    END AS gpu_hour,
    i.gpu_name,
    i.pod_count,
    t.host_num
FROM instance_run_time i
LEFT JOIN latest_task_config t
    ON i.instance_uuid = t.instance_uuid

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,8列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_gpu_hour_calculation,gpu_hour_calculation,30,gpu_hour 核心指标计算正确性
D,D_window_aggregation,window_aggregation,15,"sum_run_time_m, pod_count 等窗口聚合指标"
D,D_gpu_config_lookup,gpu_config_lookup,10,"host_gpu_num, host_num 等 GPU 配置列匹配"
F,F_join_completeness,join_completeness,5,4张源表都被关联
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,dt=2026050700

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,HARD
# 权重,"product=0.7, process=0.3"
# 范式,new
# Key列,"instance_uuid, time_5min"
# 预期列数,
# 输出表,internal_platform_db.task_instance_gpu_time_stats_cand_query_engine_022


从输入表中筛选状态为 killed 的 Notebook runner trace 数据，按天拆分后

Prompt:
## Prompt
1) 任务目标
产出 killed 状态的 Notebook 实例 Pod 运行明细宽表。

2) 输入
- internal_platform_db.gputj_gpu_info_parsed_agg_1min_query_engine_024
- internal_platform_db.notebook_span_info_query_engine_024
- internal_platform_db.dwd_gputj_service_instance_map_query_engine_024
- internal_platform_db.dwd_ml_platform_instance_podname_query_engine_024
- internal_platform_db.notebook_engine_info_query_engine_024

3) 处理规则
- 过滤条件：筛选 Notebook runner trace 中状态为 killed 的记录
- 时间处理：对 runner trace 数据按天拆分
- 关联顺序：runner trace → 服务实例 → Pod → GPU 聚合 → engine 信息
- 关联方式：按各表对应键字段进行左连接

4) 输出要求
- 输出表：internal_platform_db.dwd_notebook_killed_instance_pod_detail_d_cand_query_engine_024
- 包含 runner trace、服务实例、Pod、GPU、engine 等关联后的明细字段
- 去重规则：按主键去重（若存在）

5) 写入要求
- 写入模式：INSERT OVERWRITE
- 分区字段：按天分区（如有）

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT OVERWRITE TABLE internal_platform_db.dwd_notebook_killed_instance_pod_detail_d_query_engine_024 PARTITION (dt = '20260507')
WITH base_trace AS (
    SELECT trace_id
    FROM internal_platform_db.notebook_span_info_query_engine_024
    WHERE databus_imp_date >= '2026050700'
      AND databus_imp_date <= '2026050700'
      AND compute_type = 'ray'
      AND service_name = 'notebook-runner'
      AND span_name IN ('runner.execute', 'runner.killed')
    GROUP BY trace_id
    HAVING
        COUNT(CASE WHEN span_name = 'runner.killed' THEN 1 END) > 0
        AND
        COUNT(CASE WHEN span_name = 'runner.execute' THEN 1 END) = 0
),
combined_spans AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM internal_platform_db.notebook_span_info_query_engine_024
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.killed', 'runner.start', 'execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute')

  UNION ALL

  SELECT
      trace_id,
      'runner.execute' AS span_name,
      MAX(CASE WHEN span_name = 'runner.start' THEN start_time END) AS start_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN end_time END) AS end_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_project_id END) AS datawd_project_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_id END) AS datawd_task_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_instance_id END) AS datawd_task_instance_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN compute_type END) AS compute_type,
      2 AS status_code
  FROM internal_platform_db.notebook_span_info_query_engine_024
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.start', 'runner.killed')
  GROUP BY trace_id
),
base_data_time_fixed AS (
  SELECT
      trace_id,
      span_name,
      CASE
          WHEN span_name = 'runner.killed'
            THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                    OVER(PARTITION BY trace_id)
          ELSE start_time
      END AS start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM combined_spans
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      from_unixtime(CAST(start_time AS BIGINT) / 1000) as start_date,
      from_unixtime(CAST(end_time AS BIGINT) / 1000) as end_date,
      datediff(from_unixtime(CAST(end_time AS BIGINT) / 1000), from_unixtime(CAST(start_time AS BIGINT) / 1000)) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        date_add(b.start_date, s.pos) AS calc_date,
        CASE WHEN s.pos = 0 THEN b.start_time
             ELSE CAST(unix_timestamp(CAST(date_add(b.start_date, s.pos) AS TIMESTAMP)) * 1000 AS STRING)
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN b.end_time
             ELSE CAST((unix_timestamp(CAST(date_add(b.start_date, s.pos + 1) AS TIMESTAMP)) * 1000) - 1 AS STRING)
        END AS end_time,
        b.start_time AS span_start_time,
        b.end_time AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS instance_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS code_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS resource_wait_time,
        from_unixtime(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(start_time AS BIGINT) END) / 1000) AS code_start_time,
        from_unixtime(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(end_time AS BIGINT) END) / 1000) AS code_end_time,
        from_unixtime(MIN(CASE WHEN span_name = 'runner.execute' THEN CAST(span_start_time AS BIGINT) END) / 1000) AS instance_start_time,
        from_unixtime(MAX(CASE WHEN span_name = 'runner.execute' THEN CAST(span_end_time AS BIGINT) END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
),
gputj_dim AS (
  SELECT
    t1.service_id,
    t2.pod_name
  FROM internal_platform_db.dwd_gputj_service_instance_map_query_engine_024 t1
  INNER JOIN internal_platform_db.dwd_ml_platform_instance_podname_query_engine_024 t2
    ON t1.instance_uuid = t2.instance_uuid
    AND t1.dt=if('2026050700'>='2025080900', '2026050700', '2025080900')
    AND t2.dt=if('2026050700'>='2025060519', '2026050700', '2025060519')
  GROUP BY t1.service_id, t2.pod_name
),
gpu_metrics AS (
  SELECT
    pkg_agg_time,
    pod_name,
    k8s_container_vgpu_gpu_util_sum / k8s_container_vgpu_gpu_util_count AS gpu_util,
    k8s_container_resource_request_gpu_sum / k8s_container_resource_request_gpu_count AS gpu_count
  FROM internal_platform_db.gputj_gpu_info_parsed_agg_1min_query_engine_024
  WHERE dt >= '2026050400'
    AND dt <= '2026050700'
    AND ( k8s_container_vgpu_gpu_util_count > 0 OR k8s_container_resource_request_gpu_count > 0 )
    AND pod_name IS NOT NULL
)
SELECT
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  CAST(t1.instance_run_time AS INT),
  CAST(t1.code_run_time AS INT),
  CAST(t1.resource_wait_time AS INT),
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t2.serving_id,
  CAST(t2.is_permanent AS BOOLEAN),
  CAST(t2.apply_for_gpu_count AS INT),
  t3.pod_name,
  t4.pkg_agg_time,
  t4.gpu_util,
  CAST(t4.gpu_count AS INT),
  CAST(t1.calc_date AS STRING) AS p_date
FROM trace_time_metrics t1 LEFT JOIN (
  SELECT
    trace_id,
    MAX(serving_id) AS serving_id,
    MAX(is_permanent) AS is_permanent,
    SUM(CAST(replicas AS DOUBLE) * CAST(num_gpu AS DOUBLE)) AS apply_for_gpu_count
  FROM internal_platform_db.notebook_engine_info_query_engine_024
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND compute_type = 'ray'
    AND service_name = 'notebook-runner'
  GROUP BY trace_id
) t2 ON t1.trace_id = t2.trace_id
LEFT JOIN gputj_dim t3 ON t2.serving_id = CAST(t3.service_id AS STRING)
LEFT JOIN gpu_metrics t4
  ON t3.pod_name = t4.pod_name
  AND (
    CASE
      WHEN t1.instance_start_time IS NOT NULL AND t1.instance_end_time IS NOT NULL
        THEN t4.pkg_agg_time >= t1.instance_start_time AND t4.pkg_agg_time <= t1.instance_end_time
      WHEN t1.code_start_time IS NOT NULL AND t1.code_end_time IS NOT NULL
        THEN t4.pkg_agg_time >= t1.code_start_time AND t4.pkg_agg_time <= t1.code_end_time
      ELSE FALSE
    END
  )
;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,22列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_time_calculation,time_calculation,25,"instance_run_time, code_run_time 等时间指标"
D,D_cross_day_split,cross_day_split,15,跨天 trace 按天拆分正确性
D,D_gpu_join,gpu_join,15,"serving_id, gpu_util 等 JOIN 维度列匹配"
F,F_join_completeness,join_completeness,5,5张源表都被关联
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,分区值正确

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,EXPERT
# 权重,"product=0.8, process=0.2"
# 范式,new
# Key列,"trace_id, p_date"
# 预期列数,
# 输出表,internal_platform_db.dwd_notebook_killed_instance_pod_detail_d_cand_query_engine_024


从 notebook_span_info_query_engine_026 和 notebook_engin

Prompt:
## Prompt
任务目标：从 Notebook span 信息和 engine 信息关联产出 killed 实例运行明细表。

输入：
- internal_platform_db.notebook_span_info_query_engine_026
- internal_platform_db.notebook_engine_info_query_engine_026

处理规则：
- 两表关联查询，关联条件需基于表结构确定
- 具体过滤条件和字段选择需基于表结构确定

输出要求：
- 输出表：internal_platform_db.dwd_notebook_killed_instance_detail_d_cand_query_engine_026
- 输出字段需基于目标表结构确定

写入要求：
- 写入目标表

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT overwrite TABLE internal_platform_db.dwd_notebook_killed_instance_detail_d_query_engine_026 PARTITION (dt = '20260507')
WITH base_trace AS (
    SELECT trace_id
    FROM internal_platform_db.notebook_span_info_query_engine_026
    WHERE databus_imp_date >= '2026050700'
      AND databus_imp_date <= '2026050700'
      AND compute_type = 'ray'
      AND service_name = 'notebook-runner'
      AND span_name IN ('runner.execute', 'runner.killed')
    GROUP BY trace_id
    HAVING
        COUNT(CASE WHEN span_name = 'runner.killed' THEN 1 END) > 0
        AND
        COUNT(CASE WHEN span_name = 'runner.execute' THEN 1 END) = 0
),
combined_spans AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM internal_platform_db.notebook_span_info_query_engine_026
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.killed', 'runner.start', 'execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute')

  UNION ALL

  SELECT
      trace_id,
      'runner.execute' AS span_name,
      MAX(CASE WHEN span_name = 'runner.start' THEN start_time END) AS start_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN end_time END) AS end_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_project_id END) AS datawd_project_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_id END) AS datawd_task_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_instance_id END) AS datawd_task_instance_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN compute_type END) AS compute_type,
      2 AS status_code
  FROM internal_platform_db.notebook_span_info_query_engine_026
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.start', 'runner.killed')
  GROUP BY trace_id
),
base_data_time_fixed AS (
  SELECT
      trace_id,
      span_name,
      CASE
          WHEN span_name = 'runner.killed'
            THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                    OVER(PARTITION BY trace_id)
          ELSE start_time
      END AS start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM combined_spans
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      from_unixtime(CAST(start_time AS BIGINT) / 1000) as start_date,
      from_unixtime(CAST(end_time AS BIGINT) / 1000) as end_date,
      datediff(from_unixtime(CAST(end_time AS BIGINT) / 1000), from_unixtime(CAST(start_time AS BIGINT) / 1000)) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        date_add(b.start_date, s.pos) AS calc_date,
        CASE WHEN s.pos = 0 THEN b.start_time
             ELSE CAST(unix_timestamp(cast(date_add(b.start_date, s.pos) as timestamp)) * 1000 AS STRING)
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN b.end_time
             ELSE CAST((unix_timestamp(cast(date_add(b.start_date, s.pos + 1) as timestamp)) * 1000) - 1 AS STRING)
        END AS end_time,
        b.start_time AS span_start_time,
        b.end_time AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS instance_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS code_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN CAST(end_time AS BIGINT) END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN CAST(start_time AS BIGINT) END)) / 1000.0) AS resource_wait_time,
        from_unixtime(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(start_time AS BIGINT) END) / 1000) AS code_start_time,
        from_unixtime(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN CAST(end_time AS BIGINT) END) / 1000) AS code_end_time,
        from_unixtime(MIN(CASE WHEN span_name = 'runner.execute' THEN CAST(span_start_time AS BIGINT) END) / 1000) AS instance_start_time,
        from_unixtime(MAX(CASE WHEN span_name = 'runner.execute' THEN CAST(span_end_time AS BIGINT) END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
)
SELECT
  t1.calc_date AS p_date,
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  CAST(t1.instance_run_time AS INT) AS instance_run_time,
  CAST(t1.code_run_time AS INT) AS code_run_time,
  CAST(t1.resource_wait_time AS INT) AS resource_wait_time,
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t2.serving_id,
  CAST(t2.is_permanent AS BOOLEAN) AS is_permanent,
  CAST(t2.apply_for_gpu_count AS INT) AS apply_for_gpu_count
FROM trace_time_metrics t1 LEFT JOIN (
  SELECT
    trace_id,
    MAX(serving_id) AS serving_id,
    MAX(is_permanent) AS is_permanent,
    SUM(CAST(replicas AS INT) * CAST(num_gpu AS INT)) AS apply_for_gpu_count
  FROM internal_platform_db.notebook_engine_info_query_engine_026
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND compute_type = 'ray'
    AND service_name = 'notebook-runner'
  GROUP BY trace_id
) t2 ON t1.trace_id = t2.trace_id
;

维度,维度全名,子维度,满分,说明
A,A_executability,executability,10,result.sql 能跑通且产出非空
B,B_schema,schema,10,18列 + 列名匹配
C,C_row_alignment,row_consistency,10,行数比例 + key覆盖率
D,D_time_calculation,time_calculation,30,"instance_run_time, code_run_time 等时间指标"
D,D_cross_day_split,cross_day_split,15,跨天 trace 按天拆分正确性
D,D_engine_join,engine_join,10,"serving_id, is_permanent 等 engine 维度列匹配"
F,F_join_completeness,join_completeness,5,2张源表都被关联
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,dt=20260507

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(10) + B(10) + C(10) + D(55) + F(15)
# 难度,HARD
# 权重,"product=0.7, process=0.3"
# 范式,new
# Key列,"trace_id, p_date"
# 预期列数,
# 输出表,internal_platform_db.dwd_notebook_killed_instance_detail_d_cand_query_engine_026


From input table internal_platform_db.t_ed_socialbook_qq_high_v2_ta

Prompt:
## Prompt
Task Objective
Filter out specified `tag_id` values from the tag incremental table and write the results to the target table.

Input
- `internal_platform_db.t_ed_socialbook_qq_high_v2_tag_guid_incr_query_engine_100`

Processing Rules
1. Filter condition: `ds = 20260608` AND `tag_id NOT IN (10212021124, 10606061135, 10606061136, 10808011103)`
2. No joins, single-table processing
3. Field order: `guid`, `tag_id`, `tag_value`, `redis_sub_key`, `val_type`, `splitter1`, `splitter2`, `limit_size`, `tag_status`

Output Requirements
- Output table: `internal_platform_db.t_ed_socialbook_qq_high_v2_tag_guid_incr_filter_cand_query_engine_100`
- Output fields (9 in total, fixed order): `guid`, `tag_id`, `tag_value`, `redis_sub_key`, `val_type`, `splitter1`, `splitter2`, `limit_size`, `tag_status`
- Partition column: `ds` (BIGINT type)

Write Requirements
- Write strategy: `INSERT OVERWRITE`
- Partition value: `ds=20260608`
- Write format: ORC

Please write the final HiveSQL to `result.sql` and execute it.

insert overwrite table internal_platform_db.t_ed_socialbook_qq_high_v2_tag_guid_incr_filter_query_engine_100 partition(ds=20260608)
select
        guid,
        tag_id,
        tag_value,
        redis_sub_key,
        val_type,
        splitter1,
        splitter2,
        limit_size,
        tag_status
    from internal_platform_db.t_ed_socialbook_qq_high_v2_tag_guid_incr_query_engine_100
    where  ds = 20260608 and tag_id not in (10212021124, 10606061135, 10606061136, 10808011103)

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空/非空串比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION
F,F_partition_value,partition_value,5,ds 分区值 = 20260608
F,F_source_filter,source_filter,10,源表分区过滤 ds=20260608 + tag_id NOT IN

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"guid, tag_id"
# 预期列数,10
# 输出表,internal_platform_db.t_ed_socialbook_qq_high_v2_tag_guid_incr_filter_cand_query_engine_100


Prompt:
## Prompt
任务目标：统计广告素材在各维度下的行为指标汇总，并生成一条更新日期记录。

输入：
- internal_platform_db.dws_ug_app_ad_material_action_di_query_engine_101

处理规则：
- 无 Join，单表处理
- 过滤条件：imp_date = 20260608
- 维度字段转换（使用 CASE WHEN）：
  - material_type: 1→'图片', 2→'视频'
  - source: 2→'人工素材', 3→'人机素材', 6→'AI素材'
  - video_model_type: 1→'镜头拆分模型(老模型)', 2→'剧情理解模型(新模型)'
  - create_type: 0→'无视频剪辑方式', 1→'高光剪辑', 2→'原始剪辑', 3→'拼合剪辑', 4→'二创剪辑', 5→'定点剪辑', 6→'拆条前贴', 7→'拆条剪辑', ELSE→'未知代码'+create_type
  - purpose: 1→'拉新', 2→'拉活', 3→'唤醒', 4→'收入', 5→'电商', 6→'当日拉活'
  - provider: 1→'AI侧', 2→'内部平台D侧'
- 分组维度：上述转换后的维度字段 + channel_name
- 聚合指标（均为 SUM）：product_flag→production_cnt, audit_flag→audit_cnt, pass_flag→pass_cnt, push_madia_flag→push_cnt, imp_madia_gdt_flag→imp_cnt, click_madia_gdt_flag→clck_cnt, imp_madia_cnt→imp, click_madia_cnt→clck
- 结果由两部分 UNION ALL 构成：第一部分为分组聚合结果，第二部分为一条 update_date=20260608 且其余字段均为 NULL 的记录

输出要求：
- 输出字段顺序：update_date, material_type_name, source_name, video_model_type_name, create_type_name, purpose_name, provider_name, production_cnt, audit_cnt, pass_cnt, push_cnt, imp_cnt, clck_cnt, imp, clck, channel_name, imp_date
- 不去重

写入要求：
- 目标表：internal_platform_db.ads_ug_app_ad_material_action_di_cand_query_engine_101
- 写入方式：INSERT OVERWRITE
- 分区：imp_date=20260608

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT OVERWRITE TABLE internal_platform_db.ads_ug_app_ad_material_action_di_query_engine_101 PARTITION (imp_date=20260608)
            SELECT
                20260608 AS update_date
                , CASE
                    WHEN material_type = 1 THEN '图片'
                    WHEN material_type = 2 THEN '视频'
                END AS material_type_name
                , CASE
                    WHEN `source` = 2 THEN '人工素材'
                    WHEN `source` = 3 THEN '人机素材'
                    WHEN `source` = 6 THEN 'AI素材'
                END AS source_name
                , CASE
                    WHEN video_model_type = 1 THEN '镜头拆分模型(老模型)'
                    WHEN video_model_type = 2 THEN '剧情理解模型(新模型)'
                END AS video_model_type_name
                , CASE
                    WHEN create_type = 0 THEN '无视频剪辑方式'
                    WHEN create_type = 1 THEN '高光剪辑'
                    WHEN create_type = 2 THEN '原始剪辑'
                    WHEN create_type = 3 THEN '拼合剪辑'
                    WHEN create_type = 4 THEN '二创剪辑'
                    WHEN create_type = 5 THEN '定点剪辑'
                    WHEN create_type = 6 THEN '拆条前贴'
                    WHEN create_type = 7 THEN '拆条剪辑'
                    ELSE CONCAT('未知代码', create_type)
                END AS create_type_name
                , CASE
                    WHEN purpose = 1 THEN '拉新'
                    WHEN purpose = 2 THEN '拉活'
                    WHEN purpose = 3 THEN '唤醒'
                    WHEN purpose = 4 THEN '收入'
                    WHEN purpose = 5 THEN '电商'
                    WHEN purpose = 6 THEN '当日拉活'
                END AS purpose_name
                , CASE
                    WHEN `provider` = 1 THEN 'AI侧'
                    WHEN `provider` = 2 THEN '内部平台D侧'
                END AS provider_name
                , SUM(product_flag) AS production_cnt
                , SUM(audit_flag) AS audit_cnt
                , SUM(pass_flag) AS pass_cnt
                , SUM(push_madia_flag) AS push_cnt
                , SUM(imp_madia_gdt_flag) AS imp_cnt
                , SUM(click_madia_gdt_flag) AS clck_cnt
                , SUM(imp_madia_cnt) AS imp
                , SUM(click_madia_cnt) AS clck
                , channel_name
            FROM internal_platform_db.dws_ug_app_ad_material_action_di_query_engine_101
            WHERE imp_date = 20260608
            GROUP BY
                CASE
                    WHEN material_type = 1 THEN '图片'
                    WHEN material_type = 2 THEN '视频'
                END
                , CASE
                    WHEN `source` = 2 THEN '人工素材'
                    WHEN `source` = 3 THEN '人机素材'
                    WHEN `source` = 6 THEN 'AI素材'
                END
                , CASE
                    WHEN video_model_type = 1 THEN '镜头拆分模型(老模型)'
                    WHEN video_model_type = 2 THEN '剧情理解模型(新模型)'
                END
                , CASE
                    WHEN create_type = 0 THEN '无视频剪辑方式'
                    WHEN create_type = 1 THEN '高光剪辑'
                    WHEN create_type = 2 THEN '原始剪辑'
                    WHEN create_type = 3 THEN '拼合剪辑'
                    WHEN create_type = 4 THEN '二创剪辑'
                    WHEN create_type = 5 THEN '定点剪辑'
                    WHEN create_type = 6 THEN '拆条前贴'
                    WHEN create_type = 7 THEN '拆条剪辑'
                    ELSE CONCAT('未知代码', create_type)
                END
                , CASE
                    WHEN purpose = 1 THEN '拉新'
                    WHEN purpose = 2 THEN '拉活'
                    WHEN purpose = 3 THEN '唤醒'
                    WHEN purpose = 4 THEN '收入'
                    WHEN purpose = 5 THEN '电商'
                    WHEN purpose = 6 THEN '当日拉活'
                END
                , CASE
                    WHEN `provider` = 1 THEN 'AI侧'
                    WHEN `provider` = 2 THEN '内部平台D侧'
                END
                , channel_name

            UNION ALL
            SELECT
                20260608 AS update_date
                , NULL AS material_type_name
                , NULL AS source_name
                , NULL AS video_model_type_name
                , NULL AS create_type_name
                , NULL AS purpose_name
                , NULL AS provider_name
                , NULL AS production_cnt
                , NULL AS audit_cnt
                , NULL AS pass_cnt
                , NULL AS push_cnt
                , NULL AS imp_cnt
                , NULL AS clck_cnt
                , NULL AS imp
                , NULL AS clck
                , NULL AS channel_name

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数17 + 列名匹配
C,C_row_alignment,row_consistency,15,行数比例 + key覆盖率
D,D_case_when_labels,case_when_labels,20,7个维度标签列逐行匹配
D,D_agg_metrics,agg_metrics,20,8个聚合列按权重匹配
F,F_union_all_sentinel,union_all_sentinel,10,是否存在NULL哨兵行
F,F_insert_overwrite,insert_overwrite,5,写入模式+分区
F,F_partition_value,partition_value,5,imp_date=20260608

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"material_type_name, source_name, channel_name"
# 预期列数,
# 输出表,internal_platform_db.ads_ug_app_ad_material_action_di_cand_query_engine_101


Filter add-friend behavior from internal_platform_db.log_17047_query_engine_102

Prompt:
## Prompt
Task Objective:
Perform cluster aggregation analysis on add-friend behavior logs to identify malicious clusters, and output statistical metrics for each cluster.

Input:
- `internal_platform_db.log_17047_query_engine_102` (add-friend behavior log table)

Processing Rules:
1. No joins, single-table processing, using UNION of two subqueries
2. Time filter: `day_` between 20260608–20260609, `hour_` between 2026060823–2026060900, `timestamp_` between 202606082320 and 202606090020
3. Branch 1 filter: `appname_ in ('app_hello_txt', 'app_add_contact')`, `uinregcountry_ in ('CN','HK','MO')`, `touinregcountry_='CN'`, `length(headmd5_)>0`, `commfrinum_=0`
4. Branch 2 filter: `appname_='app_contact_verify_ok'`, `uinregcountry_ in ('CN','HK','MO')`, `touinregcountry_='CN'`, `commfrinum_=0`
5. Grouping fields: `appname_`, `clientversion_`, `scene_`, `ticketscene_`, `headmd5_` (empty string for branch 2), `uinipcountryid_`, `uinipprovinceid_`
6. Aggregation fields: `addfri_pv=count(*)`, `user_id_cnt=count(distinct user_id_)`, `low_quality_cnt=sum(if(uinhighquality_=0,1,0))`, `low_quality_rate=low_quality_cnt/addfri_pv`, `user_id_list=concat_ws(',',collect_set(cast(user_id_ as string)))`, `hello_content_list=concat_ws('|',collect_set(content_))`, `evil_cnt=sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10,1,0))`, `evil_rate=evil_cnt/addfri_pv`
7. HAVING filter: `user_id_cnt>20` AND (`low_quality_rate>0.99` OR `evil_rate>0.98`)

Output Requirements:
Field order: `appname_`, `clientversion_`, `scene_`, `ticketscene_`, `headmd5_`, `uinipcountryid_`, `uinipprovinceid_`, `addfri_pv`, `user_id_cnt`, `low_quality_cnt`, `low_quality_rate`, `user_id_list`, `hello_content_list`, `evil_cnt`, `evil_rate`

Write Requirements:
`INSERT INTO internal_platform_db.t_acct_addfri_action_cluster_minutely_cand_query_engine_102 PARTITION(ds=202606090010)`

Please write the final HiveSQL to `result.sql` and execute it.

INSERT INTO TABLE internal_platform_db.t_acct_addfri_action_cluster_minutely_query_engine_102 PARTITION(ds=202606090010)
select
appname_,
clientversion_,
scene_,
ticketscene_,
headmd5_,
uinipcountryid_,
uinipprovinceid_,
count(*) as addfri_pv,
count(distinct user_id_) as user_id_cnt,
sum(if(uinhighquality_=0, 1, 0)) as low_quality_cnt,
sum(if(uinhighquality_=0, 1, 0))/count(*) as low_quality_rate,
concat_ws(",", collect_set(cast(user_id_ as string))) as user_id_list,
concat_ws("|", collect_set(content_)) as hello_content_list,
sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0)) as evil_cnt,
sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0))/count(*) as evil_rate
from internal_platform_db.log_17047_query_engine_102
where  date_format(day_, 'yyyyMMdd')>=20260608 and date_format(day_, 'yyyyMMdd')<=20260609
and date_format(hour_, 'yyyyMMddHH')>=2026060823 and date_format(hour_, 'yyyyMMddHH')<=2026060900
and timestamp_>=unix_timestamp('202606082320','yyyyMMddHHmm')
and timestamp_<unix_timestamp('202606090020','yyyyMMddHHmm')
and appname_ in ('app_hello_txt', 'app_add_contact')
and uinregcountry_ in ("CN", "HK", "MO")
and touinregcountry_="CN"
and length(headmd5_)>0
and commfrinum_=0
group by  appname_, clientversion_, scene_, ticketscene_, headmd5_, uinipcountryid_, uinipprovinceid_
having count(distinct user_id_)>20
and (    sum(if(uinhighquality_=0, 1, 0))/count(*) > 0.99
or  sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0))/count(*) > 0.98)
union
select
appname_,
clientversion_,
scene_,
ticketscene_,
"",
uinipcountryid_,
uinipprovinceid_,
count(*) as addfri_pv,
count(distinct user_id_) as user_id_cnt,
sum(if(uinhighquality_=0, 1, 0)) as low_quality_cnt,
sum(if(uinhighquality_=0, 1, 0))/count(*) as low_quality_rate,
concat_ws(",", collect_set(cast(user_id_ as string))) as user_id_list,
concat_ws("|", collect_set(content_)) as hello_content_list,
sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0)) as evil_cnt,
sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0))/count(*) as evil_rate
from internal_platform_db.log_17047_query_engine_102
where date_format(day_, 'yyyyMMdd')>=20260608 and date_format(day_, 'yyyyMMdd')<=20260609
and date_format(hour_, 'yyyyMMddHH')>=2026060823 and date_format(hour_, 'yyyyMMddHH')<=2026060900
and timestamp_>=unix_timestamp('202606082320','yyyyMMddHHmm')
and timestamp_<unix_timestamp('202606090020','yyyyMMddHHmm')
and appname_ ="app_contact_verify_ok"
and uinregcountry_ in ("CN", "HK", "MO")
and touinregcountry_="CN"
and commfrinum_=0
group by  appname_, clientversion_, scene_, ticketscene_, uinipcountryid_, uinipprovinceid_
having count(distinct user_id_)>20
and (    sum(if(uinhighquality_=0, 1, 0))/count(*) > 0.99
or  sum(if(uinlastunbantime_>0 or opentime_+86400*90>timestamp_ or friendnum_<10, 1, 0))/count(*) > 0.98)

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数16 + 列名匹配
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_agg_metrics,agg_metrics,40,"6个聚合指标列逐行匹配(addfri_pv,user_id_cnt,low_quality_rate,evil_rate,low_quality_cnt,evil_cnt)"
F,F_insert_overwrite,insert_overwrite,5,写入模式应为 INSERT INTO
F,F_partition_value,partition_value,5,ds=202606090010
F,F_union_branches,union_branches,10,UNION两路分支均存在(app_hello_txt/app_add_contact + app_contact_verify_ok)

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,"appname_, scene_, ticketscene_, headmd5_, uinipprovinceid_"
# 预期列数,
# 输出表,internal_platform_db.t_acct_addfri_action_cluster_minutely_cand_query_engine_102


Prompt:
## Prompt
Task Objective: Incrementally insert new user records (`user_id`, `loss_day`) into the `ds=20260608` partition of the target table that have not appeared before.

Inputs:
- `internal_platform_db.t_mg_dws_user_tag_preset_24128_query_engine_103` (source table, containing fields such as `ds`, `user_id`, `loss_day`)
- `internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_query_engine_103` (existing users table, containing previously written `user_id` values)

Processing Rules:
1. Select records from the source table `internal_platform_db.t_mg_dws_user_tag_preset_24128_query_engine_103` where `ds=20260608`
2. Use a `NOT IN` subquery to exclude `user_id` values that already exist in the existing users table `internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_query_engine_103` (deduplication condition: `user_id`)
3. Extract fields: `user_id`, `loss_day`

Output Requirements:
- Output field order: `user_id`, `loss_day`
- No deduplication needed (duplicates are already filtered out via `NOT IN`)

Write Requirements:
- Target table: `internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_cand_query_engine_103`
- Write method: `INSERT INTO` (append)

Please write the final HiveSQL to `result.sql` and execute it.

insert into  internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_query_engine_103 (user_id, loss_day)
select user_id,loss_day
from internal_platform_db.t_mg_dws_user_tag_preset_24128_query_engine_103
where
ds = 20260608
and user_id not in (select user_id from internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_query_engine_103)

user_id,loss_day
user_a,5
user_b,3
user_e,1

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数匹配 + 列名匹配
C,C_row_alignment,row_consistency,15,行数比例 + key覆盖率
D,D_anti_join,anti_join,25,NOT IN 排除已存在 user_id 的正确性
D,D_value_correctness,value_correctness,20,loss_day 等数值列逐行匹配
F,F_insert_overwrite,insert_mode,5,INSERT INTO（非 OVERWRITE）
F,F_partition_value,partition_filter,5,只取 ds=20260608
F,F_partition_value,partition_value,5,分区值正确

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(45) + F(15)
# 难度,MEDIUM
# 权重,"product=0.6, process=0.4"
# 范式,new
# Key列,user_id
# 预期列数,
# 输出表,internal_platform_db.t_mg_dws_user_tag_user_id_once_24128_cand_query_engine_103


从用户活跃日志表统计每日各渠道的活跃用户数

Prompt:
## Prompt
任务目标：统计各渠道的日活跃用户数。

输入：
- internal_platform_db.dwd_user_activity_log_hi_query_engine_104（用户活跃日志表）

处理规则：
1. 过滤条件：imp_date = 20260608
2. 分组维度：channel
3. 聚合指标：COUNT(DISTINCT user_id) AS dau
4. 派生列：imp_date 固定值 20260608

输出要求：
- 字段顺序：imp_date（BIGINT，固定值20260608）、channel（STRING）、dau（BIGINT）
- 按 channel 升序排列

写入要求：
- 输出表：internal_platform_db.dws_user_daily_active_by_channel_cand_query_engine_104
- 写入方式：INSERT OVERWRITE
- 分区：ds=20260608

请将最终 HiveSQL 写入 result.sql 并执行。

insert overwrite table internal_platform_db.dws_user_daily_active_by_channel_query_engine_104 partition(ds='20260608')
select
20260608 as imp_date
, channel
, count(distinct user_id) as dau
from
internal_platform_db.dwd_user_activity_log_hi_query_engine_104
where
imp_date = 20260608
group by
channel
order by
channel

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数(4=3数据+1分区) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,25,非key字段逐列值匹配率
D,D_field_completeness,field_completeness,15,关键字段非空比例
F,F_insert_overwrite,insert_overwrite,5,INSERT OVERWRITE + PARTITION 写入模式
F,F_partition_value,partition_value,5,ds=20260608
F,F_source_filter,source_filter,10,源表分区过滤 imp_date=20260608

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(40) + F(20)
# 难度,EASY
# 权重,"product=0.5, process=0.5"
# 范式,new
# Key列,channel
# 预期列数,4
# 输出表,internal_platform_db.dws_user_daily_active_by_channel_cand_query_engine_104


从事件表和全量用户表 Join 过滤出 20260608 当天最后登录用户的事件，按 sdk_id

Prompt:
## Prompt
任务目标：按 SDK 类型和小时统计活跃用户数。

输入：
- internal_platform_db.dws_event_tracking_ul2l6h5c_events_di_query_engine_114（事件明细表）
- internal_platform_db.dws_event_tracking_ul2l6h5c_all_user_df_query_engine_114（用户维度表）

处理规则：
- Join 条件：e.user_id = u.user_id
- 过滤条件：e.imp_date = 20260608，u.imp_date = 20260608，u.last_login_date = 20260608
- 分组维度：sdk_id、event_time_hour
- 聚合指标：COUNT(DISTINCT e.user_id) as active_users
- 去重与排序：按 sdk_id 和 hour 分组，无额外排序要求

输出要求：
- 字段顺序：imp_date（固定值 20260608，BIGINT）、id（ROW_NUMBER() OVER()）、sdk_id（STRING）、hour（STRING，来源 event_time_hour）、active_users（BIGINT）

写入要求：
- 输出表：internal_platform_db.ads_event_tracking_ul2l6h5c_user_active_hour_di_cand_query_engine_114
- 写入方式：INSERT INTO

请将最终 HiveSQL 写入 result.sql 并执行。

INSERT INTO internal_platform_db.ads_event_tracking_ul2l6h5c_user_active_hour_di_query_engine_114
SELECT
20260608 as `imp_date`
, ROW_NUMBER() OVER(ORDER BY e.`sdk_id`, e.`event_time_hour`) as `id`
, e.`sdk_id` as `sdk_id`
, e.`event_time_hour` as `hour`
, COUNT(DISTINCT e.user_id) as `active_users`
FROM
internal_platform_db.dws_event_tracking_ul2l6h5c_events_di_query_engine_114 e
JOIN
internal_platform_db.dws_event_tracking_ul2l6h5c_all_user_df_query_engine_114 u
ON
u.imp_date = 20260608
and u.user_id = e.user_id
and u.last_login_date = 20260608
WHERE
e.imp_date = 20260608
GROUP BY
e.`sdk_id`
, e.`event_time_hour`

维度,维度全名,子维度,满分,说明
A,A_executability,executability,15,result.sql 能跑通且产出非空
B,B_schema,schema,10,列数(5) + 列名匹配(5)
C,C_row_alignment,row_consistency,15,行数比例(7) + key覆盖率(8)
D,D_field_value_match,field_value_match,45,非key字段逐列值匹配率
F,F_insert_overwrite,insert_mode,5,INSERT INTO 写入模式
F,F_source_filter,source_coverage,5,2张源表均被引用
F,F_join_completeness,join_condition,5,JOIN条件包含user_id

总计,,,100,

# 评分模板,评分范式: 产物100分 = A(15) + B(10) + C(15) + D(45) + F(15)
# 难度,MEDIUM
# 权重,"product=0.6, process=0.4"
# 范式,new
# Key列,"imp_date, id, sdk_id"
# 预期列数,5
# 输出表,internal_platform_db.ads_event_tracking_ul2l6h5c_user_active_hour_di_cand_query_engine_114