Spaces:
Running
Running
stouchinglhh Claude commited on
Commit ·
bb59211
1
Parent(s): d7a054c
sync processor from GitLab c697b6e: 版本 1.19.4 / 前端 1.12.4 —— 服务端 i18n(首屏不再闪英文)、推理模块并入模型页、前端抽出 shared/、缓存键改用前端号
Browse files顺带:.gitattributes 补 processor/**/*.task 走 LFS(MediaPipe 手部模型 7.5MB,
HF 不收非 LFS 二进制;与 collector 侧同一条规则)。
Co-Authored-By: Claude <noreply@anthropic.com>
This view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +2 -0
- processor/.env.example +2 -2
- processor/.gitignore +8 -0
- processor/README.md +14 -4
- processor/README.zh-CN.md +12 -4
- processor/app/ai_annotation.py +52 -26
- processor/app/api/quality.py +172 -0
- processor/app/api/worker.py +24 -4
- processor/app/cleaning.py +25 -4
- processor/app/config.py +21 -0
- processor/app/hdf5_export.py +97 -14
- processor/app/i18n.py +89 -0
- processor/app/lerobot_export.py +76 -0
- processor/app/localstore.py +30 -0
- processor/app/main.py +17 -1
- processor/app/processing/black_glove/hand_landmarker.task +3 -0
- processor/app/processing/cleaning/__init__.py +40 -0
- processor/app/processing/cleaning/checks/__init__.py +294 -0
- processor/app/processing/cleaning/checks/cross/__init__.py +5 -0
- processor/app/processing/cleaning/checks/depth/__init__.py +5 -0
- processor/app/processing/cleaning/checks/ego/__init__.py +5 -0
- processor/app/processing/cleaning/checks/glove/__init__.py +5 -0
- processor/app/processing/cleaning/checks/glove/array_shape.py +82 -0
- processor/app/processing/cleaning/checks/glove/device_status.py +107 -0
- processor/app/processing/cleaning/checks/glove/frozen_array.py +132 -0
- processor/app/processing/cleaning/checks/glove/gate_floor.py +87 -0
- processor/app/processing/cleaning/checks/glove/no_contact.py +70 -0
- processor/app/processing/cleaning/checks/umi/__init__.py +5 -0
- processor/app/processing/cleaning/checks/umi/action_present.py +118 -0
- processor/app/processing/cleaning/checks/umi/av_sync.py +84 -0
- processor/app/processing/cleaning/checks/umi/gripper_range.py +157 -0
- processor/app/processing/cleaning/checks/umi/slam_continuity.py +158 -0
- processor/app/processing/cleaning/checks/umi/slam_coverage.py +105 -0
- processor/app/processing/cleaning/checks/video/__init__.py +5 -0
- processor/app/processing/cleaning/checks/video/black_screen.py +86 -0
- processor/app/processing/cleaning/checks/video/decode_error.py +99 -0
- processor/app/processing/cleaning/checks/video/frame_drop.py +82 -0
- processor/app/processing/cleaning/checks/video/freeze.py +86 -0
- processor/app/processing/cleaning/collect.py +647 -0
- processor/app/processing/cleaning/contract.py +274 -0
- processor/app/processing/cleaning/engine.py +302 -0
- processor/app/processing/cleaning/report.py +250 -0
- processor/app/processing/cleaning/store.py +189 -0
- processor/app/processing/cleaning/trigger.py +225 -0
- processor/app/processing/modules/ai_quality_review.py +0 -36
- processor/app/processing/modules/data_quality.py +126 -0
- processor/app/project_dataset.py +60 -7
- processor/app/routes/export.py +47 -4
- processor/app/routes/pages.py +34 -4
- processor/app/routes/training.py +78 -0
.gitattributes
CHANGED
|
@@ -44,3 +44,5 @@ collector/**/*.task filter=lfs diff=lfs merge=lfs -text
|
|
| 44 |
|
| 45 |
# processor 文档截图走 LFS(HF 拒绝非 LFS 二进制)
|
| 46 |
processor/**/*.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
# processor 文档截图走 LFS(HF 拒绝非 LFS 二进制)
|
| 46 |
processor/**/*.png filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
# MediaPipe 手部模型(7.5MB 二进制)—— 与 collector 侧同一条规则,HF 不收非 LFS 二进制
|
| 48 |
+
processor/**/*.task filter=lfs diff=lfs merge=lfs -text
|
processor/.env.example
CHANGED
|
@@ -13,8 +13,8 @@ SFTP_CONNECT_TIMEOUT=15
|
|
| 13 |
API_KEY=change-me
|
| 14 |
WORKER_API_KEY=change-me-worker-key
|
| 15 |
JWT_SECRET=change-me-jwt-secret
|
| 16 |
-
EGODATA_BOOTSTRAP_USERNAME=
|
| 17 |
-
EGODATA_BOOTSTRAP_PASSWORD=
|
| 18 |
EGODATA_BOOTSTRAP_EMAIL=root@egodata.local
|
| 19 |
DEFAULT_WORKFLOW_ID=
|
| 20 |
WORKER_LEASE_SECONDS=90
|
|
|
|
| 13 |
API_KEY=change-me
|
| 14 |
WORKER_API_KEY=change-me-worker-key
|
| 15 |
JWT_SECRET=change-me-jwt-secret
|
| 16 |
+
EGODATA_BOOTSTRAP_USERNAME=Root
|
| 17 |
+
EGODATA_BOOTSTRAP_PASSWORD=Root
|
| 18 |
EGODATA_BOOTSTRAP_EMAIL=root@egodata.local
|
| 19 |
DEFAULT_WORKFLOW_ID=
|
| 20 |
WORKER_LEASE_SECONDS=90
|
processor/.gitignore
CHANGED
|
@@ -33,6 +33,14 @@ web/workflow-studio/*.tsbuildinfo
|
|
| 33 |
*.db
|
| 34 |
*.log
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
# Local model assets and scratch (large binaries, never in git).
|
| 37 |
models/
|
| 38 |
*.deb
|
|
|
|
| 33 |
*.db
|
| 34 |
*.log
|
| 35 |
|
| 36 |
+
# 编辑器/工具留下的备份与冲突残留文件 —— 一律不入库。
|
| 37 |
+
# 起因:`web/static/plyr.js.bak`(改 plyr.js 时留下的备份)被误提交进了版本库,
|
| 38 |
+
# 而仓库里没有任何规则拦它。改 JS 时留的 .bak 尤其容易混进来。
|
| 39 |
+
*.bak
|
| 40 |
+
*.orig
|
| 41 |
+
*.rej
|
| 42 |
+
*~
|
| 43 |
+
|
| 44 |
# Local model assets and scratch (large binaries, never in git).
|
| 45 |
models/
|
| 46 |
*.deb
|
processor/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
|
| 5 |
[English](README.md) | [中文](README.zh-CN.md)
|
| 6 |
|
| 7 |
-
> Documentation: 2026-09-
|
| 8 |
|
| 9 |
processor is EgoData's service for data capture, processing, review, and export. It is
|
| 10 |
designed for embodied AI and robot manipulation data. The system consists of a FastAPI backend,
|
|
@@ -523,12 +523,22 @@ Troubleshooting order:
|
|
| 523 |
<details>
|
| 524 |
<summary>Click to expand: version maintenance rules</summary>
|
| 525 |
|
| 526 |
-
- The backend version is maintained in `app/version.py`;
|
| 527 |
-
- the
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 528 |
- increment the patch version for fixes and small adjustments, the minor version for new features,
|
| 529 |
and the major version for architectural changes;
|
| 530 |
- before release, record the software version, data format, and API changes together.
|
| 531 |
|
| 532 |
-
Current versions: backend `1.
|
| 533 |
|
| 534 |
</details>
|
|
|
|
| 4 |
|
| 5 |
[English](README.md) | [中文](README.zh-CN.md)
|
| 6 |
|
| 7 |
+
> Documentation: 2026-09-28 · Backend: `1.19.4` · Frontend: `1.12.3`
|
| 8 |
|
| 9 |
processor is EgoData's service for data capture, processing, review, and export. It is
|
| 10 |
designed for embodied AI and robot manipulation data. The system consists of a FastAPI backend,
|
|
|
|
| 523 |
<details>
|
| 524 |
<summary>Click to expand: version maintenance rules</summary>
|
| 525 |
|
| 526 |
+
- The backend version is maintained in `app/version.py` and covers the backend **only**;
|
| 527 |
+
- **the frontend version is maintained in `web/VERSION`**. It covers every piece of frontend
|
| 528 |
+
code under `processor/web/` (`templates/`, `static/`, `workflow-studio/`) — **not just the
|
| 529 |
+
React SPA**. The version inside `web/workflow-studio/package.json` only represents the SPA
|
| 530 |
+
itself; all three numbers are independent and do not need to stay in sync.
|
| 531 |
+
- **Bump `web/VERSION` — that one file — whenever you change the frontend.** A single number
|
| 532 |
+
serves two purposes: ① the `?v={{ asset_version }}` cache key on static assets (so browsers
|
| 533 |
+
re-fetch changed JS/CSS), ② the "frontend X" part of commit messages. Sharing one number
|
| 534 |
+
means the commit message can never disagree with what the browser actually loaded.
|
| 535 |
+
You do **not** need to touch the backend version for a frontend change.
|
| 536 |
+
- `web/VERSION` is read once at startup and cached (`app/web_version.py`), so **restart the
|
| 537 |
+
service** after bumping it.
|
| 538 |
- increment the patch version for fixes and small adjustments, the minor version for new features,
|
| 539 |
and the major version for architectural changes;
|
| 540 |
- before release, record the software version, data format, and API changes together.
|
| 541 |
|
| 542 |
+
Current versions: backend `1.19.4`, frontend `1.12.3`.
|
| 543 |
|
| 544 |
</details>
|
processor/README.zh-CN.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
[中文](README.zh-CN.md) | [English](README.md)
|
| 4 |
|
| 5 |
-
> 文档版本:2026-09-
|
| 6 |
|
| 7 |
processor 是 EgoData 的数据采集、处理、审核和导出服务,面向具身智能与机器人
|
| 8 |
操作数据。系统由 FastAPI 后端、异步 Worker、Web 审核页面和 Workflow Studio 组成。
|
|
@@ -506,11 +506,19 @@ npm run build
|
|
| 506 |
<details>
|
| 507 |
<summary>点击展开:版本维护规则</summary>
|
| 508 |
|
| 509 |
-
- 后端版本在 `app/version.py` 中维护;
|
| 510 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 511 |
- 修复和小调整增加修订号;新增功能增加次版本号;重大架构变更增加主版本号;
|
| 512 |
- 发布前应同步记录软件版本、数据格式和 API 变化。
|
| 513 |
|
| 514 |
-
当前版本:后端 `1.
|
| 515 |
|
| 516 |
</details>
|
|
|
|
| 2 |
|
| 3 |
[中文](README.zh-CN.md) | [English](README.md)
|
| 4 |
|
| 5 |
+
> 文档版本:2026-09-28 · 后端版本:`1.19.4` · 前端版本:`1.12.3`
|
| 6 |
|
| 7 |
processor 是 EgoData 的数据采集、处理、审核和导出服务,面向具身智能与机器人
|
| 8 |
操作数据。系统由 FastAPI 后端、异步 Worker、Web 审核页面和 Workflow Studio 组成。
|
|
|
|
| 506 |
<details>
|
| 507 |
<summary>点击展开:版本维护规则</summary>
|
| 508 |
|
| 509 |
+
- 后端版本在 `app/version.py` 中维护,**只管后端**;
|
| 510 |
+
- **前端版本在 `web/VERSION` 中维护**。它覆盖 `processor/web/` 下的全部前端代码
|
| 511 |
+
(`templates/`、`static/`、`workflow-studio/`),**不只是那个 React SPA**。
|
| 512 |
+
`web/workflow-studio/package.json` 里的 version 只代表 SPA 自己,三者互相独立、不必同步。
|
| 513 |
+
- **改了前端就只 bump `web/VERSION` 这一个文件**。它一个号管两件事:
|
| 514 |
+
① 模板里的 `?v={{ asset_version }}` 缓存键(读了它浏览器才会重新拉 JS/CSS)
|
| 515 |
+
② 提交信息里的「前端 X」。
|
| 516 |
+
两个用途共用一个数,不会出现"提交信息写 1.12.3、浏览器却拿 1.12.2"这种对不上的情况。
|
| 517 |
+
改前端**不需要**动后端版本号。
|
| 518 |
+
- 改完 `web/VERSION` 要**重启服务**才生效(`app/web_version.py` 启动时读一次就缓存)。
|
| 519 |
- 修复和小调整增加修订号;新增功能增加次版本号;重大架构变更增加主版本号;
|
| 520 |
- 发布前应同步记录软件版本、数据格式和 API 变化。
|
| 521 |
|
| 522 |
+
当前版本:后端 `1.19.4`,前端 `1.12.3`。
|
| 523 |
|
| 524 |
</details>
|
processor/app/ai_annotation.py
CHANGED
|
@@ -41,7 +41,6 @@ from app.localstore import (
|
|
| 41 |
read_episode_state,
|
| 42 |
write_episode_state,
|
| 43 |
)
|
| 44 |
-
from app.routes.annotations import notify_annotations_changed
|
| 45 |
from app.video_quality import check_video_quality_async
|
| 46 |
from app.workflow_types import HAND_PROCESS_TYPES, canonical_node_type
|
| 47 |
|
|
@@ -612,23 +611,36 @@ def _set_ai_quality_state(episode_id: str, report: dict, *,
|
|
| 612 |
|
| 613 |
The web UI continues to use only Reviewing/Approved. The detailed report
|
| 614 |
is kept in episode state for diagnostics and future review tooling.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 615 |
"""
|
| 616 |
-
|
| 617 |
-
|
| 618 |
-
|
| 619 |
-
|
| 620 |
-
|
| 621 |
-
|
| 622 |
-
|
| 623 |
-
|
| 624 |
-
|
| 625 |
-
|
| 626 |
-
|
| 627 |
-
|
| 628 |
-
|
| 629 |
-
|
| 630 |
-
|
| 631 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 632 |
|
| 633 |
|
| 634 |
def _set_ai_quality_pending(episode_id: str) -> None:
|
|
@@ -2697,6 +2709,11 @@ def _persist_ai_segment(episode_id: str, generation: int, candidate: dict,
|
|
| 2697 |
index: int, media_sources: list[str], *,
|
| 2698 |
pending: bool = False, error: str = "") -> dict:
|
| 2699 |
"""Atomically insert/update one segment without waiting for its siblings."""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2700 |
record = _ai_annotation_record(
|
| 2701 |
episode_id, candidate, index, media_sources,
|
| 2702 |
pending=pending, error=error)
|
|
@@ -2823,6 +2840,8 @@ def run_ai_annotation(episode_id: str, mode: str, min_confidence: float,
|
|
| 2823 |
走本地 vLLM 抽帧图;provider=api 走 Kimi 原生视频 —— 两模式严格
|
| 2824 |
分离,配置缺失/请求失败显式报错(任务 detail 可见),绝不互回退。
|
| 2825 |
"""
|
|
|
|
|
|
|
| 2826 |
episode_id = str(episode_id)
|
| 2827 |
quality_review_enabled = bool(quality_gate or video_quality_gate)
|
| 2828 |
if quality_review_enabled:
|
|
@@ -3538,7 +3557,7 @@ def _connected_quality_node_config(graph: dict, node_configs: dict,
|
|
| 3538 |
}
|
| 3539 |
quality_nodes = {
|
| 3540 |
str(node.get("id")): node for node in nodes
|
| 3541 |
-
if canonical_node_type((node.get("data") or {}).get("nodeType")) == "
|
| 3542 |
and node.get("id") is not None
|
| 3543 |
}
|
| 3544 |
if not source_ids or not quality_nodes:
|
|
@@ -3570,25 +3589,32 @@ def _connected_quality_node_config(graph: dict, node_configs: dict,
|
|
| 3570 |
return None
|
| 3571 |
|
| 3572 |
|
| 3573 |
-
def
|
| 3574 |
-
"""Return the
|
| 3575 |
|
| 3576 |
A card sitting alone in the canvas is configuration only. The gate is
|
| 3577 |
-
enabled for a directed path ``AI Annotation -> ... ->
|
| 3578 |
allowing pass-through nodes in the middle keeps existing workflow layouts
|
| 3579 |
compatible while making a dangling quality card inert.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3580 |
"""
|
| 3581 |
return _connected_quality_node_config(
|
| 3582 |
graph, node_configs, {"ai_annotation"})
|
| 3583 |
|
| 3584 |
|
| 3585 |
-
def
|
| 3586 |
-
"""Return the
|
| 3587 |
|
| 3588 |
This enables a video-only workflow to run the post-processing media gate,
|
| 3589 |
-
while an unconnected ``
|
| 3590 |
intentionally contains existing input/process node types only; export
|
| 3591 |
nodes cannot accidentally activate a review gate.
|
|
|
|
|
|
|
|
|
|
| 3592 |
"""
|
| 3593 |
return _connected_quality_node_config(
|
| 3594 |
graph,
|
|
@@ -3656,7 +3682,7 @@ def _workflow_ai_quality_enabled(episode_id: str) -> bool:
|
|
| 3656 |
wf = get_workflow(wf_id)
|
| 3657 |
if wf is None:
|
| 3658 |
continue
|
| 3659 |
-
if
|
| 3660 |
wf.get("graph") or {}, wf.get("node_configs") or {}) is not None:
|
| 3661 |
return True
|
| 3662 |
return False
|
|
@@ -3683,7 +3709,7 @@ def _workflow_video_quality_enabled(episode_id: str) -> bool:
|
|
| 3683 |
wf = get_workflow(wf_id)
|
| 3684 |
if wf is None:
|
| 3685 |
continue
|
| 3686 |
-
if
|
| 3687 |
wf.get("graph") or {}, wf.get("node_configs") or {}) is not None:
|
| 3688 |
return True
|
| 3689 |
return False
|
|
|
|
| 41 |
read_episode_state,
|
| 42 |
write_episode_state,
|
| 43 |
)
|
|
|
|
| 44 |
from app.video_quality import check_video_quality_async
|
| 45 |
from app.workflow_types import HAND_PROCESS_TYPES, canonical_node_type
|
| 46 |
|
|
|
|
| 611 |
|
| 612 |
The web UI continues to use only Reviewing/Approved. The detailed report
|
| 613 |
is kept in episode state for diagnostics and future review tooling.
|
| 614 |
+
|
| 615 |
+
★ ``auto_approve`` 会被【数据质检】否决 —— 判 FAIL/ERROR 的批次在
|
| 616 |
+
``cleaning_summary.blocking`` 上留了粘性标记。AI 标注覆盖率合格不代表
|
| 617 |
+
数据本身没问题;只看自己那份报告的话,坏数据会被一路自动批准放行
|
| 618 |
+
(质检与视频门禁是并发跑的,谁先写没有保证,所以必须查标记而不是
|
| 619 |
+
比谁写得晚)。
|
| 620 |
"""
|
| 621 |
+
from app.localstore import mutate_episode_state
|
| 622 |
+
from app.processing.cleaning.store import is_approval_blocked
|
| 623 |
+
|
| 624 |
+
def _apply(state: dict) -> None:
|
| 625 |
+
state["ai_quality_status"] = "passed" if report.get("passed") else "failed"
|
| 626 |
+
state["ai_quality_report"] = report
|
| 627 |
+
state["updated_at"] = _utcnow()
|
| 628 |
+
current = str(state.get("status") or "")
|
| 629 |
+
if report.get("passed") and auto_approve:
|
| 630 |
+
if (current in {"processing", "to_review", "completed", "failed"}
|
| 631 |
+
and not is_approval_blocked(state)):
|
| 632 |
+
state["status"] = "reviewed"
|
| 633 |
+
if not state.get("approved_at"):
|
| 634 |
+
state["approved_at"] = _utcnow()
|
| 635 |
+
elif not report.get("passed"):
|
| 636 |
+
# Fail closed: an incomplete/failed AI result remains in Reviewing.
|
| 637 |
+
if current in {"processing", "completed", "reviewed", "approved", "failed"}:
|
| 638 |
+
state["status"] = "to_review"
|
| 639 |
+
state["approved_at"] = None
|
| 640 |
+
|
| 641 |
+
# 锁内 RMW:数据质检 / 视频门禁在同一次 run 后并发写同一份 state。分开
|
| 642 |
+
# read→write 的话,后写的会用旧快照把先写的 ``status`` 冲掉。
|
| 643 |
+
mutate_episode_state(episode_id, _apply)
|
| 644 |
|
| 645 |
|
| 646 |
def _set_ai_quality_pending(episode_id: str) -> None:
|
|
|
|
| 2709 |
index: int, media_sources: list[str], *,
|
| 2710 |
pending: bool = False, error: str = "") -> dict:
|
| 2711 |
"""Atomically insert/update one segment without waiting for its siblings."""
|
| 2712 |
+
# 惰性导入 —— 不要提到模块顶层。本模块在路由层**之下**,而 routes.annotations
|
| 2713 |
+
# 反过来惰性引用 ai_annotation(_schedule_dataset_sync)。模块级导入会形成
|
| 2714 |
+
# import 环:谁先被导入,谁就拿不到对方完整的模块对象。放进调用点后方向保持
|
| 2715 |
+
# 单向(路由 → 本模块),将来搬文件也不会因顺序踩坑。
|
| 2716 |
+
from app.routes.annotations import notify_annotations_changed
|
| 2717 |
record = _ai_annotation_record(
|
| 2718 |
episode_id, candidate, index, media_sources,
|
| 2719 |
pending=pending, error=error)
|
|
|
|
| 2840 |
走本地 vLLM 抽帧图;provider=api 走 Kimi 原生视频 —— 两模式严格
|
| 2841 |
分离,配置缺失/请求失败显式报错(任务 detail 可见),绝不互回退。
|
| 2842 |
"""
|
| 2843 |
+
# 同 _persist_ai_segment:惰性导入,避免与 routes.annotations 形成 import 环。
|
| 2844 |
+
from app.routes.annotations import notify_annotations_changed
|
| 2845 |
episode_id = str(episode_id)
|
| 2846 |
quality_review_enabled = bool(quality_gate or video_quality_gate)
|
| 2847 |
if quality_review_enabled:
|
|
|
|
| 3557 |
}
|
| 3558 |
quality_nodes = {
|
| 3559 |
str(node.get("id")): node for node in nodes
|
| 3560 |
+
if canonical_node_type((node.get("data") or {}).get("nodeType")) == "data_quality"
|
| 3561 |
and node.get("id") is not None
|
| 3562 |
}
|
| 3563 |
if not source_ids or not quality_nodes:
|
|
|
|
| 3589 |
return None
|
| 3590 |
|
| 3591 |
|
| 3592 |
+
def annotation_coverage_gate_config(graph: dict, node_configs: dict) -> dict | None:
|
| 3593 |
+
"""Return the Data Quality card only when it is connected to AI Annotation.
|
| 3594 |
|
| 3595 |
A card sitting alone in the canvas is configuration only. The gate is
|
| 3596 |
+
enabled for a directed path ``AI Annotation -> ... -> Data Quality``;
|
| 3597 |
allowing pass-through nodes in the middle keeps existing workflow layouts
|
| 3598 |
compatible while making a dangling quality card inert.
|
| 3599 |
+
|
| 3600 |
+
以前叫 ``ai_quality_review_node_config`` —— 当时节点叫 AI Quality Review。
|
| 3601 |
+
节点合并为 Data Quality 后按"它能开启哪项检查"重命名,函数名不再绑定
|
| 3602 |
+
已废弃的 slug。
|
| 3603 |
"""
|
| 3604 |
return _connected_quality_node_config(
|
| 3605 |
graph, node_configs, {"ai_annotation"})
|
| 3606 |
|
| 3607 |
|
| 3608 |
+
def video_quality_gate_config(graph: dict, node_configs: dict) -> dict | None:
|
| 3609 |
+
"""Return the Data Quality card when it is connected to a media/data source.
|
| 3610 |
|
| 3611 |
This enables a video-only workflow to run the post-processing media gate,
|
| 3612 |
+
while an unconnected ``Data Quality`` card remains inert. The list
|
| 3613 |
intentionally contains existing input/process node types only; export
|
| 3614 |
nodes cannot accidentally activate a review gate.
|
| 3615 |
+
|
| 3616 |
+
历史上这段逻辑属于 ``AI Quality Review`` 卡片 —— 视频检查当时"寄居"在
|
| 3617 |
+
标注节点里,好让没有 AI 标注的纯视频工作流也有质量门禁。
|
| 3618 |
"""
|
| 3619 |
return _connected_quality_node_config(
|
| 3620 |
graph,
|
|
|
|
| 3682 |
wf = get_workflow(wf_id)
|
| 3683 |
if wf is None:
|
| 3684 |
continue
|
| 3685 |
+
if annotation_coverage_gate_config(
|
| 3686 |
wf.get("graph") or {}, wf.get("node_configs") or {}) is not None:
|
| 3687 |
return True
|
| 3688 |
return False
|
|
|
|
| 3709 |
wf = get_workflow(wf_id)
|
| 3710 |
if wf is None:
|
| 3711 |
continue
|
| 3712 |
+
if video_quality_gate_config(
|
| 3713 |
wf.get("graph") or {}, wf.get("node_configs") or {}) is not None:
|
| 3714 |
return True
|
| 3715 |
return False
|
processor/app/api/quality.py
ADDED
|
@@ -0,0 +1,172 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""数据质检 API —— 检查项目录、报告查询、手动触发。
|
| 2 |
+
|
| 3 |
+
给前端两件事:
|
| 4 |
+
* 设置面板的**数据源** —— ``/checks`` 返回检查项目录 + 按设备卡片的分组,
|
| 5 |
+
面板按它渲染 tab 和每一项的阈值输入框。新增检查项前端零改动。
|
| 6 |
+
* 报告的**读取入口** —— 列表页和详情页。
|
| 7 |
+
|
| 8 |
+
写端点少而克制:只有"重跑某集"。质检本身不做任何数据修改,
|
| 9 |
+
所以没有删除/清理类的接口。
|
| 10 |
+
"""
|
| 11 |
+
|
| 12 |
+
from __future__ import annotations
|
| 13 |
+
|
| 14 |
+
import asyncio
|
| 15 |
+
from pathlib import Path
|
| 16 |
+
|
| 17 |
+
from fastapi import APIRouter, HTTPException, Query
|
| 18 |
+
|
| 19 |
+
router = APIRouter(prefix="/api/v1/quality", tags=["quality"])
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def _check_catalog() -> dict:
|
| 23 |
+
"""检查项目录 + 按设备卡片分组 —— 前端设置面板的渲染数据。
|
| 24 |
+
|
| 25 |
+
``checks`` 是平铺清单(每项带 ``device_cards`` / ``default_params``);
|
| 26 |
+
``by_device`` 是按设备卡片分好组的视图,前端 tab 直接用。
|
| 27 |
+
两者给同一份数据,前端按方便取用。
|
| 28 |
+
"""
|
| 29 |
+
from app.processing.cleaning.checks import all_checks, all_check_specs
|
| 30 |
+
from app.processing.cleaning.contract import (
|
| 31 |
+
DEVICE_MODALITIES, MODALITY_CHANNELS,
|
| 32 |
+
)
|
| 33 |
+
from app.processing.cleaning.engine import ruleset_revision
|
| 34 |
+
|
| 35 |
+
specs = all_check_specs()
|
| 36 |
+
by_device: dict[str, list[str]] = {}
|
| 37 |
+
for modality in DEVICE_MODALITIES:
|
| 38 |
+
by_device[modality] = [
|
| 39 |
+
spec["slug"] for spec in specs
|
| 40 |
+
if modality in (spec.get("device_cards") or [])
|
| 41 |
+
]
|
| 42 |
+
|
| 43 |
+
return {
|
| 44 |
+
"ruleset_revision": ruleset_revision(),
|
| 45 |
+
"checks": specs,
|
| 46 |
+
"by_device": by_device,
|
| 47 |
+
"device_labels": _device_labels(),
|
| 48 |
+
"device_channels": {
|
| 49 |
+
modality: list(channels)
|
| 50 |
+
for modality, channels in MODALITY_CHANNELS.items()
|
| 51 |
+
},
|
| 52 |
+
"severity_labels": _severity_labels(),
|
| 53 |
+
"total": len(all_checks()),
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def _device_labels() -> dict[str, str]:
|
| 58 |
+
"""设备卡片的显示名 —— 复用 device_naming,不另起一套。"""
|
| 59 |
+
try:
|
| 60 |
+
from app.device_naming import DEVICE_LABELS
|
| 61 |
+
return dict(DEVICE_LABELS)
|
| 62 |
+
except Exception:
|
| 63 |
+
return {}
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def _severity_labels() -> dict[str, str]:
|
| 67 |
+
from app.processing.cleaning.contract import SEVERITY_LABELS
|
| 68 |
+
return dict(SEVERITY_LABELS)
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
@router.get("/checks")
|
| 72 |
+
def list_checks() -> dict:
|
| 73 |
+
"""检查项目录(含按设备卡片分组)—— 设置面板用。"""
|
| 74 |
+
return _check_catalog()
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
@router.get("/summary")
|
| 78 |
+
def list_summaries(project: str | None = Query(None),
|
| 79 |
+
status: str | None = Query(None),
|
| 80 |
+
limit: int = Query(200, ge=1, le=2000)) -> dict:
|
| 81 |
+
"""质检报告列表(摘要),按时间倒序。
|
| 82 |
+
|
| 83 |
+
``status`` 过滤按 episode 结论(pass / warn / fail / pending_review / error)。
|
| 84 |
+
"""
|
| 85 |
+
from app.processing.cleaning.store import list_reports
|
| 86 |
+
|
| 87 |
+
items = list_reports()
|
| 88 |
+
if project:
|
| 89 |
+
items = [item for item in items if str(item.get("project")) == project]
|
| 90 |
+
if status:
|
| 91 |
+
items = [item for item in items if item.get("status") == status]
|
| 92 |
+
|
| 93 |
+
counts: dict[str, int] = {}
|
| 94 |
+
for item in items:
|
| 95 |
+
key = str(item.get("status") or "unknown")
|
| 96 |
+
counts[key] = counts.get(key, 0) + 1
|
| 97 |
+
|
| 98 |
+
return {"reports": items[:limit], "total": len(items), "counts": counts}
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
@router.get("/episodes/{episode_id}")
|
| 102 |
+
def get_report(episode_id: str) -> dict:
|
| 103 |
+
"""某一集的完整质检报告(四层结果)。"""
|
| 104 |
+
from app.processing.cleaning.store import load_report, load_summary
|
| 105 |
+
|
| 106 |
+
report = load_report(episode_id)
|
| 107 |
+
if report is not None:
|
| 108 |
+
return report
|
| 109 |
+
summary = load_summary(episode_id)
|
| 110 |
+
if summary is not None:
|
| 111 |
+
# 完整报告被清掉了但摘要还在(state 里)—— 至少把摘要给出去
|
| 112 |
+
return {"episode_id": episode_id, "summary_only": True, **summary}
|
| 113 |
+
raise HTTPException(status_code=404, detail="No quality report for this episode")
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
@router.post("/episodes/{episode_id}/run")
|
| 117 |
+
async def run_episode_checks(
|
| 118 |
+
episode_id: str,
|
| 119 |
+
probe_video: bool = Query(False, description="是否跑视频检查(慢)"),
|
| 120 |
+
persist: bool = Query(True, description="是否把报告写进 episode state"),
|
| 121 |
+
) -> dict:
|
| 122 |
+
"""重跑某一集的质检。
|
| 123 |
+
|
| 124 |
+
跑在**线程池**里 —— 读 parquet/视频是阻塞 IO,直接放事件循环上会把
|
| 125 |
+
页面请求卡住(仓库里 ``routes/devices.py`` 有同样的处理)。
|
| 126 |
+
"""
|
| 127 |
+
episode = await asyncio.to_thread(_find_episode, episode_id)
|
| 128 |
+
if episode is None:
|
| 129 |
+
raise HTTPException(status_code=404, detail="Episode not found")
|
| 130 |
+
|
| 131 |
+
report = await asyncio.to_thread(_run_checks_for, episode, probe_video)
|
| 132 |
+
saved = False
|
| 133 |
+
if persist:
|
| 134 |
+
from app.processing.cleaning.store import save_report
|
| 135 |
+
saved = await asyncio.to_thread(save_report, episode_id, report)
|
| 136 |
+
return {"episode_id": episode_id, "saved": saved, "report": report}
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
def _find_episode(episode_id: str) -> dict | None:
|
| 140 |
+
"""在会话索引里找这一集(含 ``path``,质检要用它读数据)。"""
|
| 141 |
+
from app.localstore import scan_sessions
|
| 142 |
+
|
| 143 |
+
for episode in scan_sessions():
|
| 144 |
+
if str(episode.get("id")) == episode_id:
|
| 145 |
+
return episode
|
| 146 |
+
return None
|
| 147 |
+
|
| 148 |
+
|
| 149 |
+
def _run_checks_for(episode: dict, probe_video: bool) -> dict:
|
| 150 |
+
from app.processing.cleaning.engine import run_checks
|
| 151 |
+
|
| 152 |
+
path = Path(str(episode.get("path") or ""))
|
| 153 |
+
if not path.is_dir():
|
| 154 |
+
raise HTTPException(status_code=409, detail="Episode data directory missing")
|
| 155 |
+
|
| 156 |
+
# 设备卡片取自 episode 的设备信息 —— 与工作流派发用的是同一套口径
|
| 157 |
+
modality = None
|
| 158 |
+
try:
|
| 159 |
+
from app.workflow_dispatch import _episode_input_groups
|
| 160 |
+
groups = _episode_input_groups(episode) or []
|
| 161 |
+
if groups:
|
| 162 |
+
modality = str(groups[0].get("input_type") or "") or None
|
| 163 |
+
except Exception:
|
| 164 |
+
modality = None
|
| 165 |
+
|
| 166 |
+
return run_checks(
|
| 167 |
+
path,
|
| 168 |
+
episode_id=str(episode.get("id") or ""),
|
| 169 |
+
project=str(episode.get("project") or ""),
|
| 170 |
+
probe_video=probe_video,
|
| 171 |
+
modality_hint=modality,
|
| 172 |
+
)
|
processor/app/api/worker.py
CHANGED
|
@@ -520,14 +520,14 @@ async def complete_job(
|
|
| 520 |
from app.ai_annotation import (
|
| 521 |
run_ai_annotation,
|
| 522 |
ai_annotation_node_config,
|
| 523 |
-
|
| 524 |
-
|
| 525 |
)
|
| 526 |
graph = run.get("graph") or {}
|
| 527 |
node_configs = run.get("node_configs") or {}
|
| 528 |
cfg = ai_annotation_node_config(graph, node_configs)
|
| 529 |
-
quality_cfg =
|
| 530 |
-
video_quality_cfg =
|
| 531 |
if cfg is not None and not run.get("ai_suggest_triggered"):
|
| 532 |
run["ai_suggest_triggered"] = True
|
| 533 |
save_run(run)
|
|
@@ -572,6 +572,26 @@ async def complete_job(
|
|
| 572 |
except Exception as exc:
|
| 573 |
print(f"[Worker] AI annotation trigger skipped: {exc}")
|
| 574 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 575 |
# 归档联动:处理完成后后台把该批次增量推送到 NAS(只推不删;rsync
|
| 576 |
# 逐块校验+断点续传,失败仅记日志不阻塞响应;每晚 cron 兜底补传)。
|
| 577 |
# 脚本不存在(如 Windows 环境)时静默跳过。
|
|
|
|
| 520 |
from app.ai_annotation import (
|
| 521 |
run_ai_annotation,
|
| 522 |
ai_annotation_node_config,
|
| 523 |
+
annotation_coverage_gate_config,
|
| 524 |
+
video_quality_gate_config,
|
| 525 |
)
|
| 526 |
graph = run.get("graph") or {}
|
| 527 |
node_configs = run.get("node_configs") or {}
|
| 528 |
cfg = ai_annotation_node_config(graph, node_configs)
|
| 529 |
+
quality_cfg = annotation_coverage_gate_config(graph, node_configs)
|
| 530 |
+
video_quality_cfg = video_quality_gate_config(graph, node_configs)
|
| 531 |
if cfg is not None and not run.get("ai_suggest_triggered"):
|
| 532 |
run["ai_suggest_triggered"] = True
|
| 533 |
save_run(run)
|
|
|
|
| 572 |
except Exception as exc:
|
| 573 |
print(f"[Worker] AI annotation trigger skipped: {exc}")
|
| 574 |
|
| 575 |
+
# 数据质检联动:工作流里 Data Quality 卡片**接了数据链路**就跑一次检查。
|
| 576 |
+
#
|
| 577 |
+
# 与上面两条是两套东西:ai_annotation / video_quality 会改 episode 状态
|
| 578 |
+
# (通过→reviewed、失败→卡到 to_review),而质检**只把结论记进**
|
| 579 |
+
# state 的 cleaning_report,不参与状态流转。
|
| 580 |
+
#
|
| 581 |
+
# 两者会同时触发(相机类工作流就同时命中 video_quality_gate 和这里),而且
|
| 582 |
+
# 都各要解码一遍视频 —— 所以它们几乎同时结束、同时写状态,状态写入一律走
|
| 583 |
+
# localstore.mutate_episode_state(锁内 RMW),否则后写的会用旧快照把先写的
|
| 584 |
+
# 冲掉(尤其 status)。
|
| 585 |
+
#
|
| 586 |
+
# 必须放后台任务:probe_video 要解码视频(实测约 4.8s/集),不能拖住完成响应。
|
| 587 |
+
try:
|
| 588 |
+
from app.processing.cleaning.trigger import spawn_cleaning_for_run
|
| 589 |
+
if ep is not None and spawn_cleaning_for_run(run, ep):
|
| 590 |
+
print(f"[Worker] Data quality checks triggered "
|
| 591 |
+
f"for {run['episode_id']}")
|
| 592 |
+
except Exception as exc:
|
| 593 |
+
print(f"[Worker] Data quality trigger skipped: {exc}")
|
| 594 |
+
|
| 595 |
# 归档联动:处理完成后后台把该批次增量推送到 NAS(只推不删;rsync
|
| 596 |
# 逐块校验+断点续传,失败仅记日志不阻塞响应;每晚 cron 兜底补传)。
|
| 597 |
# 脚本不存在(如 Windows 环境)时静默跳过。
|
processor/app/cleaning.py
CHANGED
|
@@ -281,20 +281,41 @@ def _check_frame_alignment_fs(batch_dir: Path) -> dict:
|
|
| 281 |
except Exception:
|
| 282 |
continue
|
| 283 |
parquet_frames = len(seen)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 284 |
video_frames = 0
|
| 285 |
-
for mp4 in
|
| 286 |
cap = cv2.VideoCapture(str(mp4))
|
| 287 |
try:
|
| 288 |
video_frames = max(video_frames, int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 0)
|
| 289 |
finally:
|
| 290 |
cap.release()
|
| 291 |
-
if parquet_frames == 0:
|
| 292 |
-
return {"name": "frame_alignment", "passed": True, "detail": "No parquet data"}
|
| 293 |
ratio = abs(parquet_frames - video_frames) / max(video_frames, 1)
|
| 294 |
return {"name": "frame_alignment", "passed": ratio < 0.10,
|
| 295 |
"detail": f"parquet={parquet_frames}, video={video_frames}, dev={ratio:.1%}"}
|
| 296 |
except Exception as e:
|
| 297 |
-
|
|
|
|
|
|
|
|
|
|
| 298 |
|
| 299 |
|
| 300 |
def _check_sensor_completeness_fs(batch_dir: Path) -> dict:
|
|
|
|
| 281 |
except Exception:
|
| 282 |
continue
|
| 283 |
parquet_frames = len(seen)
|
| 284 |
+
if parquet_frames == 0:
|
| 285 |
+
return {"name": "frame_alignment", "passed": True, "detail": "No parquet data"}
|
| 286 |
+
|
| 287 |
+
# 2026-09-23 修:此处此前直接用 cv2 却没 import,异常被下方 except 吞掉,
|
| 288 |
+
# 整个帧对齐检查在文件级路径下**永远静默通过**(detail 里写的是
|
| 289 |
+
# "Check error (skipped)" 但 passed=True,不会引起任何人注意)。
|
| 290 |
+
try:
|
| 291 |
+
import cv2
|
| 292 |
+
except ImportError:
|
| 293 |
+
return {"name": "frame_alignment", "passed": True,
|
| 294 |
+
"detail": "OpenCV not available — video frame count skipped"}
|
| 295 |
+
|
| 296 |
+
mp4s = sorted(batch_dir.rglob("*.mp4"))
|
| 297 |
+
if not mp4s:
|
| 298 |
+
# 与 _check_video_decode_fs 同款处理:没有视频就没有可比对的对象,
|
| 299 |
+
# 判"未适用"而不是"不匹配"——只有传感器的批次(如纯手套)是合法的。
|
| 300 |
+
# 修复 cv2 漏 import 后,这里若不给守卫会退化成 dev=10000% 的误报。
|
| 301 |
+
return {"name": "frame_alignment", "passed": True,
|
| 302 |
+
"detail": "No video to compare against"}
|
| 303 |
+
|
| 304 |
video_frames = 0
|
| 305 |
+
for mp4 in mp4s:
|
| 306 |
cap = cv2.VideoCapture(str(mp4))
|
| 307 |
try:
|
| 308 |
video_frames = max(video_frames, int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 0)
|
| 309 |
finally:
|
| 310 |
cap.release()
|
|
|
|
|
|
|
| 311 |
ratio = abs(parquet_frames - video_frames) / max(video_frames, 1)
|
| 312 |
return {"name": "frame_alignment", "passed": ratio < 0.10,
|
| 313 |
"detail": f"parquet={parquet_frames}, video={video_frames}, dev={ratio:.1%}"}
|
| 314 |
except Exception as e:
|
| 315 |
+
# 保持 passed=True(不改变既有行为),但 detail 必须说清是"降级通过",
|
| 316 |
+
# 免得再出现"看起来全过了其实根本没检查"的情况。
|
| 317 |
+
return {"name": "frame_alignment", "passed": True,
|
| 318 |
+
"detail": f"Degraded pass — check errored: {e}"}
|
| 319 |
|
| 320 |
|
| 321 |
def _check_sensor_completeness_fs(batch_dir: Path) -> dict:
|
processor/app/config.py
CHANGED
|
@@ -56,6 +56,27 @@ class Settings(BaseSettings):
|
|
| 56 |
EXPORT_SHARD_SIZE: int = 100000
|
| 57 |
VIDEO_SHARD_FRAMES: int = 5000
|
| 58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
model_config = {
|
| 60 |
"env_prefix": "",
|
| 61 |
"case_sensitive": False,
|
|
|
|
| 56 |
EXPORT_SHARD_SIZE: int = 100000
|
| 57 |
VIDEO_SHARD_FRAMES: int = 5000
|
| 58 |
|
| 59 |
+
# Training. Checkpoints are produced *outside* the platform by the official
|
| 60 |
+
# ``lerobot-train`` CLI (conda env ``lerobot052_base_piper``), so this points
|
| 61 |
+
# at an existing directory we only ever read. Empty = use the default below.
|
| 62 |
+
TRAIN_OUTPUT_ROOT: str = ""
|
| 63 |
+
|
| 64 |
+
@property
|
| 65 |
+
def train_root(self) -> Path:
|
| 66 |
+
"""Trained-model directory — **read-only**, never created here.
|
| 67 |
+
|
| 68 |
+
Defaults to ``<repo>/文件测试/umi_act_model``, i.e. beside ``processor/``.
|
| 69 |
+
The directory is git-ignored and may legitimately be absent (fresh
|
| 70 |
+
clone, or training never run on this host): callers must treat a
|
| 71 |
+
missing path as "no runs" rather than creating it. Creating it would
|
| 72 |
+
make an empty training dir appear out of nowhere on every boot.
|
| 73 |
+
"""
|
| 74 |
+
raw = str(self.TRAIN_OUTPUT_ROOT or "").strip()
|
| 75 |
+
if raw:
|
| 76 |
+
return Path(raw).expanduser()
|
| 77 |
+
return (Path(__file__).resolve().parent.parent.parent
|
| 78 |
+
/ "文件测试" / "umi_act_model")
|
| 79 |
+
|
| 80 |
model_config = {
|
| 81 |
"env_prefix": "",
|
| 82 |
"case_sensitive": False,
|
processor/app/hdf5_export.py
CHANGED
|
@@ -34,6 +34,7 @@ from app.localstore import get_episode
|
|
| 34 |
from app.lerobot_export import (
|
| 35 |
_annotation_maps,
|
| 36 |
_calibration_docs,
|
|
|
|
| 37 |
_hand_columns,
|
| 38 |
_infer_hand_3d_unit,
|
| 39 |
_key_matches_any,
|
|
@@ -288,6 +289,66 @@ def build_hdf5_dataset(dataset_name: str, episode_ids: list[str], output_dir: Pa
|
|
| 288 |
"video_info": {"video.fps": vfps, "video.is_depth_map": False},
|
| 289 |
}
|
| 290 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 291 |
# 2) 表列:传感器 + 手部骨骼 + 标注(与 LeRobot 列同构,按帧号对齐)
|
| 292 |
columns: dict[str, dict[int, Any]] = {}
|
| 293 |
str_columns: set[str] = set()
|
|
@@ -406,11 +467,11 @@ def build_hdf5_dataset(dataset_name: str, episode_ids: list[str], output_dir: Pa
|
|
| 406 |
annotation_defs, ensure_ascii=False)
|
| 407 |
|
| 408 |
# 6) 标定 → /meta/calibration/<episode>/<name> attrs(JSON 字符串)
|
| 409 |
-
#
|
| 410 |
-
#
|
| 411 |
-
# 回退
|
| 412 |
cal_group = meta.create_group(f"calibration/{episode_id}")
|
| 413 |
-
cal_docs = _calibration_docs(session_dir)
|
| 414 |
for name, value in cal_docs.items():
|
| 415 |
cal_group.attrs[name] = json.dumps(value, ensure_ascii=False)
|
| 416 |
|
|
@@ -506,7 +567,16 @@ def _build_readme(dataset_name: str, h5: Any) -> str:
|
|
| 506 |
img_keys = sorted(n for n in all_names
|
| 507 |
if n.startswith(f"{root}/observation/images/"))
|
| 508 |
if img_keys:
|
| 509 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 510 |
hand_coordinate = next(
|
| 511 |
(coord for coord in ("world", "3d", "2d")
|
| 512 |
if f"{root}/observation/state/hand_left_{coord}" in all_names),
|
|
@@ -558,6 +628,27 @@ with open("hand_skeleton.mp4", "wb") as f:
|
|
| 558 |
```
|
| 559 |
"""
|
| 560 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 561 |
readme = f"""# {dataset_name} 数据文件说明(HDF5 格式)
|
| 562 |
|
| 563 |
> 本文件由 EgoData 数据采集系统工作流自动生成
|
|
@@ -588,15 +679,7 @@ with open("hand_skeleton.mp4", "wb") as f:
|
|
| 588 |
## 四、如何读取
|
| 589 |
|
| 590 |
```python
|
| 591 |
-
|
| 592 |
-
|
| 593 |
-
with h5py.File("dataset.h5", "r") as h5:
|
| 594 |
-
# 读一帧左目图像
|
| 595 |
-
img = h5["episode_000000/observation/images/stereo_left"][0]
|
| 596 |
-
# 读第一帧右手骨骼(字段可能是 world / 3d / 2d)
|
| 597 |
-
hand3d = h5["episode_000000/observation/state/hand_right_{hand_coordinate or '3d'}"][0]
|
| 598 |
-
# 读全部标注
|
| 599 |
-
labels = h5["episode_000000/observation/annotation"][:]
|
| 600 |
```
|
| 601 |
{video_extract}
|
| 602 |
其他工具:HDFView、Panoply、MATLAB(h5read)等支持 HDF5 的软件均可打开。
|
|
|
|
| 34 |
from app.lerobot_export import (
|
| 35 |
_annotation_maps,
|
| 36 |
_calibration_docs,
|
| 37 |
+
_depth_video_paths,
|
| 38 |
_hand_columns,
|
| 39 |
_infer_hand_3d_unit,
|
| 40 |
_key_matches_any,
|
|
|
|
| 289 |
"video_info": {"video.fps": vfps, "video.is_depth_map": False},
|
| 290 |
}
|
| 291 |
|
| 292 |
+
# 1b) 深度流 → 逐帧 uint16 毫米(单独一组,不和 RGB 混在 images/ 下)
|
| 293 |
+
#
|
| 294 |
+
# 深度**不在** camera_streams 里 —— 它单独放在 depth/<source>/,
|
| 295 |
+
# 所以上面的 RGB 循环根本看不到它,必须用 lerobot_export 的发现函数。
|
| 296 |
+
#
|
| 297 |
+
# 解码必须走 DepthVideoReader(ffmpeg 解 gray12le):OpenCV 会把
|
| 298 |
+
# 12-bit 深度当普通显示图转成 8-bit,真实毫米值全丢。
|
| 299 |
+
#
|
| 300 |
+
# **0 表示"没测到深度"的无效像素,不是 0 毫米** —— 与采集端语义一致,
|
| 301 |
+
# 原样保留,不要当数值去算均值。
|
| 302 |
+
for source_key, depth_path in _depth_video_paths(session_dir, episode_id):
|
| 303 |
+
if include_video_keys is not None and not _key_matches_any(
|
| 304 |
+
source_key, include_video_keys):
|
| 305 |
+
continue
|
| 306 |
+
reader = None
|
| 307 |
+
try:
|
| 308 |
+
from app.lerobot_v21 import DEPTH_VIDEO_ENCODING, DepthVideoReader
|
| 309 |
+
reader = DepthVideoReader(depth_path)
|
| 310 |
+
dheight, dwidth = reader.height, reader.width
|
| 311 |
+
if dheight <= 0 or dwidth <= 0 or T <= 0:
|
| 312 |
+
continue
|
| 313 |
+
dfps = float(reader.fps or episode_fps)
|
| 314 |
+
ds = obs.create_dataset(
|
| 315 |
+
f"depth/{source_key}", shape=(T, dheight, dwidth),
|
| 316 |
+
dtype="uint16", chunks=(1, dheight, dwidth),
|
| 317 |
+
compression=compression, compression_opts=level)
|
| 318 |
+
ds.attrs["unit"] = "millimetre"
|
| 319 |
+
ds.attrs["invalid_value"] = 0
|
| 320 |
+
ds.attrs["fps"] = dfps
|
| 321 |
+
ds.attrs["source_key"] = source_key
|
| 322 |
+
ds.attrs["source_episode"] = episode_id
|
| 323 |
+
ds.attrs["encoding"] = DEPTH_VIDEO_ENCODING
|
| 324 |
+
i = 0
|
| 325 |
+
while i < T:
|
| 326 |
+
frame = reader.read_mm()
|
| 327 |
+
if frame is None:
|
| 328 |
+
break
|
| 329 |
+
ds[i] = frame
|
| 330 |
+
i += 1
|
| 331 |
+
features[f"observation.depth.{source_key}"] = {
|
| 332 |
+
"dtype": "uint16", "shape": [T, dheight, dwidth],
|
| 333 |
+
"unit": "millimetre",
|
| 334 |
+
"note": "0 = 无效像素(该点未测到深度),不是 0 毫米",
|
| 335 |
+
"video_info": {
|
| 336 |
+
"video.fps": dfps,
|
| 337 |
+
"video.is_depth_map": True,
|
| 338 |
+
"video.height": dheight,
|
| 339 |
+
"video.width": dwidth,
|
| 340 |
+
},
|
| 341 |
+
}
|
| 342 |
+
except Exception as exc:
|
| 343 |
+
print(f"[hdf5_export] depth stream {source_key} "
|
| 344 |
+
f"skipped: {exc}")
|
| 345 |
+
finally:
|
| 346 |
+
if reader is not None:
|
| 347 |
+
try:
|
| 348 |
+
reader.close()
|
| 349 |
+
except Exception:
|
| 350 |
+
pass
|
| 351 |
+
|
| 352 |
# 2) 表列:传感器 + 手部骨骼 + 标注(与 LeRobot 列同构,按帧号对齐)
|
| 353 |
columns: dict[str, dict[int, Any]] = {}
|
| 354 |
str_columns: set[str] = set()
|
|
|
|
| 467 |
annotation_defs, ensure_ascii=False)
|
| 468 |
|
| 469 |
# 6) 标定 → /meta/calibration/<episode>/<name> attrs(JSON 字符串)
|
| 470 |
+
# 每集一份,所以把 episode_id 一起传进去 —— 批次内各集标定可以不同,
|
| 471 |
+
# 不传就只会看到批次级的 calibration/ 目录。
|
| 472 |
+
# 缺失/全零时的默认标定回退统一在 lerobot_export._calibration_docs。
|
| 473 |
cal_group = meta.create_group(f"calibration/{episode_id}")
|
| 474 |
+
cal_docs = _calibration_docs(session_dir, episode_id)
|
| 475 |
for name, value in cal_docs.items():
|
| 476 |
cal_group.attrs[name] = json.dumps(value, ensure_ascii=False)
|
| 477 |
|
|
|
|
| 567 |
img_keys = sorted(n for n in all_names
|
| 568 |
if n.startswith(f"{root}/observation/images/"))
|
| 569 |
if img_keys:
|
| 570 |
+
# 不写"全量" —— 视频可能被工作流的 include_video_keys 过滤过,而且
|
| 571 |
+
# 深度流走 depth_sources、不经 camera_streams,本文件**从来不导**。
|
| 572 |
+
# 声称"全量"会让客户以为拿到的是全部视频。
|
| 573 |
+
rows.append(f"| **原视频** | {len(img_keys)} 路原始帧(逐帧 uint8 RGB)"
|
| 574 |
+
" | ✅ 已导出,清单见上方结构树 |")
|
| 575 |
+
depth_keys = sorted(n for n in all_names
|
| 576 |
+
if n.startswith(f"{root}/observation/depth/"))
|
| 577 |
+
if depth_keys:
|
| 578 |
+
rows.append(f"| **深度图** | {len(depth_keys)} 路,uint16 **毫米**"
|
| 579 |
+
" | ✅ 0 = 无效像素(未测到),不是 0 毫米 |")
|
| 580 |
hand_coordinate = next(
|
| 581 |
(coord for coord in ("world", "3d", "2d")
|
| 582 |
if f"{root}/observation/state/hand_left_{coord}" in all_names),
|
|
|
|
| 628 |
```
|
| 629 |
"""
|
| 630 |
|
| 631 |
+
# 读取示例:只给文件里**确实存在**的路径。原来是左目写死 stereo_left、
|
| 632 |
+
# 手部无数据时仍回落到 hand_right_3d —— 客户照抄都是 KeyError;而且
|
| 633 |
+
# `import h5py` 在 0 列、`with` 却缩进了 4 格,示例本身就不是合法 Python。
|
| 634 |
+
first_img = img_keys[0].split("/")[-1] if img_keys else None
|
| 635 |
+
example = ["import h5py", "", 'with h5py.File("dataset.h5", "r") as h5:']
|
| 636 |
+
if first_img:
|
| 637 |
+
example.append(f" # 读一帧图像({first_img})")
|
| 638 |
+
example.append(f' img = h5["{root}/observation/images/{first_img}"][0]')
|
| 639 |
+
if depth_keys:
|
| 640 |
+
first_depth = depth_keys[0].split("/")[-1]
|
| 641 |
+
example.append(f" # 读一帧深度({first_depth};uint16 毫米,0 = 无效像素)")
|
| 642 |
+
example.append(f' depth = h5["{root}/observation/depth/{first_depth}"][0]')
|
| 643 |
+
if hand_coordinate:
|
| 644 |
+
example.append(" # 读第一帧右手骨骼(字段可能是 world / 3d / 2d)")
|
| 645 |
+
example.append(f' hand3d = h5["{root}/observation/state/'
|
| 646 |
+
f'hand_right_{hand_coordinate}"][0]')
|
| 647 |
+
if f"{root}/observation/annotation" in all_names:
|
| 648 |
+
example.append(" # 读全部标注")
|
| 649 |
+
example.append(f' labels = h5["{root}/observation/annotation"][:]')
|
| 650 |
+
read_example = "\n".join(example)
|
| 651 |
+
|
| 652 |
readme = f"""# {dataset_name} 数据文件说明(HDF5 格式)
|
| 653 |
|
| 654 |
> 本文件由 EgoData 数据采集系统工作流自动生成
|
|
|
|
| 679 |
## 四、如何读取
|
| 680 |
|
| 681 |
```python
|
| 682 |
+
{read_example}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 683 |
```
|
| 684 |
{video_extract}
|
| 685 |
其他工具:HDFView、Panoply、MATLAB(h5read)等支持 HDF5 的软件均可打开。
|
processor/app/i18n.py
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""服务端 i18n —— 让首屏 HTML 直接是正确语言,消掉"先英文后中文"的闪烁。
|
| 2 |
+
|
| 3 |
+
**问题**:语言原本只存在浏览器的 ``localStorage`` 里,服务端不知道。所以渲染出来
|
| 4 |
+
的 ``data-i18n`` 元素只能填**默认语言(英文)**,再由 ``DOMContentLoaded`` 里的 JS
|
| 5 |
+
改成中文 —— 改的时机在首屏之后,于是每次整页导航都闪一下。
|
| 6 |
+
|
| 7 |
+
**做法**:语言同时写进 cookie,服务端渲染时就用它取词。首屏直接是对的语言,
|
| 8 |
+
不需要任何"先渲染再纠正"。
|
| 9 |
+
|
| 10 |
+
**词典的真源**是 ``web/static/js/i18n-data.js`` —— 一个合法 JSON 载荷外面包一行
|
| 11 |
+
``window.__I18N__ = …``:
|
| 12 |
+
|
| 13 |
+
* 浏览器把它当**普通脚本**同步加载(带 ``?v=`` 缓存,不会每页重下)
|
| 14 |
+
* 这里剥掉包装后 ``json.loads``,拿到的就是同一份数据
|
| 15 |
+
|
| 16 |
+
两边读同一份文件,所以**不会分叉**。改词条只改那一个文件。
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from __future__ import annotations
|
| 20 |
+
|
| 21 |
+
import json
|
| 22 |
+
from contextvars import ContextVar
|
| 23 |
+
from pathlib import Path
|
| 24 |
+
|
| 25 |
+
from starlette.middleware.base import BaseHTTPMiddleware
|
| 26 |
+
|
| 27 |
+
# cookie 名与前端 localStorage 的键保持一致,便于两边对照排查。
|
| 28 |
+
LANG_COOKIE = "lang"
|
| 29 |
+
LANGUAGES = ("zh", "en")
|
| 30 |
+
# 没选过语言时用哪个。刻意保持 **en** —— 与前端原来的行为一致(切语言前的默认),
|
| 31 |
+
# 不借这次改动偷偷改变默认观感。要改成 zh 的话两边一起改(i18n.js 的 currentLang)。
|
| 32 |
+
DEFAULT_LANG = "en"
|
| 33 |
+
|
| 34 |
+
_DICT_PATH = (Path(__file__).resolve().parent.parent
|
| 35 |
+
/ "web" / "static" / "js" / "i18n-data.js")
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
def _load() -> dict[str, dict[str, str]]:
|
| 39 |
+
"""读 ``i18n-data.js`` 并把 JSON 载荷解析出来。
|
| 40 |
+
|
| 41 |
+
解析失败**不抛异常**:页面照常起得来,只是文案退回 key 本身 —— 比整个服务
|
| 42 |
+
因为一个词典文件挂掉好。真要查的话看 ``data`` 是不是空的。
|
| 43 |
+
"""
|
| 44 |
+
try:
|
| 45 |
+
raw = _DICT_PATH.read_text(encoding="utf-8")
|
| 46 |
+
payload = raw[raw.index("{"):raw.rindex("}") + 1]
|
| 47 |
+
data = json.loads(payload)
|
| 48 |
+
except (OSError, ValueError, IndexError):
|
| 49 |
+
return {}
|
| 50 |
+
return data if isinstance(data, dict) else {}
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
DATA: dict[str, dict[str, str]] = _load()
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def normalize(lang: str | None) -> str:
|
| 57 |
+
"""把任意输入收敛成一个可用语言码。认不出来就回默认。"""
|
| 58 |
+
value = str(lang or "").strip().lower()
|
| 59 |
+
return value if value in DATA else DEFAULT_LANG
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
_current: ContextVar[str] = ContextVar("egodata_lang", default=DEFAULT_LANG)
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def get_current_lang() -> str:
|
| 66 |
+
return _current.get()
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def t(key: str, lang: str | None = None) -> str:
|
| 70 |
+
"""取词。**取不到就返回 key 本身** —— 页面上会显示出一个明显的
|
| 71 |
+
``nav_foo`` 而不是空白,一眼看得出是漏翻,比静默空着好排查。"""
|
| 72 |
+
use = normalize(lang) if lang is not None else _current.get()
|
| 73 |
+
table = DATA.get(use) or DATA.get(DEFAULT_LANG) or {}
|
| 74 |
+
return table.get(key) or key
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
class LanguageMiddleware(BaseHTTPMiddleware):
|
| 78 |
+
"""从 cookie 读语言,放进 ContextVar —— Jinja 里注册的 ``t`` 直接用。
|
| 79 |
+
|
| 80 |
+
用 ContextVar 而不是往每个 ``TemplateResponse`` 的 context 里塞:后者要改
|
| 81 |
+
20 多处调用点,而 ``t`` 是**每个模板都要用**的东西,漏一个就是一片英文。
|
| 82 |
+
"""
|
| 83 |
+
|
| 84 |
+
async def dispatch(self, request, call_next):
|
| 85 |
+
token = _current.set(normalize(request.cookies.get(LANG_COOKIE)))
|
| 86 |
+
try:
|
| 87 |
+
return await call_next(request)
|
| 88 |
+
finally:
|
| 89 |
+
_current.reset(token)
|
processor/app/lerobot_export.py
CHANGED
|
@@ -1806,6 +1806,82 @@ def _depth_assets(session_dir: Path, frame_count: int,
|
|
| 1806 |
return assets
|
| 1807 |
|
| 1808 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1809 |
def _device_metadata(session_dir: Path) -> list[dict]:
|
| 1810 |
"""Return physical-device declarations for export-side pairing metadata."""
|
| 1811 |
for name in ("metadata.json", "meta/info.json"):
|
|
|
|
| 1806 |
return assets
|
| 1807 |
|
| 1808 |
|
| 1809 |
+
# 内置默认双目标定 —— 批次标定缺失或整份全零时的兜底。键名取自该文件的
|
| 1810 |
+
# ``name`` 字段(head_stereo),与会话里真有一份 head_stereo.json 时算出的键一致。
|
| 1811 |
+
_DEFAULT_CALIBRATION_PATH = (Path(__file__).resolve().parent
|
| 1812 |
+
/ "processing" / "modules"
|
| 1813 |
+
/ "calibration_default_head_stereo.json")
|
| 1814 |
+
|
| 1815 |
+
|
| 1816 |
+
def _calibration_is_blank(doc: Any) -> bool:
|
| 1817 |
+
"""整份标定里找不出任何非零数值 —— 视为未生效的空白标定。
|
| 1818 |
+
|
| 1819 |
+
只递归看数值叶子(标定本质上就是嵌套的数值结构);字符串等非数值字段
|
| 1820 |
+
不构成"有效标定"的证据。所以 ``{"depth_scale": 0.001}`` 不算空白,
|
| 1821 |
+
而全零的 intrinsic/rotation/translation 算。
|
| 1822 |
+
"""
|
| 1823 |
+
if isinstance(doc, bool):
|
| 1824 |
+
return True # bool 是 int 子类,必须先挡掉
|
| 1825 |
+
if isinstance(doc, (int, float)):
|
| 1826 |
+
return doc == 0
|
| 1827 |
+
if isinstance(doc, dict):
|
| 1828 |
+
return all(_calibration_is_blank(v) for v in doc.values())
|
| 1829 |
+
if isinstance(doc, (list, tuple)):
|
| 1830 |
+
return all(_calibration_is_blank(v) for v in doc)
|
| 1831 |
+
return True
|
| 1832 |
+
|
| 1833 |
+
|
| 1834 |
+
def _default_calibration_docs() -> dict[str, Any]:
|
| 1835 |
+
"""读内置默认标定;文件缺失/损坏时返回 {} (调用方自行回落到空)。"""
|
| 1836 |
+
try:
|
| 1837 |
+
doc = json.loads(_DEFAULT_CALIBRATION_PATH.read_text(encoding="utf-8"))
|
| 1838 |
+
except (OSError, ValueError):
|
| 1839 |
+
return {}
|
| 1840 |
+
if not isinstance(doc, dict):
|
| 1841 |
+
return {}
|
| 1842 |
+
return {str(doc.get("name") or _DEFAULT_CALIBRATION_PATH.stem): doc}
|
| 1843 |
+
|
| 1844 |
+
|
| 1845 |
+
def _calibration_docs(session_dir: str | Path,
|
| 1846 |
+
episode_id: str | None = None) -> dict[str, Any]:
|
| 1847 |
+
"""会话标定 → ``{名称: 标定内容}``,供导出侧写进数据集 meta。
|
| 1848 |
+
|
| 1849 |
+
目录口径与其余读取点一致(media_groups / depth_hand_3d / lerobot_v21 /
|
| 1850 |
+
本文件的深度标定段):先看 ``calibration/``;给了 ``episode_id`` 时优先
|
| 1851 |
+
``meta/calibration/<episode_id>/``,再回退 ``meta/calibration/``。键名优先
|
| 1852 |
+
取 JSON 里的 ``name`` 字段,否则用文件名 stem。
|
| 1853 |
+
|
| 1854 |
+
**批次标定缺失、或整份全零时回退内置默认标定**
|
| 1855 |
+
(``processing/modules/calibration_default_head_stereo.json``)。不这么做的话,
|
| 1856 |
+
导出的 meta 会声明一个从未生效的标定,下游照它复现 3D 全是错的。
|
| 1857 |
+
|
| 1858 |
+
只有**每一份**都空白才回退 —— 只要有一份像样的就用会话里的。
|
| 1859 |
+
"""
|
| 1860 |
+
session_dir = Path(session_dir)
|
| 1861 |
+
calib_dir = session_dir / "calibration"
|
| 1862 |
+
if episode_id:
|
| 1863 |
+
namespaced = session_dir / "meta" / "calibration" / str(episode_id)
|
| 1864 |
+
if namespaced.is_dir():
|
| 1865 |
+
calib_dir = namespaced
|
| 1866 |
+
if not calib_dir.is_dir():
|
| 1867 |
+
calib_dir = session_dir / "meta" / "calibration"
|
| 1868 |
+
|
| 1869 |
+
docs: dict[str, Any] = {}
|
| 1870 |
+
if calib_dir.is_dir():
|
| 1871 |
+
for path in sorted(calib_dir.glob("*.json")):
|
| 1872 |
+
try:
|
| 1873 |
+
value = json.loads(path.read_text(encoding="utf-8"))
|
| 1874 |
+
except (OSError, ValueError, TypeError):
|
| 1875 |
+
continue
|
| 1876 |
+
if not isinstance(value, dict):
|
| 1877 |
+
continue
|
| 1878 |
+
docs[str(value.get("name") or path.stem)] = value
|
| 1879 |
+
|
| 1880 |
+
if docs and not all(_calibration_is_blank(v) for v in docs.values()):
|
| 1881 |
+
return docs
|
| 1882 |
+
return _default_calibration_docs() or docs
|
| 1883 |
+
|
| 1884 |
+
|
| 1885 |
def _device_metadata(session_dir: Path) -> list[dict]:
|
| 1886 |
"""Return physical-device declarations for export-side pairing metadata."""
|
| 1887 |
for name in ("metadata.json", "meta/info.json"):
|
processor/app/localstore.py
CHANGED
|
@@ -704,6 +704,36 @@ def write_episode_state(episode_id: str, state: dict) -> None:
|
|
| 704 |
invalidate_session_cache()
|
| 705 |
|
| 706 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 707 |
def set_episode_status(episode_id: str, status: str) -> dict:
|
| 708 |
with _lock:
|
| 709 |
state = read_episode_state(episode_id)
|
|
|
|
| 704 |
invalidate_session_cache()
|
| 705 |
|
| 706 |
|
| 707 |
+
def mutate_episode_state(
|
| 708 |
+
episode_id: str,
|
| 709 |
+
mutator: Callable[[dict], _MutationResult],
|
| 710 |
+
) -> _MutationResult | None:
|
| 711 |
+
"""Atomically read, mutate, and persist one episode's state.
|
| 712 |
+
|
| 713 |
+
``read_episode_state`` / ``write_episode_state`` each take ``_lock`` on
|
| 714 |
+
their own, so a read-modify-write *across* them is not atomic. Two writers
|
| 715 |
+
that overlap lose one side's fields — and when the lost field is ``status``,
|
| 716 |
+
a batch already pushed back to ``to_review`` can be silently restored to
|
| 717 |
+
``reviewed``.
|
| 718 |
+
|
| 719 |
+
That window is not theoretical: the post-run quality tasks (the video gate
|
| 720 |
+
and data quality) both decode the same episode's video and therefore finish
|
| 721 |
+
within milliseconds of each other, which is exactly when they overlap.
|
| 722 |
+
|
| 723 |
+
Mutate the ``state`` dict in place inside ``mutator``; the return value is
|
| 724 |
+
passed straight through. Returns ``None`` (and writes nothing) when the
|
| 725 |
+
episode has no state — callers must not invent one.
|
| 726 |
+
"""
|
| 727 |
+
key = str(episode_id)
|
| 728 |
+
with _lock:
|
| 729 |
+
state = read_episode_state(key)
|
| 730 |
+
if not state:
|
| 731 |
+
return None
|
| 732 |
+
result = mutator(state)
|
| 733 |
+
write_episode_state(key, state)
|
| 734 |
+
return result
|
| 735 |
+
|
| 736 |
+
|
| 737 |
def set_episode_status(episode_id: str, status: str) -> dict:
|
| 738 |
with _lock:
|
| 739 |
state = read_episode_state(episode_id)
|
processor/app/main.py
CHANGED
|
@@ -12,8 +12,10 @@ from starlette.middleware.gzip import GZipMiddleware
|
|
| 12 |
|
| 13 |
from app.config import settings
|
| 14 |
from app.routes import ingestion, video, export, pages, session, annotations, dashboard, devices, auth
|
| 15 |
-
from app.
|
|
|
|
| 16 |
from app.api import users
|
|
|
|
| 17 |
from app.middleware import AuthMiddleware
|
| 18 |
from app.paths import STATIC_DIR
|
| 19 |
from app.version import __version__
|
|
@@ -75,7 +77,10 @@ app = FastAPI(
|
|
| 75 |
app.add_middleware(GZipMiddleware, minimum_size=500)
|
| 76 |
|
| 77 |
# Auth middleware — protect routes (after gzip, before everything else)
|
|
|
|
|
|
|
| 78 |
app.add_middleware(AuthMiddleware)
|
|
|
|
| 79 |
|
| 80 |
# 媒体/压缩文件下载不走 gzip:zip/mp4/parquet/hdf5 本身已是压缩格式,
|
| 81 |
# 再压一遍只是烧 CPU(zlib level 9 压缩 180MB ≈ 10s),而且流式压缩会
|
|
@@ -109,6 +114,15 @@ async def static_cache_headers(request, call_next):
|
|
| 109 |
response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
|
| 110 |
elif request.url.query:
|
| 111 |
response.headers["Cache-Control"] = "public, max-age=86400"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
else:
|
| 113 |
response.headers["Cache-Control"] = "public, max-age=3600"
|
| 114 |
return response
|
|
@@ -139,4 +153,6 @@ app.include_router(workflows.router)
|
|
| 139 |
app.include_router(worker.router)
|
| 140 |
app.include_router(projects.router)
|
| 141 |
app.include_router(exceptions.router)
|
|
|
|
| 142 |
app.include_router(users.router)
|
|
|
|
|
|
| 12 |
|
| 13 |
from app.config import settings
|
| 14 |
from app.routes import ingestion, video, export, pages, session, annotations, dashboard, devices, auth
|
| 15 |
+
from app.routes import training
|
| 16 |
+
from app.api import workflows, worker, projects, exceptions, quality
|
| 17 |
from app.api import users
|
| 18 |
+
from app.i18n import LanguageMiddleware
|
| 19 |
from app.middleware import AuthMiddleware
|
| 20 |
from app.paths import STATIC_DIR
|
| 21 |
from app.version import __version__
|
|
|
|
| 77 |
app.add_middleware(GZipMiddleware, minimum_size=500)
|
| 78 |
|
| 79 |
# Auth middleware — protect routes (after gzip, before everything else)
|
| 80 |
+
# 语言中间件要在 Auth 之外(先于它)跑:鉴权失败跳登录页时,那个页面本身也要
|
| 81 |
+
# 按 cookie 里的语言渲染。add_middleware 是**后加的先跑**,所以放在 Auth 之后加。
|
| 82 |
app.add_middleware(AuthMiddleware)
|
| 83 |
+
app.add_middleware(LanguageMiddleware)
|
| 84 |
|
| 85 |
# 媒体/压缩文件下载不走 gzip:zip/mp4/parquet/hdf5 本身已是压缩格式,
|
| 86 |
# 再压一遍只是烧 CPU(zlib level 9 压缩 180MB ≈ 10s),而且流式压缩会
|
|
|
|
| 114 |
response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
|
| 115 |
elif request.url.query:
|
| 116 |
response.headers["Cache-Control"] = "public, max-age=86400"
|
| 117 |
+
elif path.startswith("/static/js/"):
|
| 118 |
+
# ES module 的相对 import 解析出来的 URL **不带查询串** —— 就算入口
|
| 119 |
+
# script 写了 ?v=,`import './api.js'` 也永远是同一个地址。给它任何
|
| 120 |
+
# 正数的 max-age,升级后都会出现「新入口 + 旧子模块」混用,直接抛
|
| 121 |
+
# TypeError。这类文件必须每次回源校验(未变则 304,开销可忽略)。
|
| 122 |
+
#
|
| 123 |
+
# 范围刻意收窄到 /static/js/:带 ?v= 的老脚本走上面那支不受影响;
|
| 124 |
+
# /static/ 顶层的第三方库(plyr.js、iconify-*)也不在范围里。
|
| 125 |
+
response.headers["Cache-Control"] = "no-cache"
|
| 126 |
else:
|
| 127 |
response.headers["Cache-Control"] = "public, max-age=3600"
|
| 128 |
return response
|
|
|
|
| 153 |
app.include_router(worker.router)
|
| 154 |
app.include_router(projects.router)
|
| 155 |
app.include_router(exceptions.router)
|
| 156 |
+
app.include_router(quality.router)
|
| 157 |
app.include_router(users.router)
|
| 158 |
+
app.include_router(training.router)
|
processor/app/processing/black_glove/hand_landmarker.task
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fbc2a30080c3c557093b5ddfc334698132eb341044ccee322ccf8bcf3607cde1
|
| 3 |
+
size 7819105
|
processor/app/processing/cleaning/__init__.py
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""数据清洗 —— 按数据类型检查 episode 质量,异常退回人工审核。
|
| 2 |
+
|
| 3 |
+
对应《以 Episode 为单位的数据清洗方案》(PDF)。核心设计:
|
| 4 |
+
|
| 5 |
+
* **按信道声明需求**,不按"这是 UMI 还是 EGO" —— 检查项自己声明需要哪些信道,
|
| 6 |
+
实际有哪些信道由数据决定,两者匹配上才跑(见 ``checks.applicable_checks``)。
|
| 7 |
+
* **四层结果** —— 数据流 / 时间区间 / episode / 训练用途(见 ``report.build_report``)。
|
| 8 |
+
* **只出结论,不产出数据** —— 原始数据一字不改,也不生成 clean_v1 之类的副本。
|
| 9 |
+
区间信息完整保留在报告里,下游按 ``training_use`` 过滤即可。
|
| 10 |
+
* **可插拔** —— 新增检查项 = 在 ``checks/`` 下放一个 .py(与 processing/modules 同款)。
|
| 11 |
+
|
| 12 |
+
这个包的算法实现与工作流适配器是分开的,与 ``app/processing/black_glove`` 同构:
|
| 13 |
+
|
| 14 |
+
app/processing/cleaning/ ← 本包:契约 + 检查项 + 报告组装
|
| 15 |
+
app/processing/modules/data_cleaning.py ← 工作流节点(artifact 契约 + 端口 + 配置)
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from app.processing.cleaning.contract import (
|
| 19 |
+
DEVICE_MODALITIES, CAMERA_MODALITIES, CHANNELS, MODALITY_CHANNELS,
|
| 20 |
+
PASS, WARN, FAIL, PENDING, ERROR,
|
| 21 |
+
SEVERITY_ORDER, SEVERITY_LABELS,
|
| 22 |
+
Finding, Range, StreamResult,
|
| 23 |
+
channels_for, normalize_device_modality, worst, is_blocking,
|
| 24 |
+
)
|
| 25 |
+
from app.processing.cleaning.report import (
|
| 26 |
+
SCHEMA_VERSION, TRAINING_USE_RULES,
|
| 27 |
+
build_report, build_timeline, rollup_episode, rollup_training_use,
|
| 28 |
+
empty_report, passed_for,
|
| 29 |
+
)
|
| 30 |
+
|
| 31 |
+
__all__ = [
|
| 32 |
+
"DEVICE_MODALITIES", "CAMERA_MODALITIES", "CHANNELS", "MODALITY_CHANNELS",
|
| 33 |
+
"PASS", "WARN", "FAIL", "PENDING", "ERROR",
|
| 34 |
+
"SEVERITY_ORDER", "SEVERITY_LABELS",
|
| 35 |
+
"Finding", "Range", "StreamResult",
|
| 36 |
+
"channels_for", "normalize_device_modality", "worst", "is_blocking",
|
| 37 |
+
"SCHEMA_VERSION", "TRAINING_USE_RULES",
|
| 38 |
+
"build_report", "build_timeline", "rollup_episode", "rollup_training_use",
|
| 39 |
+
"empty_report", "passed_for",
|
| 40 |
+
]
|
processor/app/processing/cleaning/checks/__init__.py
ADDED
|
@@ -0,0 +1,294 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""清洗检查项注册表 —— 一个文件 = 一个检查项,自动发现。
|
| 2 |
+
|
| 3 |
+
与 app/processing/modules/ 同一套思路(见那里的 __init__.py):
|
| 4 |
+
|
| 5 |
+
for _m in pkgutil.iter_modules(__path__):
|
| 6 |
+
importlib.import_module(...)
|
| 7 |
+
|
| 8 |
+
新增检查项只需在 checks/ 下放一个 .py,内部用 @register_check 装饰类定义。
|
| 9 |
+
|
| 10 |
+
检查项**按信道/模态声明需求**,不按"这个 episode 是 UMI 还是 EGO"。
|
| 11 |
+
所以同一份检查项集合能自动适配任何数据组合:
|
| 12 |
+
UMI 夹爪来了 → 只跑需要 slam/force/gripper_state 的那些
|
| 13 |
+
触觉手套来了 → 只跑需要 tactile 的那些
|
| 14 |
+
EGO 来了 → 只跑需要 rgb/keypoints 的那些
|
| 15 |
+
这就是 PDF §2「用采集模板决定跑哪些清洗规则」的落地方式。
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
import importlib
|
| 21 |
+
import pkgutil
|
| 22 |
+
from dataclasses import dataclass, field as dc_field
|
| 23 |
+
from pathlib import Path
|
| 24 |
+
from typing import Any, Callable, Iterable
|
| 25 |
+
|
| 26 |
+
from app.processing.cleaning.contract import (
|
| 27 |
+
Finding, Range, PASS, WARN, FAIL, PENDING, ERROR,
|
| 28 |
+
SEVERITY_LABELS, worst, is_blocking,
|
| 29 |
+
)
|
| 30 |
+
|
| 31 |
+
__all__ = [
|
| 32 |
+
"CheckContext", "StreamEvidence", "CleaningCheck", "register_check",
|
| 33 |
+
"get_check", "all_checks", "all_check_specs", "applicable_checks",
|
| 34 |
+
"checks_for_device_cards",
|
| 35 |
+
"Finding", "Range", "PASS", "WARN", "FAIL", "PENDING", "ERROR",
|
| 36 |
+
"SEVERITY_LABELS", "worst", "is_blocking",
|
| 37 |
+
]
|
| 38 |
+
|
| 39 |
+
_registry: dict[str, "CleaningCheck"] = {}
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
@dataclass(frozen=True)
|
| 43 |
+
class StreamEvidence:
|
| 44 |
+
"""一条数据流的证据 —— 由 evidence.py 采集,检查项**只读不采**。
|
| 45 |
+
|
| 46 |
+
把"读磁盘"和"做判断"分开,是让检查项可纯测的关键:测试直接构造本对象,
|
| 47 |
+
不需要真的 mp4 / parquet / cv2。
|
| 48 |
+
"""
|
| 49 |
+
|
| 50 |
+
key: str # 流标识,如 "umi_rgb" / "slam_trajectory"
|
| 51 |
+
channel: str = "" # rgb | depth | slam | tactile | ...
|
| 52 |
+
path: str = "" # 相对批次目录的路径(仅用于报错定位)
|
| 53 |
+
# video_quality 的报告(黑屏/冻结/丢帧/解码)。非视频流为空 dict。
|
| 54 |
+
# 结构见 app/video_quality.py::_check_stream 的返回。
|
| 55 |
+
video: dict[str, Any] = dc_field(default_factory=dict)
|
| 56 |
+
# parquet 概览
|
| 57 |
+
columns: tuple[str, ...] = ()
|
| 58 |
+
rows: int = 0
|
| 59 |
+
fps: float = 0.0
|
| 60 |
+
# 逐帧数值序列(按帧对齐)。evidence.py 采集时**派生好的**,检查项不自己算。
|
| 61 |
+
# 例:SLAM 流 → {"pose": ((x,y,z,qx,qy,qz,qw), ...), "t": (...)}
|
| 62 |
+
# 夹爪流 → {"gripper_state": ((open, gripped, raw), ...),
|
| 63 |
+
# "force": ((fx,fy,fz), ...)}
|
| 64 |
+
# ★ 派生复用 app/umi_slam_action.py 的函数,保证与 action 口径同源。
|
| 65 |
+
series: dict[str, tuple] = dc_field(default_factory=dict)
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
@dataclass
|
| 69 |
+
class CheckContext:
|
| 70 |
+
"""检查项执行上下文。
|
| 71 |
+
|
| 72 |
+
IO 走注入(``read_columns`` / ``probe_video`` …),这样检查项本体是纯逻辑,
|
| 73 |
+
测试可以喂假数据而不需要真的 mp4 或 cv2。
|
| 74 |
+
"""
|
| 75 |
+
|
| 76 |
+
batch_dir: Path # 批次目录(含 data/ videos/ meta/)
|
| 77 |
+
episode_id: str = ""
|
| 78 |
+
channels: frozenset[str] = frozenset() # 实际存在的信道
|
| 79 |
+
modalities: tuple[str, ...] = () # 实际存在的设备模态
|
| 80 |
+
params: dict[str, Any] = dc_field(default_factory=dict) # 节点配置里的阈值
|
| 81 |
+
duration_sec: float = 0.0
|
| 82 |
+
fps: float = 0.0
|
| 83 |
+
progress: Callable[[float], None] = lambda _value: None
|
| 84 |
+
|
| 85 |
+
# ── 证据包(evidence.py 采集好,检查项只读)────────────────
|
| 86 |
+
streams: tuple["StreamEvidence", ...] = ()
|
| 87 |
+
# parquet 侧的汇总:行数 / 时间戳列等,跨流共用的信息
|
| 88 |
+
data_rows: int = 0
|
| 89 |
+
data_columns: tuple[str, ...] = ()
|
| 90 |
+
|
| 91 |
+
# ── 注入式 IO ────────────────────────────────────────────
|
| 92 |
+
read_columns: Callable[[Path], list[str]] = lambda _p: []
|
| 93 |
+
read_head: Callable[[Path, int], list[dict]] = lambda _p, _n: []
|
| 94 |
+
probe_video: Callable[[Path], dict] = lambda _p: {}
|
| 95 |
+
|
| 96 |
+
# ── 证据查询便捷方法 ──────────────────────────────────────
|
| 97 |
+
def streams_of(self, channel: str) -> tuple["StreamEvidence", ...]:
|
| 98 |
+
"""取某信道的全部流。检查项用它筛选自己要看的流。"""
|
| 99 |
+
return tuple(item for item in self.streams if item.channel == channel)
|
| 100 |
+
|
| 101 |
+
def param(self, name: str, default: Any = None) -> Any:
|
| 102 |
+
"""取阈值配置;节点未配置时回落到检查项声明的默认值。"""
|
| 103 |
+
value = self.params.get(name)
|
| 104 |
+
return default if value is None else value
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
class CleaningCheck:
|
| 108 |
+
"""清洗检查项基类。
|
| 109 |
+
|
| 110 |
+
子类填元数据 + 实现 ``run()``,用 ``@register_check`` 装饰即可被自动发现。
|
| 111 |
+
"""
|
| 112 |
+
|
| 113 |
+
slug: str = "" # "video.frame_drop"
|
| 114 |
+
label: str = ""
|
| 115 |
+
version: str = "1.0" # 改了要 bump —— 进 ruleset_revision
|
| 116 |
+
description: str = ""
|
| 117 |
+
|
| 118 |
+
# 必需的信道/设备模态;全部满足才跑
|
| 119 |
+
requires_channels: tuple[str, ...] = ()
|
| 120 |
+
requires_modality: tuple[str, ...] = ()
|
| 121 |
+
# 需要 ≥2 类设备模态同时存在(PDF 的"跨模态联合检查")
|
| 122 |
+
cross_modal: bool = False
|
| 123 |
+
|
| 124 |
+
# ★ 本项属于哪些【设备卡片】—— 前端按它把检查项分到设置面板的各 tab。
|
| 125 |
+
#
|
| 126 |
+
# 取值来自 contract.DEVICE_MODALITIES,空元组 = 不绑定单个设备
|
| 127 |
+
# (跨设备检查用,它们只在"跨设备" tab 里出现)。
|
| 128 |
+
#
|
| 129 |
+
# 为什么用声明而不是靠目录分:同一个检查项往往多张卡片都要用 ——
|
| 130 |
+
# ``video.frame_drop`` 对 RGB Camera、RGB-D Camera、UMI Gripper 都成立,
|
| 131 |
+
# 按目录分只能放一处。声明式的话它能同时出现在多个 tab 里,
|
| 132 |
+
# 且各自的阈值配置互不影响(见 node.data.config.devices.<卡片>.params)。
|
| 133 |
+
device_cards: tuple[str, ...] = ()
|
| 134 |
+
|
| 135 |
+
default_severity: str = FAIL
|
| 136 |
+
# 阈值默认值 —— 节点配置可覆盖。PDF 明确要求"阈值不要照搬",
|
| 137 |
+
# 所以这里给的是**平台初始值**,必须在真实数据上验证后再固化。
|
| 138 |
+
default_params: dict[str, Any] = dc_field(default_factory=dict)
|
| 139 |
+
|
| 140 |
+
def param(self, ctx: "CheckContext", name: str, fallback: Any = None) -> Any:
|
| 141 |
+
"""取阈值。顺序:节点配置 → 本项声明的 default_params → fallback。
|
| 142 |
+
|
| 143 |
+
检查项一律用这个方法读阈值,不要写 ``ctx.params.get(..., 字面量)`` ——
|
| 144 |
+
那样声明的 default_params 就成了死代码,改它不生效。
|
| 145 |
+
"""
|
| 146 |
+
value = (ctx.params or {}).get(name)
|
| 147 |
+
if value is not None:
|
| 148 |
+
return value
|
| 149 |
+
if name in self.default_params:
|
| 150 |
+
return self.default_params[name]
|
| 151 |
+
return fallback
|
| 152 |
+
|
| 153 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 154 |
+
raise NotImplementedError
|
| 155 |
+
|
| 156 |
+
# ── 便捷构造 ─────────────────────────────────────────────
|
| 157 |
+
def finding(self, ctx: CheckContext, status: str, *,
|
| 158 |
+
stream: str = "", message: str = "",
|
| 159 |
+
metrics: dict | None = None,
|
| 160 |
+
ranges: Iterable[Range] = ()) -> Finding:
|
| 161 |
+
return Finding(
|
| 162 |
+
rule=self.slug, version=self.version, status=status,
|
| 163 |
+
scope="cross_modal" if self.cross_modal else "stream",
|
| 164 |
+
stream=stream, message=message, metrics=dict(metrics or {}),
|
| 165 |
+
ranges=tuple(ranges),
|
| 166 |
+
)
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
def checks_for_device_cards(cards: Iterable[str]) -> dict[str, list[CleaningCheck]]:
|
| 170 |
+
"""按【设备卡片】把检查项分组 —— 前端设置面板的 tab 就按这个渲染。
|
| 171 |
+
|
| 172 |
+
只返回传进来的卡片(= 工作流里实际连接的设备),未连接的不会出现。
|
| 173 |
+
一个检查项可能落在多个卡片下(如 ``video.frame_drop`` 对每种带摄像头的
|
| 174 |
+
设备都成立),各 tab 里的阈值配置互不影响。
|
| 175 |
+
|
| 176 |
+
例::
|
| 177 |
+
|
| 178 |
+
checks_for_device_cards(["rgbd_camera", "glove_sensor"])
|
| 179 |
+
→ {"rgbd_camera": [video.frame_drop, video.black_screen, ...],
|
| 180 |
+
"glove_sensor": [glove.tactile_health, ...]}
|
| 181 |
+
"""
|
| 182 |
+
wanted = [str(card) for card in cards]
|
| 183 |
+
grouped: dict[str, list[CleaningCheck]] = {card: [] for card in wanted}
|
| 184 |
+
for check in all_checks():
|
| 185 |
+
for card in check.device_cards:
|
| 186 |
+
if card in grouped:
|
| 187 |
+
grouped[card].append(check)
|
| 188 |
+
return grouped
|
| 189 |
+
|
| 190 |
+
|
| 191 |
+
def register_check(cls):
|
| 192 |
+
"""按 slug 注册检查项(装饰器)。"""
|
| 193 |
+
if not cls.slug:
|
| 194 |
+
raise ValueError(f"Check {cls.__name__} has no slug")
|
| 195 |
+
if cls.slug in _registry:
|
| 196 |
+
raise ValueError(f"Duplicate check slug: {cls.slug}")
|
| 197 |
+
_registry[cls.slug] = cls()
|
| 198 |
+
return cls
|
| 199 |
+
|
| 200 |
+
|
| 201 |
+
def get_check(slug: str) -> CleaningCheck | None:
|
| 202 |
+
return _registry.get(slug)
|
| 203 |
+
|
| 204 |
+
|
| 205 |
+
def all_checks() -> list[CleaningCheck]:
|
| 206 |
+
"""全部已注册检查项(按 slug 排序,保证 ruleset_revision 稳定)。"""
|
| 207 |
+
return [_registry[key] for key in sorted(_registry)]
|
| 208 |
+
|
| 209 |
+
|
| 210 |
+
def all_check_specs() -> list[dict]:
|
| 211 |
+
"""检查项目录 —— 给 API / UI 用。
|
| 212 |
+
|
| 213 |
+
前端配置面板直接渲染这个:按 ``category`` 分组,每个检查项一行开关,
|
| 214 |
+
下面按 ``default_params`` 动态生成输入框。新增检查项前端零改动。
|
| 215 |
+
"""
|
| 216 |
+
return [
|
| 217 |
+
{
|
| 218 |
+
"slug": check.slug,
|
| 219 |
+
# 分类取自 slug 前缀(video.frame_drop → video),与 checks/ 下的
|
| 220 |
+
# 目录名一致。前端按它分组。
|
| 221 |
+
"category": check.slug.split(".", 1)[0] if "." in check.slug else "other",
|
| 222 |
+
"label": check.label,
|
| 223 |
+
"version": check.version,
|
| 224 |
+
"description": check.description,
|
| 225 |
+
"requires_channels": list(check.requires_channels),
|
| 226 |
+
"requires_modality": list(check.requires_modality),
|
| 227 |
+
"cross_modal": check.cross_modal,
|
| 228 |
+
"device_cards": list(check.device_cards),
|
| 229 |
+
"default_severity": check.default_severity,
|
| 230 |
+
"default_params": dict(check.default_params),
|
| 231 |
+
}
|
| 232 |
+
for check in all_checks()
|
| 233 |
+
]
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
def applicable_checks(channels: Iterable[str],
|
| 237 |
+
modalities: Iterable[str],
|
| 238 |
+
*,
|
| 239 |
+
cross_modal_active: bool = False,
|
| 240 |
+
enabled: Iterable[str] | None = None,
|
| 241 |
+
disabled: Iterable[str] | None = None) -> list[CleaningCheck]:
|
| 242 |
+
"""筛出该跑哪些检查项。纯函数。
|
| 243 |
+
|
| 244 |
+
- ``requires_channels`` / ``requires_modality`` 必须**全部**具备
|
| 245 |
+
- ``cross_modal`` 的额外要求至少两类模态同时存在
|
| 246 |
+
- ``enabled`` / ``disabled`` 是节点配置里的按类型开关
|
| 247 |
+
"""
|
| 248 |
+
have_channels = {str(item) for item in channels}
|
| 249 |
+
have_modalities = {str(item) for item in modalities}
|
| 250 |
+
allow = {str(item) for item in enabled} if enabled is not None else None
|
| 251 |
+
deny = {str(item) for item in disabled or ()}
|
| 252 |
+
|
| 253 |
+
result: list[CleaningCheck] = []
|
| 254 |
+
for check in all_checks():
|
| 255 |
+
if not set(check.requires_channels) <= have_channels:
|
| 256 |
+
continue
|
| 257 |
+
if not set(check.requires_modality) <= have_modalities:
|
| 258 |
+
continue
|
| 259 |
+
# ★ 设备卡片闸门 —— 声明了归属的检查项只在【对应设备】上跑。
|
| 260 |
+
#
|
| 261 |
+
# 光按信道筛不够:`umi.action_present` 只要"有 action 列"就满足,
|
| 262 |
+
# 而手套项目同样有一列(采集端写的全零占位)—— 结果 UMI 的检查跑到了
|
| 263 |
+
# 手套项目上、报出一个对方根本不关心的失败。
|
| 264 |
+
# 空 device_cards = 跨设备检查,不设闸门。
|
| 265 |
+
if check.device_cards and not (set(check.device_cards) & have_modalities):
|
| 266 |
+
continue
|
| 267 |
+
if check.cross_modal and not cross_modal_active:
|
| 268 |
+
continue
|
| 269 |
+
if allow is not None and check.slug not in allow:
|
| 270 |
+
continue
|
| 271 |
+
if check.slug in deny:
|
| 272 |
+
continue
|
| 273 |
+
result.append(check)
|
| 274 |
+
return result
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
# 自动发现检查项。
|
| 278 |
+
#
|
| 279 |
+
# 用 walk_packages 而非 iter_modules —— 前者递归进子包,后者只认平铺的 .py。
|
| 280 |
+
# 检查项按数据类型分类放在子包里,目录即分类,便于定位:
|
| 281 |
+
#
|
| 282 |
+
# checks/video/frame_drop.py 视频
|
| 283 |
+
# checks/umi/slam_continuity.py UMI 夹爪
|
| 284 |
+
# checks/glove/baseline_drift.py 触觉手套
|
| 285 |
+
# checks/ego/hand_side.py EGO 头戴
|
| 286 |
+
# checks/depth/presence.py 深度
|
| 287 |
+
# checks/cross/video_data_sync.py 跨模态
|
| 288 |
+
#
|
| 289 |
+
# 子包不需要写任何注册代码 —— 空 __init__.py 即可,里面的模块会被自动导入。
|
| 290 |
+
# 新增一项检查 = 在对应目录放一个 .py,注册表/节点/前端都不用改。
|
| 291 |
+
for _info in pkgutil.walk_packages(__path__, prefix=f"{__name__}."):
|
| 292 |
+
if _info.name.rsplit(".", 1)[-1].startswith("_"):
|
| 293 |
+
continue
|
| 294 |
+
importlib.import_module(_info.name)
|
processor/app/processing/cleaning/checks/cross/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""跨模态检查 —— 需要 ≥2 类设备模态同时存在。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/depth/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""深度类检查 —— 是否存在/有效占比。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/ego/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""EGO 头戴检查 —— 关键点/左右手/曝光。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/glove/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""触觉手套检查 —— 阵列健康/基线漂移/缺帧。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/glove/array_shape.py
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""触觉阵列形状 —— 每帧必须是一整块 16×16。
|
| 2 |
+
|
| 3 |
+
为什么值得单列一项
|
| 4 |
+
------------------
|
| 5 |
+
看着像废话,但它是**静默截断的哨兵**:采集层曾经因为宽度表里没有 ``tactile``
|
| 6 |
+
而落到 3 的兜底值,256 个感应点被悄悄截成 3 个 —— 有列、有值、能加载,检查项
|
| 7 |
+
只看得到 1% 的阵列,而且不报任何错。同一类问题还会再来(换手套、换固件、改列
|
| 8 |
+
宽),所以这里明确把"整块阵列"写成契约。
|
| 9 |
+
|
| 10 |
+
判据
|
| 11 |
+
----
|
| 12 |
+
每帧长度必须等于 ``expected_size``(默认 256 = 16×16)。空元组(``None`` 帧)
|
| 13 |
+
也算不合格 —— 它意味着这一帧根本没读到。
|
| 14 |
+
|
| 15 |
+
超出 ``max_bad_ratio`` 判 FAIL,否则 WARN:单帧缺失多半是偶发,系统性截断会是
|
| 16 |
+
接近 100% 的比例。
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from __future__ import annotations
|
| 20 |
+
|
| 21 |
+
from app.processing.cleaning.checks import (
|
| 22 |
+
CheckContext, CleaningCheck, register_check,
|
| 23 |
+
Finding, PASS, WARN, FAIL,
|
| 24 |
+
)
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
@register_check
|
| 28 |
+
class GloveArrayShapeCheck(CleaningCheck):
|
| 29 |
+
slug = "glove.array_shape"
|
| 30 |
+
label = "Tactile Array Integrity"
|
| 31 |
+
version = "1.0"
|
| 32 |
+
description = "Every frame must be a complete 16×16 array (catches silent truncation)"
|
| 33 |
+
|
| 34 |
+
requires_channels = ("tactile",)
|
| 35 |
+
device_cards = ("glove_sensor",)
|
| 36 |
+
default_params = {
|
| 37 |
+
"expected_size": 256, # 16×16
|
| 38 |
+
"max_bad_ratio": 0.01, # 不合格帧超过 1% 判 FAIL
|
| 39 |
+
}
|
| 40 |
+
default_severity = WARN
|
| 41 |
+
|
| 42 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 43 |
+
findings: list[Finding] = []
|
| 44 |
+
expected = int(self.param(ctx, "expected_size", 256))
|
| 45 |
+
max_ratio = float(self.param(ctx, "max_bad_ratio", 0.01))
|
| 46 |
+
|
| 47 |
+
for stream in ctx.streams_of("tactile"):
|
| 48 |
+
values = tuple(stream.series.get("tactile") or ())
|
| 49 |
+
total = len(values)
|
| 50 |
+
if not total:
|
| 51 |
+
# 列在但一行都没有 —— 与"列不存在"同义,交给别的检查去报
|
| 52 |
+
continue
|
| 53 |
+
|
| 54 |
+
bad = [len(item) for item in values if len(item) != expected]
|
| 55 |
+
ratio = len(bad) / total
|
| 56 |
+
# 出现过的长度 —— 一眼看出是被截断(如 3)还是别的形状
|
| 57 |
+
observed = sorted({len(item) for item in values})
|
| 58 |
+
|
| 59 |
+
if not bad:
|
| 60 |
+
findings.append(self.finding(
|
| 61 |
+
ctx, PASS, stream=stream.key,
|
| 62 |
+
message=f"{stream.key}: {expected}-wide for the whole episode",
|
| 63 |
+
metrics={"frames": total, "expected_size": expected},
|
| 64 |
+
))
|
| 65 |
+
continue
|
| 66 |
+
|
| 67 |
+
status = FAIL if ratio > max_ratio else WARN
|
| 68 |
+
findings.append(self.finding(
|
| 69 |
+
ctx, status, stream=stream.key,
|
| 70 |
+
message=(f"{stream.key}: {len(bad)}/{total} frames are not "
|
| 71 |
+
f"{expected}-wide (observed lengths {observed[:5]})"),
|
| 72 |
+
metrics={
|
| 73 |
+
"frames": total, "bad_frames": len(bad),
|
| 74 |
+
"bad_ratio": round(ratio, 4),
|
| 75 |
+
"expected_size": expected,
|
| 76 |
+
"observed_sizes": observed[:10],
|
| 77 |
+
"first_bad_frame": next(
|
| 78 |
+
(i for i, item in enumerate(values) if len(item) != expected), None),
|
| 79 |
+
},
|
| 80 |
+
))
|
| 81 |
+
|
| 82 |
+
return findings
|
processor/app/processing/cleaning/checks/glove/device_status.py
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""手套连接状态 —— 录制途中掉线。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
手套掉线时阵列读数会**静默保持最后一个值**,看起来完全正常:有列、有 16×16、
|
| 6 |
+
数值也在合理范围。只有状态列说实话。所以这一项是唯一能看见"设备其实没了"
|
| 7 |
+
的检查。
|
| 8 |
+
|
| 9 |
+
判定口径(分两种,因为状态词表只见过一个值)
|
| 10 |
+
------------------------------------------
|
| 11 |
+
``connected`` 是实测见到的**唯一**取值(S80C / D435 两个项目、全部集、全部帧)。
|
| 12 |
+
其余词表未知,所以:
|
| 13 |
+
|
| 14 |
+
录制中途**状态发生变化** → **FAIL**
|
| 15 |
+
无论变成什么,从"连着"变成别的就是掉线,这个判断不依赖词表。
|
| 16 |
+
|
| 17 |
+
全程恒定但不是 ``connected`` → **WARN**
|
| 18 |
+
可能是掉线,也可能是我们没见过的正常状态词(如 "ok"/"active")。
|
| 19 |
+
词表未证实之前不拦批次 —— 拦错了用户会开始无脑点通过,比不拦更糟。
|
| 20 |
+
|
| 21 |
+
空字符串 → **WARN**(这一帧没读到状态)
|
| 22 |
+
|
| 23 |
+
★ 阈值不放进 ``default_params``:词表是字符串,而前端设置面板把所有参数都渲染
|
| 24 |
+
成数字输入框。硬塞会把面板显示成 NaN。真需要可配时先改面板的类型支持。
|
| 25 |
+
"""
|
| 26 |
+
|
| 27 |
+
from __future__ import annotations
|
| 28 |
+
|
| 29 |
+
from app.processing.cleaning.checks import (
|
| 30 |
+
CheckContext, CleaningCheck, register_check,
|
| 31 |
+
Finding, PASS, WARN, FAIL,
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
# 实测唯一见过的取值。其余一律按"词表未知"处理,不直接判失败。
|
| 35 |
+
_HEALTHY = "connected"
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
@register_check
|
| 39 |
+
class GloveDeviceStatusCheck(CleaningCheck):
|
| 40 |
+
slug = "glove.device_status"
|
| 41 |
+
label = "Glove Connection"
|
| 42 |
+
version = "1.0"
|
| 43 |
+
description = "Device dropped out mid-recording (from status.*_glove)"
|
| 44 |
+
|
| 45 |
+
requires_channels = ("device_status",)
|
| 46 |
+
device_cards = ("glove_sensor",)
|
| 47 |
+
default_params: dict = {}
|
| 48 |
+
default_severity = FAIL
|
| 49 |
+
|
| 50 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 51 |
+
findings: list[Finding] = []
|
| 52 |
+
|
| 53 |
+
for stream in ctx.streams_of("device_status"):
|
| 54 |
+
values = tuple(stream.series.get("device_status") or ())
|
| 55 |
+
# 流名就是那一列的最后一段(left_glove / right_glove),用它报位置
|
| 56 |
+
hand = stream.key
|
| 57 |
+
if not values:
|
| 58 |
+
findings.append(self.finding(
|
| 59 |
+
ctx, WARN, stream=hand,
|
| 60 |
+
message=f"{hand}: no status column data",
|
| 61 |
+
metrics={"frames": 0},
|
| 62 |
+
))
|
| 63 |
+
continue
|
| 64 |
+
|
| 65 |
+
blanks = sum(1 for item in values if not item)
|
| 66 |
+
distinct: list[str] = []
|
| 67 |
+
for item in values:
|
| 68 |
+
if item and item not in distinct:
|
| 69 |
+
distinct.append(item)
|
| 70 |
+
|
| 71 |
+
# ① 中途变过 —— 与词表无关,见到即掉线
|
| 72 |
+
if len(distinct) > 1:
|
| 73 |
+
changes = [i for i in range(1, len(values)) if values[i] != values[i - 1]]
|
| 74 |
+
findings.append(self.finding(
|
| 75 |
+
ctx, FAIL, stream=hand,
|
| 76 |
+
message=(f"{hand}: status changed {len(changes)} times "
|
| 77 |
+
f"during recording: {' -> '.join(distinct)}"),
|
| 78 |
+
metrics={"frames": len(values), "distinct": distinct,
|
| 79 |
+
"first_change_frame": changes[0] if changes else None,
|
| 80 |
+
"blank_frames": blanks},
|
| 81 |
+
))
|
| 82 |
+
continue
|
| 83 |
+
|
| 84 |
+
status = distinct[0] if distinct else ""
|
| 85 |
+
# ② 恒定但词表未知 / ③ 全空
|
| 86 |
+
if not status:
|
| 87 |
+
findings.append(self.finding(
|
| 88 |
+
ctx, WARN, stream=hand,
|
| 89 |
+
message=f"{hand}: status column is entirely blank",
|
| 90 |
+
metrics={"frames": len(values), "blank_frames": blanks},
|
| 91 |
+
))
|
| 92 |
+
elif status != _HEALTHY:
|
| 93 |
+
findings.append(self.finding(
|
| 94 |
+
ctx, WARN, stream=hand,
|
| 95 |
+
message=(f"{hand}: status is constantly {status!r} (only "
|
| 96 |
+
f"{_HEALTHY!r} has ever been observed; vocabulary "
|
| 97 |
+
f"unconfirmed - please verify manually)"),
|
| 98 |
+
metrics={"frames": len(values), "status": status},
|
| 99 |
+
))
|
| 100 |
+
else:
|
| 101 |
+
findings.append(self.finding(
|
| 102 |
+
ctx, PASS, stream=hand,
|
| 103 |
+
message=f"{hand}: {_HEALTHY} for the whole episode",
|
| 104 |
+
metrics={"frames": len(values), "status": status},
|
| 105 |
+
))
|
| 106 |
+
|
| 107 |
+
return findings
|
processor/app/processing/cleaning/checks/glove/frozen_array.py
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""触觉阵列卡死 —— 设备停止更新,录制端却继续写。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
这是最难发现的一类:**录制链路完全正常**。帧数对得上、时间戳连续、阵列是完整
|
| 6 |
+
的 16×16、数值也在合理区间 —— 只是内容是**很久以前的那一帧**,一直没变。
|
| 7 |
+
事后看数据一切正常,只有对比相邻帧才发现它静止了。
|
| 8 |
+
|
| 9 |
+
判定口径(阈值有实测依据)
|
| 10 |
+
--------------------------
|
| 11 |
+
**静止本身不是问题**。实测设备更新率约 7.5Hz,而录制是 30fps,所以同一个值会
|
| 12 |
+
**连续保持 4 帧** —— 这是正常的,不是卡死。
|
| 13 |
+
|
| 14 |
+
S80C 全部集:受压时的保持段长度**只有 4**(`{4}`,无其他取值)
|
| 15 |
+
D435 全部集:没有受压保持段
|
| 16 |
+
|
| 17 |
+
(本文件统计的是"保持帧数"= 一个取值连续占了多少帧。同一现象另一种常见记法是
|
| 18 |
+
"相邻帧相等的对数",那个数是 3 —— 4 帧有 3 个间隔。两个数不矛盾,别混。)
|
| 19 |
+
|
| 20 |
+
所以阈值有充足余量:``warn_frames=40``(约 1.3 秒)已经远超正常值。取 40 而不是
|
| 21 |
+
"比 4 大一点",是给将来换更新率更低的设备留空间 —— 卡死检测宁可慢一点发现,
|
| 22 |
+
也不该把正常的低更新率设备全报一遍。
|
| 23 |
+
|
| 24 |
+
★ 只在**受压时**才判。全零段天然"完全相同"(都是零),把它算进去的话,一集里
|
| 25 |
+
88% 都是全零帧的批次会报出几百帧的假静止(实测踩过:最长静止段 575 帧,全部
|
| 26 |
+
来自全零段)。
|
| 27 |
+
"""
|
| 28 |
+
|
| 29 |
+
from __future__ import annotations
|
| 30 |
+
|
| 31 |
+
from app.processing.cleaning.checks import (
|
| 32 |
+
CheckContext, CleaningCheck, register_check,
|
| 33 |
+
Finding, PASS, WARN, FAIL,
|
| 34 |
+
)
|
| 35 |
+
from app.processing.cleaning.contract import Range
|
| 36 |
+
|
| 37 |
+
# 报告的区间数上限 —— 真卡死时可能几百段,全塞进报告没意义
|
| 38 |
+
_MAX_RANGES = 20
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def _static_runs(values: tuple, min_length: int) -> list[tuple[int, int]]:
|
| 42 |
+
"""受压时完全相同的连续帧段 ``[(起始帧, 长度), ...]``。
|
| 43 |
+
|
| 44 |
+
只统计**非空且有压力**的帧:全零段天然完全相同,算进来会造出大片假区间。
|
| 45 |
+
"""
|
| 46 |
+
runs: list[tuple[int, int]] = []
|
| 47 |
+
total = len(values)
|
| 48 |
+
index = 0
|
| 49 |
+
while index < total:
|
| 50 |
+
# 向后找同一取值的最长延伸
|
| 51 |
+
end = index
|
| 52 |
+
while end + 1 < total and values[end + 1] == values[index]:
|
| 53 |
+
end += 1
|
| 54 |
+
length = end - index + 1
|
| 55 |
+
frame = values[index]
|
| 56 |
+
pressed = bool(frame) and any(value > 0.0 for value in frame)
|
| 57 |
+
if length >= min_length and pressed:
|
| 58 |
+
runs.append((index, length))
|
| 59 |
+
index = end + 1
|
| 60 |
+
return runs
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
@register_check
|
| 64 |
+
class GloveFrozenArrayCheck(CleaningCheck):
|
| 65 |
+
slug = "glove.frozen_array"
|
| 66 |
+
label = "Tactile Array Frozen"
|
| 67 |
+
version = "1.0"
|
| 68 |
+
description = "Array unchanged for a long stretch while pressed (device stopped updating)"
|
| 69 |
+
|
| 70 |
+
requires_channels = ("tactile",)
|
| 71 |
+
device_cards = ("glove_sensor",)
|
| 72 |
+
default_params = {
|
| 73 |
+
"warn_frames": 40, # 约 1.3 秒 @30fps;实测正常值 3 帧
|
| 74 |
+
"fail_frames": 300, # 约 10 秒
|
| 75 |
+
}
|
| 76 |
+
default_severity = WARN
|
| 77 |
+
|
| 78 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 79 |
+
findings: list[Finding] = []
|
| 80 |
+
fps = float(ctx.fps or 0.0) or 30.0
|
| 81 |
+
warn_frames = int(self.param(ctx, "warn_frames", 40))
|
| 82 |
+
fail_frames = int(self.param(ctx, "fail_frames", 300))
|
| 83 |
+
|
| 84 |
+
for stream in ctx.streams_of("tactile"):
|
| 85 |
+
values = tuple(stream.series.get("tactile") or ())
|
| 86 |
+
if not values:
|
| 87 |
+
continue
|
| 88 |
+
|
| 89 |
+
runs = _static_runs(values, warn_frames)
|
| 90 |
+
if not runs:
|
| 91 |
+
findings.append(self.finding(
|
| 92 |
+
ctx, PASS, stream=stream.key,
|
| 93 |
+
message=f"{stream.key}: no abnormal frozen stretch",
|
| 94 |
+
metrics={"frames": len(values), "warn_frames": warn_frames},
|
| 95 |
+
))
|
| 96 |
+
continue
|
| 97 |
+
|
| 98 |
+
longest = max(length for _, length in runs)
|
| 99 |
+
status = FAIL if longest >= fail_frames else WARN
|
| 100 |
+
ranges = tuple(
|
| 101 |
+
Range(
|
| 102 |
+
start_sec=start / fps,
|
| 103 |
+
end_sec=(start + length - 1) / fps,
|
| 104 |
+
code=self.slug,
|
| 105 |
+
severity=FAIL if length >= fail_frames else WARN,
|
| 106 |
+
stream=stream.key,
|
| 107 |
+
channel="tactile",
|
| 108 |
+
start_frame=start,
|
| 109 |
+
end_frame=start + length - 1,
|
| 110 |
+
rule=self.slug,
|
| 111 |
+
message=f"array unchanged for {length} frames",
|
| 112 |
+
evidence={"frozen_frames": length},
|
| 113 |
+
)
|
| 114 |
+
for start, length in runs[:_MAX_RANGES]
|
| 115 |
+
)
|
| 116 |
+
|
| 117 |
+
findings.append(self.finding(
|
| 118 |
+
ctx, status, stream=stream.key,
|
| 119 |
+
message=(f"{stream.key}: {len(runs)} frozen stretches while "
|
| 120 |
+
f"pressed (longest {longest} frames ~= {longest / fps:.1f}s)"),
|
| 121 |
+
metrics={
|
| 122 |
+
"frames": len(values),
|
| 123 |
+
"static_runs": len(runs),
|
| 124 |
+
"longest_run_frames": longest,
|
| 125 |
+
"longest_run_sec": round(longest / fps, 3),
|
| 126 |
+
"warn_frames": warn_frames,
|
| 127 |
+
"fail_frames": fail_frames,
|
| 128 |
+
},
|
| 129 |
+
ranges=ranges,
|
| 130 |
+
))
|
| 131 |
+
|
| 132 |
+
return findings
|
processor/app/processing/cleaning/checks/glove/gate_floor.py
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""触觉阵列取值下限 —— 检测**编码/量纲漂移**。
|
| 2 |
+
|
| 3 |
+
为什么需要它
|
| 4 |
+
------------
|
| 5 |
+
实测这份手套数据有个硬门限:取值**要么是 0,要么 ≥ 500.02**,0 到 500 之间
|
| 6 |
+
一个值都没有(从 0 直接跳到 508.09,没有过渡)。所以
|
| 7 |
+
|
| 8 |
+
``0`` 的含义是"未接触",**不是"力为零"**
|
| 9 |
+
|
| 10 |
+
这个门限是固件行为,也是所有下游解读的前提。它一旦变了,**别的一切看起来都还
|
| 11 |
+
正常**:列还在、还是 16×16、数还在合理区间 —— 但含义已经不同了。典型触发:
|
| 12 |
+
|
| 13 |
+
* 固件改版,直接输出原始 ADC(0–4095)而不是门限后的值
|
| 14 |
+
* 标定系数 ``scale`` 被应用了两次(或该应用而没应用)
|
| 15 |
+
* 换了另一款手套
|
| 16 |
+
|
| 17 |
+
判据
|
| 18 |
+
----
|
| 19 |
+
非零值里低于 ``min_contact_value``(默认 500)的**占比**。
|
| 20 |
+
超出 ``fail_ratio`` 判 FAIL,否则 WARN。实测两个项目全部集都是 0 个越界值,
|
| 21 |
+
所以正常数据有充足余量。
|
| 22 |
+
"""
|
| 23 |
+
|
| 24 |
+
from __future__ import annotations
|
| 25 |
+
|
| 26 |
+
from app.processing.cleaning.checks import (
|
| 27 |
+
CheckContext, CleaningCheck, register_check,
|
| 28 |
+
Finding, PASS, WARN, FAIL,
|
| 29 |
+
)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
@register_check
|
| 33 |
+
class GloveGateFloorCheck(CleaningCheck):
|
| 34 |
+
slug = "glove.gate_floor"
|
| 35 |
+
label = "Contact Threshold"
|
| 36 |
+
version = "1.0"
|
| 37 |
+
description = "Non-zero values must exceed the contact floor (catches silent encoding changes)"
|
| 38 |
+
|
| 39 |
+
requires_channels = ("tactile",)
|
| 40 |
+
device_cards = ("glove_sensor",)
|
| 41 |
+
default_params = {
|
| 42 |
+
# 实测最小非零值 500.02。留一点余量给固件微调,但远低于它就没有意义了
|
| 43 |
+
# —— 那正是"编码变了"的样子。
|
| 44 |
+
"min_contact_value": 500.0,
|
| 45 |
+
"fail_ratio": 0.05, # 非零值里超过 5% 低于门限 → FAIL
|
| 46 |
+
}
|
| 47 |
+
default_severity = WARN
|
| 48 |
+
|
| 49 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 50 |
+
findings: list[Finding] = []
|
| 51 |
+
floor = float(self.param(ctx, "min_contact_value", 500.0))
|
| 52 |
+
fail_ratio = float(self.param(ctx, "fail_ratio", 0.05))
|
| 53 |
+
|
| 54 |
+
for stream in ctx.streams_of("tactile"):
|
| 55 |
+
values = tuple(stream.series.get("tactile") or ())
|
| 56 |
+
flat = [float(v) for item in values if item for v in item]
|
| 57 |
+
nonzero = [v for v in flat if v > 0.0]
|
| 58 |
+
if not nonzero:
|
| 59 |
+
# 整集没有非零值 —— 那是 glove.no_contact 的事,这里不重复报
|
| 60 |
+
continue
|
| 61 |
+
|
| 62 |
+
below = [v for v in nonzero if v < floor]
|
| 63 |
+
ratio = len(below) / len(nonzero)
|
| 64 |
+
|
| 65 |
+
if not below:
|
| 66 |
+
findings.append(self.finding(
|
| 67 |
+
ctx, PASS, stream=stream.key,
|
| 68 |
+
message=f"{stream.key}: all non-zero values above the floor",
|
| 69 |
+
metrics={"nonzero": len(nonzero), "min_value": round(min(nonzero), 3)},
|
| 70 |
+
))
|
| 71 |
+
continue
|
| 72 |
+
|
| 73 |
+
status = FAIL if ratio > fail_ratio else WARN
|
| 74 |
+
findings.append(self.finding(
|
| 75 |
+
ctx, status, stream=stream.key,
|
| 76 |
+
message=(f"{stream.key}: {len(below)}/{len(nonzero)} non-zero values"
|
| 77 |
+
f" are below the contact floor {floor:g} (min {min(below):.3f})"
|
| 78 |
+
f" - the array encoding may have changed"),
|
| 79 |
+
metrics={
|
| 80 |
+
"nonzero": len(nonzero), "below_floor": len(below),
|
| 81 |
+
"below_ratio": round(ratio, 4),
|
| 82 |
+
"min_contact_value": floor,
|
| 83 |
+
"min_value": round(min(nonzero), 3),
|
| 84 |
+
},
|
| 85 |
+
))
|
| 86 |
+
|
| 87 |
+
return findings
|
processor/app/processing/cleaning/checks/glove/no_contact.py
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""整集无接触 —— 手套没戴上、没开机,或阵列整体失效。
|
| 2 |
+
|
| 3 |
+
为什么单列一项
|
| 4 |
+
--------------
|
| 5 |
+
它和"某一根手指没碰"完全不同量级:后者是正常的(实测单集里 30–86% 的感应点
|
| 6 |
+
全程为 0,因为只压到阵列的一部分)。**整只手套一帧都没有值**才是设备层面的问题。
|
| 7 |
+
|
| 8 |
+
判据保守是故意的
|
| 9 |
+
----------------
|
| 10 |
+
实测最低的一集也有 **11.4%** 的帧存在压力,所以"全零"离正常值有极大余量,
|
| 11 |
+
不需要为它编阈值。``min_active_ratio`` 默认 0.0 = 只在**整集恒为 0** 时才报,
|
| 12 |
+
用户想更严可以往上调。
|
| 13 |
+
|
| 14 |
+
只报 WARN:手套没戴上也可能是**操作员故意的**(这一段就是不给触觉),
|
| 15 |
+
没到判失败的程度。
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
from app.processing.cleaning.checks import (
|
| 21 |
+
CheckContext, CleaningCheck, register_check,
|
| 22 |
+
Finding, PASS, WARN,
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@register_check
|
| 27 |
+
class GloveNoContactCheck(CleaningCheck):
|
| 28 |
+
slug = "glove.no_contact"
|
| 29 |
+
label = "No Contact"
|
| 30 |
+
version = "1.0"
|
| 31 |
+
description = "No pressure readings at all in this episode (not worn / not working)"
|
| 32 |
+
|
| 33 |
+
requires_channels = ("tactile",)
|
| 34 |
+
device_cards = ("glove_sensor",)
|
| 35 |
+
default_params = {
|
| 36 |
+
# 0.0 = 只在整集恒为 0 时报。实测最低 11.4%,余量充足。
|
| 37 |
+
"min_active_ratio": 0.0,
|
| 38 |
+
}
|
| 39 |
+
default_severity = WARN
|
| 40 |
+
|
| 41 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 42 |
+
findings: list[Finding] = []
|
| 43 |
+
min_ratio = float(self.param(ctx, "min_active_ratio", 0.0))
|
| 44 |
+
|
| 45 |
+
for stream in ctx.streams_of("tactile"):
|
| 46 |
+
values = tuple(stream.series.get("tactile") or ())
|
| 47 |
+
if not values:
|
| 48 |
+
continue
|
| 49 |
+
|
| 50 |
+
active = sum(1 for frame in values
|
| 51 |
+
if frame and any(value > 0.0 for value in frame))
|
| 52 |
+
ratio = active / len(values)
|
| 53 |
+
|
| 54 |
+
if ratio > min_ratio:
|
| 55 |
+
findings.append(self.finding(
|
| 56 |
+
ctx, PASS, stream=stream.key,
|
| 57 |
+
message=f"{stream.key}: has contact frames",
|
| 58 |
+
metrics={"frames": len(values), "active_frames": active,
|
| 59 |
+
"active_ratio": round(ratio, 4)},
|
| 60 |
+
))
|
| 61 |
+
else:
|
| 62 |
+
findings.append(self.finding(
|
| 63 |
+
ctx, WARN, stream=stream.key,
|
| 64 |
+
message=(f"{stream.key}: no pressure readings in any of the "
|
| 65 |
+
f"{len(values)} frames (not worn, or the array failed)"),
|
| 66 |
+
metrics={"frames": len(values), "active_frames": active,
|
| 67 |
+
"active_ratio": round(ratio, 4)},
|
| 68 |
+
))
|
| 69 |
+
|
| 70 |
+
return findings
|
processor/app/processing/cleaning/checks/umi/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""UMI 夹爪检查 —— SLAM 连续性/帧同步/夹爪量程。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/umi/action_present.py
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""action 有效性 —— 监督信号是否真的存在、是否可用。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
**action 是 ACT 的监督信号**(``L1(预测, action)``)。没有它就算不出 loss。
|
| 6 |
+
而它是**派生**出来的,不是采出来的,所以有四种失效方式:
|
| 7 |
+
|
| 8 |
+
① 工作流没跑 / 跑失败 → action 还是采集端的**全零占位**
|
| 9 |
+
★ 最阴险:有列、有 7 维、能加载,全是 0 —— 不报任何错
|
| 10 |
+
② 整集派生失败(位姿样本不足)→ 列根本不写 → 导出时缺失
|
| 11 |
+
★ 训练直接 KeyError(2026-09-11 踩过)
|
| 12 |
+
③ 部分帧派生失败 → 那些帧的 action 是 null
|
| 13 |
+
④ 派生成功但数值离群 → 归一化统计量被撑大(交给 umi.slam_continuity 解释)
|
| 14 |
+
|
| 15 |
+
严重度
|
| 16 |
+
------
|
| 17 |
+
① ② 判 **FAIL** —— 没有监督信号,这一集对训练毫无价值。
|
| 18 |
+
③ 判 **WARN** —— 部分帧缺失,剩余帧仍可训练。
|
| 19 |
+
|
| 20 |
+
关于"全零"
|
| 21 |
+
----------
|
| 22 |
+
采集端在 UMI 硬件上写的就是全零(硬件不录 action),所以**必须区分**
|
| 23 |
+
"还没派生"和"派生了但动作真的是零"。判据是**整列方差**:
|
| 24 |
+
派生的 action 即使动作很小,帧间也会有变化;全零占位则方差严格为 0。
|
| 25 |
+
"""
|
| 26 |
+
|
| 27 |
+
from __future__ import annotations
|
| 28 |
+
|
| 29 |
+
from app.processing.cleaning.checks import (
|
| 30 |
+
CheckContext, CleaningCheck, register_check,
|
| 31 |
+
Finding, PASS, WARN, FAIL,
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
@register_check
|
| 36 |
+
class UmiActionPresentCheck(CleaningCheck):
|
| 37 |
+
slug = "umi.action_present"
|
| 38 |
+
label = "Action Validity"
|
| 39 |
+
version = "1.0"
|
| 40 |
+
description = "Missing supervision signal, all-zero placeholder, or frames never derived"
|
| 41 |
+
|
| 42 |
+
requires_channels = ("action",)
|
| 43 |
+
|
| 44 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 45 |
+
|
| 46 |
+
device_cards = ("gripper_device",)
|
| 47 |
+
|
| 48 |
+
default_params = {
|
| 49 |
+
"min_coverage": 0.95, # 有 action 的帧占比下限
|
| 50 |
+
"zero_epsilon": 1e-9, # 判定"全零"的容差
|
| 51 |
+
}
|
| 52 |
+
default_severity = FAIL
|
| 53 |
+
|
| 54 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 55 |
+
findings: list[Finding] = []
|
| 56 |
+
min_coverage = float(self.param(ctx, "min_coverage", 0.95))
|
| 57 |
+
epsilon = float(self.param(ctx, "zero_epsilon", 1e-9))
|
| 58 |
+
|
| 59 |
+
for stream in ctx.streams_of("action"):
|
| 60 |
+
rows = int(stream.rows or 0)
|
| 61 |
+
if not rows:
|
| 62 |
+
# 列存在但一行都没有 —— 与"列不存在"同义
|
| 63 |
+
findings.append(self.finding(
|
| 64 |
+
ctx, FAIL, stream=stream.key,
|
| 65 |
+
message="the action column has no data at all (derivation failed for the whole episode)",
|
| 66 |
+
metrics={"rows": 0},
|
| 67 |
+
))
|
| 68 |
+
continue
|
| 69 |
+
|
| 70 |
+
values = stream.series.get("action") or ()
|
| 71 |
+
usable = [item for item in values if item]
|
| 72 |
+
coverage = len(usable) / rows if rows else 0.0
|
| 73 |
+
|
| 74 |
+
# 整列方差 —— 区分"全零占位"和"动作真的很小"
|
| 75 |
+
flat: list[float] = []
|
| 76 |
+
for item in usable[:5000]: # 抽样即可,全零占位必然处处为零
|
| 77 |
+
try:
|
| 78 |
+
flat.extend(float(value) for value in item)
|
| 79 |
+
except (TypeError, ValueError):
|
| 80 |
+
continue
|
| 81 |
+
spread = (max(flat) - min(flat)) if flat else 0.0
|
| 82 |
+
all_zero = spread <= epsilon
|
| 83 |
+
|
| 84 |
+
metrics = {
|
| 85 |
+
"rows": rows,
|
| 86 |
+
"usable": len(usable),
|
| 87 |
+
"coverage": round(coverage, 4),
|
| 88 |
+
"value_spread": spread,
|
| 89 |
+
}
|
| 90 |
+
|
| 91 |
+
if all_zero and usable:
|
| 92 |
+
findings.append(self.finding(
|
| 93 |
+
ctx, FAIL, stream=stream.key,
|
| 94 |
+
message="action is all zeros - likely still the recorder placeholder (workflow never ran, or derivation failed)",
|
| 95 |
+
metrics=metrics,
|
| 96 |
+
))
|
| 97 |
+
continue
|
| 98 |
+
|
| 99 |
+
if not usable:
|
| 100 |
+
findings.append(self.finding(
|
| 101 |
+
ctx, FAIL, stream=stream.key,
|
| 102 |
+
message="the action column exists but every value is empty (derivation produced no frames)",
|
| 103 |
+
metrics=metrics,
|
| 104 |
+
))
|
| 105 |
+
continue
|
| 106 |
+
|
| 107 |
+
if coverage < min_coverage:
|
| 108 |
+
findings.append(self.finding(
|
| 109 |
+
ctx, WARN, stream=stream.key,
|
| 110 |
+
message=f"only {len(usable)}/{rows} frames have action"
|
| 111 |
+
f"({coverage:.1%}),其余帧无监督信号",
|
| 112 |
+
metrics=metrics,
|
| 113 |
+
))
|
| 114 |
+
continue
|
| 115 |
+
|
| 116 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 117 |
+
metrics=metrics))
|
| 118 |
+
return findings
|
processor/app/processing/cleaning/checks/umi/av_sync.py
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""视频 ↔ 数据 帧同步 —— 各数据流的帧数与 RGB 视频是否一致。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
UMI 的 action 按帧号对齐:第 i 行动作对应第 i 帧画面。任何一路数据与视频
|
| 6 |
+
帧数不一致,都会让"看这一帧、预测下一步"的监督信号整体错位。
|
| 7 |
+
|
| 8 |
+
与 ``video.frame_drop`` 的分工
|
| 9 |
+
-----------------------------
|
| 10 |
+
``video.frame_drop`` 比的是 **视频帧数 vs parquet 总行数**(整体量)。
|
| 11 |
+
本项比的是 **每一路数据流各自的帧数**(结构性)——
|
| 12 |
+
UMI 特有:Slam / gripper_state / force 是三条独立写入的列,
|
| 13 |
+
采集端丢帧时可能只有其中一路短了,总数却对得上。
|
| 14 |
+
|
| 15 |
+
这也是用户提的"整个的数据有没有根据 RGB 视频进行帧同步"。
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
from app.processing.cleaning.checks import (
|
| 21 |
+
CheckContext, CleaningCheck, register_check,
|
| 22 |
+
Finding, PASS, WARN, FAIL,
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@register_check
|
| 27 |
+
class UmiAvSyncCheck(CleaningCheck):
|
| 28 |
+
slug = "umi.av_sync"
|
| 29 |
+
label = "Video ↔ Data Frame Sync"
|
| 30 |
+
version = "1.0"
|
| 31 |
+
description = "Frame counts differ between data streams"
|
| 32 |
+
|
| 33 |
+
# 需要视频和至少一路数据同时存在才有意义。
|
| 34 |
+
# ★ 不加 cross_modal —— 它比的是【同一个 UMI 设备内部】的视频与各数据列,
|
| 35 |
+
# 不是两张设备卡片之间。requires_channels 已经把"必须有 rgb"表达清楚了,
|
| 36 |
+
# 再加 cross_modal 会把它错误地挤进"跨设备" tab。
|
| 37 |
+
requires_channels = ("rgb", "time")
|
| 38 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 39 |
+
device_cards = ("gripper_device",)
|
| 40 |
+
|
| 41 |
+
default_params = {"tolerance": 2}
|
| 42 |
+
default_severity = FAIL
|
| 43 |
+
|
| 44 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 45 |
+
# 视频侧:取第一路 RGB 的帧数作为基准
|
| 46 |
+
rgb_streams = ctx.streams_of("rgb")
|
| 47 |
+
if not rgb_streams:
|
| 48 |
+
return []
|
| 49 |
+
reference = None
|
| 50 |
+
reference_key = ""
|
| 51 |
+
for stream in rgb_streams:
|
| 52 |
+
frames = int((stream.video or {}).get("frame_count") or stream.rows or 0)
|
| 53 |
+
if frames:
|
| 54 |
+
reference, reference_key = frames, stream.key
|
| 55 |
+
break
|
| 56 |
+
if not reference:
|
| 57 |
+
return []
|
| 58 |
+
|
| 59 |
+
tolerance = int(self.param(ctx, "tolerance", 2))
|
| 60 |
+
drifted: dict[str, int] = {}
|
| 61 |
+
for stream in ctx.streams:
|
| 62 |
+
if stream.channel in ("rgb", "depth") or not stream.rows:
|
| 63 |
+
continue
|
| 64 |
+
if stream.key == reference_key:
|
| 65 |
+
continue
|
| 66 |
+
diff = abs(int(stream.rows) - reference)
|
| 67 |
+
if diff > tolerance:
|
| 68 |
+
drifted[stream.key] = int(stream.rows)
|
| 69 |
+
|
| 70 |
+
metrics = {"reference_stream": reference_key,
|
| 71 |
+
"reference_frames": reference,
|
| 72 |
+
"tolerance": tolerance,
|
| 73 |
+
"drifted": drifted}
|
| 74 |
+
|
| 75 |
+
if not drifted:
|
| 76 |
+
return [self.finding(ctx, PASS, metrics=metrics)]
|
| 77 |
+
|
| 78 |
+
detail = ",".join(f"{key}={rows}" for key, rows in sorted(drifted.items()))
|
| 79 |
+
return [self.finding(
|
| 80 |
+
ctx,
|
| 81 |
+
FAIL,
|
| 82 |
+
message=f"these streams do not match the video frame count ({reference}): {detail}",
|
| 83 |
+
metrics=metrics,
|
| 84 |
+
)]
|
processor/app/processing/cleaning/checks/umi/gripper_range.py
ADDED
|
@@ -0,0 +1,157 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""夹爪量程与力基线 —— 开合度越界、力读数漂移。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
**开合度越界**:``umi_slam_action._gripper_fraction()`` 会把开合度
|
| 6 |
+
``np.clip(0, 1)``。也就是说越界值在派生 action 时被**静默夹掉**了 ——
|
| 7 |
+
标定异常的集看起来"正常",但动作里的夹爪维度是失真的。本项就是在
|
| 8 |
+
clip 之前把它抓出来(原始列 ``observation.gripper_state[0]`` 是 0–100)。
|
| 9 |
+
|
| 10 |
+
**力基线漂移**:夹爪没接触物体时力读数应在零附近。若整集的静息基线缓慢
|
| 11 |
+
上移,说明传感器漂移/温漂,用力做阈值判断的模型会失真。
|
| 12 |
+
|
| 13 |
+
判定口径
|
| 14 |
+
--------
|
| 15 |
+
开合度越界默认判 **WARN**(数据还能用,但要知道标定有问题);
|
| 16 |
+
力漂移默认判 **WARN**(同上)。两者都不拦数据,只报告。
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from __future__ import annotations
|
| 20 |
+
|
| 21 |
+
import math
|
| 22 |
+
|
| 23 |
+
from app.processing.cleaning.checks import (
|
| 24 |
+
CheckContext, CleaningCheck, register_check,
|
| 25 |
+
Finding, Range, PASS, WARN,
|
| 26 |
+
)
|
| 27 |
+
|
| 28 |
+
# 原始列里开合度的量程(采集端契约:[open_pct, gripped, fz_mn])
|
| 29 |
+
OPEN_PCT_MIN = 0.0
|
| 30 |
+
OPEN_PCT_MAX = 100.0
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def _median(values: list[float]) -> float:
|
| 34 |
+
if not values:
|
| 35 |
+
return 0.0
|
| 36 |
+
ordered = sorted(values)
|
| 37 |
+
middle = len(ordered) // 2
|
| 38 |
+
if len(ordered) % 2:
|
| 39 |
+
return ordered[middle]
|
| 40 |
+
return (ordered[middle - 1] + ordered[middle]) / 2.0
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
@register_check
|
| 44 |
+
class UmiGripperRangeCheck(CleaningCheck):
|
| 45 |
+
slug = "umi.gripper_range"
|
| 46 |
+
label = "Gripper Range / Force Baseline"
|
| 47 |
+
version = "1.0"
|
| 48 |
+
description = "Gripper opening out of range, or force baseline drift"
|
| 49 |
+
|
| 50 |
+
requires_channels = ("gripper_state",)
|
| 51 |
+
|
| 52 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 53 |
+
|
| 54 |
+
device_cards = ("gripper_device",)
|
| 55 |
+
|
| 56 |
+
default_params = {
|
| 57 |
+
"max_out_of_range_ratio": 0.01, # 越界帧占比上限
|
| 58 |
+
"max_force_drift": 200.0, # 前后半段基线差(mN)
|
| 59 |
+
}
|
| 60 |
+
default_severity = WARN
|
| 61 |
+
|
| 62 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 63 |
+
findings: list[Finding] = []
|
| 64 |
+
max_ratio = float(self.param(ctx, "max_out_of_range_ratio", 0.01))
|
| 65 |
+
max_drift = float(self.param(ctx, "max_force_drift", 200.0))
|
| 66 |
+
|
| 67 |
+
for stream in ctx.streams_of("gripper_state"):
|
| 68 |
+
samples = stream.series.get("gripper_state") or ()
|
| 69 |
+
if not samples:
|
| 70 |
+
continue
|
| 71 |
+
fps = float(stream.fps or ctx.fps or 30.0)
|
| 72 |
+
|
| 73 |
+
out_of_range: list[Range] = []
|
| 74 |
+
values: list[float] = []
|
| 75 |
+
for index, sample in enumerate(samples):
|
| 76 |
+
if not sample:
|
| 77 |
+
continue
|
| 78 |
+
try:
|
| 79 |
+
pct = float(sample[0])
|
| 80 |
+
except (TypeError, ValueError, IndexError):
|
| 81 |
+
continue
|
| 82 |
+
values.append(pct)
|
| 83 |
+
if OPEN_PCT_MIN <= pct <= OPEN_PCT_MAX:
|
| 84 |
+
continue
|
| 85 |
+
out_of_range.append(Range(
|
| 86 |
+
start_sec=index / fps, end_sec=(index + 1) / fps,
|
| 87 |
+
code=self.slug, severity=WARN,
|
| 88 |
+
stream=stream.key, channel="gripper_state",
|
| 89 |
+
start_frame=index, end_frame=index,
|
| 90 |
+
rule=self.slug,
|
| 91 |
+
message=f"opening {pct:.1f} is outside the 0-100 range"
|
| 92 |
+
f"(派生 action 时会被静默夹掉)",
|
| 93 |
+
evidence={"open_pct": round(pct, 2)},
|
| 94 |
+
))
|
| 95 |
+
|
| 96 |
+
if not values:
|
| 97 |
+
continue
|
| 98 |
+
|
| 99 |
+
ratio = len(out_of_range) / len(values)
|
| 100 |
+
metrics: dict = {
|
| 101 |
+
"samples": len(values),
|
| 102 |
+
"out_of_range": len(out_of_range),
|
| 103 |
+
"out_of_range_ratio": round(ratio, 4),
|
| 104 |
+
"min": round(min(values), 2),
|
| 105 |
+
"max": round(max(values), 2),
|
| 106 |
+
}
|
| 107 |
+
|
| 108 |
+
if ratio > max_ratio:
|
| 109 |
+
findings.append(self.finding(
|
| 110 |
+
ctx, WARN, stream=stream.key,
|
| 111 |
+
message=f"{len(out_of_range)}/{len(values)} frames out of range"
|
| 112 |
+
f"({ratio:.1%}),标定可能异常",
|
| 113 |
+
metrics=metrics,
|
| 114 |
+
ranges=out_of_range[:50],
|
| 115 |
+
))
|
| 116 |
+
else:
|
| 117 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 118 |
+
metrics=metrics))
|
| 119 |
+
|
| 120 |
+
# 力基线漂移 —— 独立的信道,单独一轮
|
| 121 |
+
for stream in ctx.streams_of("force"):
|
| 122 |
+
forces = stream.series.get("force") or ()
|
| 123 |
+
if len(forces) < 20:
|
| 124 |
+
continue
|
| 125 |
+
magnitudes = []
|
| 126 |
+
for sample in forces:
|
| 127 |
+
if not sample:
|
| 128 |
+
continue
|
| 129 |
+
try:
|
| 130 |
+
magnitudes.append(_norm3(sample))
|
| 131 |
+
except (TypeError, ValueError):
|
| 132 |
+
continue
|
| 133 |
+
if len(magnitudes) < 20:
|
| 134 |
+
continue
|
| 135 |
+
|
| 136 |
+
half = len(magnitudes) // 2
|
| 137 |
+
first, second = _median(magnitudes[:half]), _median(magnitudes[half:])
|
| 138 |
+
drift = abs(second - first)
|
| 139 |
+
metrics = {"baseline_first_half": round(first, 2),
|
| 140 |
+
"baseline_second_half": round(second, 2),
|
| 141 |
+
"drift_mN": round(drift, 2)}
|
| 142 |
+
|
| 143 |
+
if drift <= max_drift:
|
| 144 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 145 |
+
metrics=metrics))
|
| 146 |
+
continue
|
| 147 |
+
findings.append(self.finding(
|
| 148 |
+
ctx, WARN, stream=stream.key,
|
| 149 |
+
message=f"force baseline drift {drift:.0f} mN"
|
| 150 |
+
f"(前半 {first:.0f} → 后半 {second:.0f}),疑似传感器漂移",
|
| 151 |
+
metrics=metrics,
|
| 152 |
+
))
|
| 153 |
+
return findings
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
def _norm3(sample) -> float:
|
| 157 |
+
return math.sqrt(sum(float(value) ** 2 for value in sample[:3]))
|
processor/app/processing/cleaning/checks/umi/slam_continuity.py
ADDED
|
@@ -0,0 +1,158 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""SLAM 轨迹连续性 —— 相邻位姿之间出现不可能的速度/角速度跳变。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
UMI 的 action 是从 SLAM 位姿**差分**出来的(见 ``app/umi_slam_action.py``)。
|
| 6 |
+
位姿一旦跳变,差分出来的动作就是一个巨大的假位移;模型会学到"突然瞬移"。
|
| 7 |
+
官方也承认 ORB_SLAM3 是整条流水线"最脆弱的部分"。
|
| 8 |
+
|
| 9 |
+
常见来源:视觉跟踪丢失后重定位、弱纹理/暗光、快速转手腕导致运动模糊。
|
| 10 |
+
|
| 11 |
+
判定分两档(这正是本项的设计,不是"只报告不判定")
|
| 12 |
+
--------------------------------------------------
|
| 13 |
+
方案明确警告:「SLAM 单帧跳了 113.9 毫米可以作为问题线索,但不能马上定成
|
| 14 |
+
全平台红线」。113.9mm 来自本平台实测(`umi_slam_action.py` 的 docstring 记录了
|
| 15 |
+
插值方案会横跨重定位跳变),它是**观测线索**不是阈值来源。
|
| 16 |
+
|
| 17 |
+
所以**单次跳变只判 WARN** —— 一次重定位不该废掉一整集(实测
|
| 18 |
+
UMIGripper_Action_AI_000001 就有一次真实的 6.8 m/s 跳变,属"该警告、不该拦")。
|
| 19 |
+
|
| 20 |
+
但跳变**达到 ``fail_jumps``(默认 10 处)判 FAIL**:那已经不是偶发重定位,而是
|
| 21 |
+
SLAM 全程没跟住,差分出来的动作整段都是假的 —— 这种数据确实不能训,按"数据不对
|
| 22 |
+
就拦回人工审核"处理。
|
| 23 |
+
|
| 24 |
+
★ 本文件的旧注释曾写成"默认判 WARN 而非 FAIL……只报告",与实现不符(实现一直
|
| 25 |
+
是分档的)。以本段为准。阈值本身仍须用人工确认过的好/坏数据校准后才固化。
|
| 26 |
+
"""
|
| 27 |
+
|
| 28 |
+
from __future__ import annotations
|
| 29 |
+
|
| 30 |
+
import math
|
| 31 |
+
|
| 32 |
+
from app.processing.cleaning.checks import (
|
| 33 |
+
CheckContext, CleaningCheck, register_check,
|
| 34 |
+
Finding, Range, PASS, WARN, FAIL,
|
| 35 |
+
)
|
| 36 |
+
from app.processing.cleaning.contract import pose_is_valid
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def _norm3(vector) -> float:
|
| 40 |
+
return math.sqrt(sum(float(value) ** 2 for value in vector[:3]))
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
@register_check
|
| 44 |
+
class UmiSlamContinuityCheck(CleaningCheck):
|
| 45 |
+
slug = "umi.slam_continuity"
|
| 46 |
+
label = "SLAM Trajectory Continuity"
|
| 47 |
+
version = "1.0"
|
| 48 |
+
description = "Impossible linear or angular velocity between adjacent poses"
|
| 49 |
+
|
| 50 |
+
requires_channels = ("slam",)
|
| 51 |
+
|
| 52 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 53 |
+
|
| 54 |
+
device_cards = ("gripper_device",)
|
| 55 |
+
|
| 56 |
+
# 单位:米/秒、弧度/秒。人手快速挥动约 2 m/s、6 rad/s 量级,
|
| 57 |
+
# 这里给的是"明显超出人类可达"的起点值。
|
| 58 |
+
#
|
| 59 |
+
# 判级用两个计数阈值,而不是"速度超过多少倍"—— 后者在默认值下会退化成
|
| 60 |
+
# "任何一次跳变都判失败"(一次跳变就废掉一整集太重了)。实测 episode 1
|
| 61 |
+
# 帧 293 有一次 6.8 m/s 的真实重定位跳变,属"该警告、但不该直接拦"。
|
| 62 |
+
default_params = {
|
| 63 |
+
"max_linear_mps": 5.0,
|
| 64 |
+
"max_angular_rps": 20.0,
|
| 65 |
+
"warn_jumps": 1, # 达到此数即警告
|
| 66 |
+
"fail_jumps": 10, # 达到此数判失败(说明 SLAM 全程不稳)
|
| 67 |
+
}
|
| 68 |
+
default_severity = WARN
|
| 69 |
+
|
| 70 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 71 |
+
findings: list[Finding] = []
|
| 72 |
+
max_linear = float(self.param(ctx, "max_linear_mps", 5.0))
|
| 73 |
+
max_angular = float(self.param(ctx, "max_angular_rps", 20.0))
|
| 74 |
+
fail_jumps = int(self.param(ctx, "fail_jumps", 10))
|
| 75 |
+
|
| 76 |
+
for stream in ctx.streams_of("slam"):
|
| 77 |
+
poses = stream.series.get("pose") or ()
|
| 78 |
+
stamps = stream.series.get("t") or ()
|
| 79 |
+
if len(poses) < 2:
|
| 80 |
+
continue
|
| 81 |
+
fps = float(stream.fps or ctx.fps or 30.0)
|
| 82 |
+
dt_default = 1.0 / fps if fps > 0 else 1.0 / 30.0
|
| 83 |
+
|
| 84 |
+
jumps: list[Range] = []
|
| 85 |
+
peak_linear = 0.0
|
| 86 |
+
peak_angular = 0.0
|
| 87 |
+
|
| 88 |
+
for index in range(1, len(poses)):
|
| 89 |
+
prev, cur = poses[index - 1], poses[index]
|
| 90 |
+
# ★ 必须用 pose_is_valid,不能只判长度。
|
| 91 |
+
#
|
| 92 |
+
# 全零占位位姿长度也是 7,能过 len 检查,但四元数模为 0 ——
|
| 93 |
+
# 两个零四元数点积为 0,acos(0)=π/2 ⇒ 夹角 180° ⇒ 每帧
|
| 94 |
+
# 94.25 rad/s。实测 Test94 的 slam_pose 有 437/599 行是全零,
|
| 95 |
+
# 于是 599 帧报出 465 处"跳变",整集被假判 FAIL 并拦住批次。
|
| 96 |
+
if not pose_is_valid(prev) or not pose_is_valid(cur):
|
| 97 |
+
continue
|
| 98 |
+
try:
|
| 99 |
+
delta = [float(cur[i]) - float(prev[i]) for i in range(3)]
|
| 100 |
+
except (TypeError, ValueError):
|
| 101 |
+
continue
|
| 102 |
+
|
| 103 |
+
# 时间间隔取真实时间戳;缺失时回落到 fps
|
| 104 |
+
dt = dt_default
|
| 105 |
+
if len(stamps) > index and stamps[index] and stamps[index - 1]:
|
| 106 |
+
try:
|
| 107 |
+
span = float(stamps[index]) - float(stamps[index - 1])
|
| 108 |
+
if 0 < span < 1.0: # 超过 1 秒说明中间丢了整段
|
| 109 |
+
dt = span
|
| 110 |
+
except (TypeError, ValueError):
|
| 111 |
+
pass
|
| 112 |
+
|
| 113 |
+
linear = _norm3(delta) / dt
|
| 114 |
+
# 姿态差:四元数点积 → 夹角。|dot| 接近 1 表示几乎没转。
|
| 115 |
+
dot = sum(float(prev[3 + i]) * float(cur[3 + i]) for i in range(4))
|
| 116 |
+
angle = 2.0 * math.acos(max(-1.0, min(1.0, abs(dot))))
|
| 117 |
+
angular = angle / dt
|
| 118 |
+
|
| 119 |
+
peak_linear = max(peak_linear, linear)
|
| 120 |
+
peak_angular = max(peak_angular, angular)
|
| 121 |
+
|
| 122 |
+
if linear <= max_linear and angular <= max_angular:
|
| 123 |
+
continue
|
| 124 |
+
jumps.append(Range(
|
| 125 |
+
start_sec=index / fps, end_sec=(index + 1) / fps,
|
| 126 |
+
code=self.slug, severity=WARN,
|
| 127 |
+
stream=stream.key, channel="slam",
|
| 128 |
+
start_frame=index - 1, end_frame=index,
|
| 129 |
+
rule=self.slug,
|
| 130 |
+
message=f"pose jump at frame {index}: "
|
| 131 |
+
f"{_norm3(delta) * 1000:.1f}mm / {math.degrees(angle):.1f}°",
|
| 132 |
+
evidence={"linear_mps": round(linear, 3),
|
| 133 |
+
"angular_rps": round(angular, 3)},
|
| 134 |
+
))
|
| 135 |
+
|
| 136 |
+
metrics = {
|
| 137 |
+
"frames": len(poses),
|
| 138 |
+
"jumps": len(jumps),
|
| 139 |
+
"peak_linear_mps": round(peak_linear, 3),
|
| 140 |
+
"peak_angular_rps": round(peak_angular, 3),
|
| 141 |
+
}
|
| 142 |
+
if not jumps:
|
| 143 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 144 |
+
metrics=metrics))
|
| 145 |
+
continue
|
| 146 |
+
|
| 147 |
+
findings.append(self.finding(
|
| 148 |
+
ctx,
|
| 149 |
+
FAIL if len(jumps) >= fail_jumps else WARN,
|
| 150 |
+
stream=stream.key,
|
| 151 |
+
message=f"{len(jumps)} pose jumps, peak "
|
| 152 |
+
f"{peak_linear:.1f} m/s / "
|
| 153 |
+
f"{math.degrees(peak_angular):.0f} deg/s "
|
| 154 |
+
f"(verify against the video)",
|
| 155 |
+
metrics=metrics,
|
| 156 |
+
ranges=jumps[:50], # 报告里最多带 50 个,避免 JSON 过大
|
| 157 |
+
))
|
| 158 |
+
return findings
|
processor/app/processing/cleaning/checks/umi/slam_coverage.py
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""SLAM 位姿覆盖率 —— 多少帧的位姿是**插值补出来的**。
|
| 2 |
+
|
| 3 |
+
★ 别把"自己格子为空"当成问题
|
| 4 |
+
--------------------------------
|
| 5 |
+
采集端的位姿流是 20/20/60ms **突发**,而数据帧是 1/30s 均匀网格。某个
|
| 6 |
+
33ms 窗口里一个位姿都没到,这一格的缓冲就是空的 —— **这是设计使然**。
|
| 7 |
+
|
| 8 |
+
实测(52 集训练集):**28.2% 的帧自己格子为空**(p90 32.8%)。
|
| 9 |
+
第一版检查项拿这个当指标,结果 52/52 集全部报警 —— 典型的阈值照搬。
|
| 10 |
+
|
| 11 |
+
真正该关心的是:既没有本地样本、**相邻格子也没有**,只能靠插值硬补的帧。
|
| 12 |
+
``poses_from_trajectory`` 汇总所有格子按时间取最近样本,实测能救回绝大多数
|
| 13 |
+
空帧,只有剩下的才插值。
|
| 14 |
+
|
| 15 |
+
实测(同上,4 集抽样):
|
| 16 |
+
集 0: 插值 2/326 = 0.6%
|
| 17 |
+
集 1: 插值 17/313 = 5.4% ← ★ 明显高于其他集
|
| 18 |
+
集 2: 插值 4/384 = 1.0%
|
| 19 |
+
集 7: 插值 0/308 = 0.0%
|
| 20 |
+
|
| 21 |
+
★ 集 1 正是 ``umi.slam_continuity`` 报出 6.8 m/s 跳变的那一集 ——
|
| 22 |
+
两个独立指标指向同一件事:那一集 SLAM 跟丢过。
|
| 23 |
+
所以这两项要**一起看**:插值率高 + 有跳变 = 基本可以确定是跟踪丢失。
|
| 24 |
+
|
| 25 |
+
判定口径
|
| 26 |
+
--------
|
| 27 |
+
默认 **WARN**:插值帧本身不代表数据不可用(插值的是平滑轨迹),
|
| 28 |
+
它是"这段 SLAM 不稳"的信号。
|
| 29 |
+
"""
|
| 30 |
+
|
| 31 |
+
from __future__ import annotations
|
| 32 |
+
|
| 33 |
+
from app.processing.cleaning.checks import (
|
| 34 |
+
CheckContext, CleaningCheck, register_check,
|
| 35 |
+
Finding, PASS, WARN, FAIL,
|
| 36 |
+
)
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
@register_check
|
| 40 |
+
class UmiSlamCoverageCheck(CleaningCheck):
|
| 41 |
+
slug = "umi.slam_coverage"
|
| 42 |
+
label = "SLAM Pose Coverage"
|
| 43 |
+
version = "2.0"
|
| 44 |
+
description = "Poses only recoverable by interpolation (no real sample in nearby slots)"
|
| 45 |
+
|
| 46 |
+
requires_channels = ("slam",)
|
| 47 |
+
|
| 48 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 49 |
+
|
| 50 |
+
device_cards = ("gripper_device",)
|
| 51 |
+
|
| 52 |
+
# 实测正常集 0~1%,异常集 5.4%。起点值取 2% / 10%:
|
| 53 |
+
# 低于 2% 属正常抖动,超过 10% 说明这一段基本没有真实位姿。
|
| 54 |
+
# ★ 这两个数是拿 52 集训练集跑出来的,不是拍脑袋 —— 但样本只有 52 集,
|
| 55 |
+
# 上生产前应拿更多数据复核。
|
| 56 |
+
default_params = {
|
| 57 |
+
"warn_ratio": 0.02,
|
| 58 |
+
"fail_ratio": 0.10,
|
| 59 |
+
}
|
| 60 |
+
default_severity = WARN
|
| 61 |
+
|
| 62 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 63 |
+
findings: list[Finding] = []
|
| 64 |
+
warn_ratio = float(self.param(ctx, "warn_ratio", 0.02))
|
| 65 |
+
fail_ratio = float(self.param(ctx, "fail_ratio", 0.10))
|
| 66 |
+
|
| 67 |
+
for stream in ctx.streams_of("slam"):
|
| 68 |
+
# evidence 提供两种粒度:
|
| 69 |
+
# "interpolated" —— 逐帧布尔(本检查项用这个,能出区间)
|
| 70 |
+
# "coverage" —— 直接给汇总数字({"frames","direct","interpolated"})
|
| 71 |
+
flags = stream.series.get("interpolated")
|
| 72 |
+
summary = stream.series.get("coverage") or {}
|
| 73 |
+
if flags:
|
| 74 |
+
total = len(flags)
|
| 75 |
+
count = sum(1 for item in flags if item)
|
| 76 |
+
elif summary:
|
| 77 |
+
total = int(summary.get("frames") or 0)
|
| 78 |
+
count = int(summary.get("interpolated") or 0)
|
| 79 |
+
else:
|
| 80 |
+
continue
|
| 81 |
+
if not total:
|
| 82 |
+
continue
|
| 83 |
+
ratio = count / total
|
| 84 |
+
|
| 85 |
+
metrics = {
|
| 86 |
+
"frames": total,
|
| 87 |
+
"interpolated_frames": count,
|
| 88 |
+
"interpolated_ratio": round(ratio, 4),
|
| 89 |
+
}
|
| 90 |
+
|
| 91 |
+
if ratio < warn_ratio:
|
| 92 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 93 |
+
metrics=metrics))
|
| 94 |
+
continue
|
| 95 |
+
findings.append(self.finding(
|
| 96 |
+
ctx,
|
| 97 |
+
FAIL if ratio >= fail_ratio else WARN,
|
| 98 |
+
stream=stream.key,
|
| 99 |
+
message=f"{count}/{total} poses ({ratio:.1%}) are interpolation-"
|
| 100 |
+
f"only - no real sample in the local or neighbouring slots,"
|
| 101 |
+
f" which suggests SLAM lost tracking (review together with"
|
| 102 |
+
f" SLAM Trajectory Continuity)",
|
| 103 |
+
metrics=metrics,
|
| 104 |
+
))
|
| 105 |
+
return findings
|
processor/app/processing/cleaning/checks/video/__init__.py
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""视频类检查 —— 解码/黑屏/冻结/丢帧。
|
| 2 |
+
|
| 3 |
+
本目录下一个 .py = 一个检查项(用 @register_check 注册)。
|
| 4 |
+
新增检查项只需在此放文件,注册表会自动发现(见上级 __init__.py 的 walk_packages)。
|
| 5 |
+
"""
|
processor/app/processing/cleaning/checks/video/black_screen.py
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""黑屏 —— 镜头被挡、补光灯没开、或采集程序写了空帧。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
黑屏段里的视觉信息为零。模型在这些帧上学到的是"画面全黑时该做什么",
|
| 6 |
+
通常是噪声;若黑屏占比高,还会把图像归一化统计量带偏。
|
| 7 |
+
|
| 8 |
+
判定口径
|
| 9 |
+
----------
|
| 10 |
+
检测本身由 ``app/video_quality.py`` 完成(按灰度均值 + 暗像素占比双阈值)。
|
| 11 |
+
本检查项只负责把它的区间翻成统一的 Finding/Range 并套用节点配置的阈值 ——
|
| 12 |
+
**不重复实现检测算法**,避免两处口径漂移。
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
from app.processing.cleaning.contract import VIDEO_CAPABLE_MODALITIES
|
| 18 |
+
from app.processing.cleaning.checks import (
|
| 19 |
+
CheckContext, CleaningCheck, register_check,
|
| 20 |
+
Finding, Range, PASS, WARN, FAIL,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
@register_check
|
| 25 |
+
class VideoBlackScreenCheck(CleaningCheck):
|
| 26 |
+
slug = "video.black_screen"
|
| 27 |
+
label = "Black Screen"
|
| 28 |
+
version = "1.0"
|
| 29 |
+
description = "Frame is fully or nearly black"
|
| 30 |
+
|
| 31 |
+
requires_channels = ("rgb",)
|
| 32 |
+
|
| 33 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 34 |
+
|
| 35 |
+
device_cards = VIDEO_CAPABLE_MODALITIES
|
| 36 |
+
|
| 37 |
+
default_params = {"min_sec": 0.5, "fail_ratio": 0.05}
|
| 38 |
+
default_severity = WARN
|
| 39 |
+
|
| 40 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 41 |
+
findings: list[Finding] = []
|
| 42 |
+
for stream in ctx.streams_of("rgb"):
|
| 43 |
+
report = stream.video or {}
|
| 44 |
+
ranges = report.get("black_ranges") or []
|
| 45 |
+
if not ranges:
|
| 46 |
+
continue
|
| 47 |
+
|
| 48 |
+
fps = float(report.get("fps") or stream.fps or ctx.fps or 30.0)
|
| 49 |
+
min_sec = float(self.param(ctx, "min_sec", 0.5))
|
| 50 |
+
converted: list[Range] = []
|
| 51 |
+
total_sec = 0.0
|
| 52 |
+
for item in ranges:
|
| 53 |
+
try:
|
| 54 |
+
start, end = float(item[0]), float(item[1])
|
| 55 |
+
except (TypeError, ValueError, IndexError):
|
| 56 |
+
continue
|
| 57 |
+
if end - start < min_sec:
|
| 58 |
+
continue
|
| 59 |
+
total_sec += end - start
|
| 60 |
+
converted.append(Range(
|
| 61 |
+
start_sec=start, end_sec=end,
|
| 62 |
+
code=self.slug, severity=WARN,
|
| 63 |
+
stream=stream.key, channel="rgb",
|
| 64 |
+
start_frame=int(start * fps), end_frame=int(end * fps),
|
| 65 |
+
rule=self.slug,
|
| 66 |
+
message=f"black screen for {end - start:.1f}s",
|
| 67 |
+
))
|
| 68 |
+
|
| 69 |
+
if not converted:
|
| 70 |
+
continue
|
| 71 |
+
|
| 72 |
+
duration = float(ctx.duration_sec or 0.0)
|
| 73 |
+
ratio = (total_sec / duration) if duration > 0 else 0.0
|
| 74 |
+
fail_ratio = float(self.param(ctx, "fail_ratio", 0.05))
|
| 75 |
+
status = FAIL if ratio >= fail_ratio else WARN
|
| 76 |
+
|
| 77 |
+
findings.append(self.finding(
|
| 78 |
+
ctx, status, stream=stream.key,
|
| 79 |
+
message=f"{len(converted)} black-screen stretches, {total_sec:.1f}s total"
|
| 80 |
+
+ (f"(占 {ratio:.1%})" if duration else ""),
|
| 81 |
+
metrics={"segments": len(converted),
|
| 82 |
+
"total_sec": round(total_sec, 3),
|
| 83 |
+
"ratio": round(ratio, 4)},
|
| 84 |
+
ranges=converted,
|
| 85 |
+
))
|
| 86 |
+
return findings
|
processor/app/processing/cleaning/checks/video/decode_error.py
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""解码失败 / 文件打不开 —— 最硬的一类问题。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
解码失败意味着这段视频在任何下游都无法使用:训练读取会抛异常、浏览器预览
|
| 6 |
+
会黑屏、导出会失败。而且它常常是文件损坏的前兆 —— 与其等到训练时才发现,
|
| 7 |
+
不如在入库后就标出来。
|
| 8 |
+
|
| 9 |
+
严重度
|
| 10 |
+
------
|
| 11 |
+
本项默认 **FAIL**(其余视频检查默认 WARN):能解码是"数据可用"的最低门槛,
|
| 12 |
+
不存在"阈值调松就能接受"的余地。
|
| 13 |
+
|
| 14 |
+
opencv 不可用是**环境问题不是数据问题** —— 判 ERROR(处理失败)而不是 FAIL,
|
| 15 |
+
避免把环境缺依赖记成数据质量事故。
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
from app.processing.cleaning.contract import VIDEO_CAPABLE_MODALITIES
|
| 21 |
+
from app.processing.cleaning.checks import (
|
| 22 |
+
CheckContext, CleaningCheck, register_check,
|
| 23 |
+
Finding, Range, PASS, FAIL, ERROR,
|
| 24 |
+
)
|
| 25 |
+
|
| 26 |
+
# 这些 reason 说明"没检查成",不是"数据坏了"
|
| 27 |
+
_ENV_REASONS = {"opencv_unavailable"}
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
@register_check
|
| 31 |
+
class VideoDecodeErrorCheck(CleaningCheck):
|
| 32 |
+
slug = "video.decode_error"
|
| 33 |
+
label = "Decode Errors"
|
| 34 |
+
version = "1.0"
|
| 35 |
+
description = "Video won't open, or contains undecodable frames"
|
| 36 |
+
|
| 37 |
+
requires_channels = ("rgb",)
|
| 38 |
+
|
| 39 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 40 |
+
|
| 41 |
+
device_cards = VIDEO_CAPABLE_MODALITIES
|
| 42 |
+
|
| 43 |
+
default_params = {"max_errors": 0}
|
| 44 |
+
default_severity = FAIL
|
| 45 |
+
|
| 46 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 47 |
+
findings: list[Finding] = []
|
| 48 |
+
for stream in ctx.streams_of("rgb"):
|
| 49 |
+
report = stream.video or {}
|
| 50 |
+
if not report:
|
| 51 |
+
continue
|
| 52 |
+
|
| 53 |
+
reason = str(report.get("reason") or "")
|
| 54 |
+
if reason in _ENV_REASONS:
|
| 55 |
+
findings.append(self.finding(
|
| 56 |
+
ctx, ERROR, stream=stream.key,
|
| 57 |
+
message="OpenCV unavailable, could not check (environment issue, not a data issue)",
|
| 58 |
+
metrics={"reason": reason},
|
| 59 |
+
))
|
| 60 |
+
continue
|
| 61 |
+
|
| 62 |
+
# 整条流打不开 —— 比丢几帧严重得多
|
| 63 |
+
if reason in ("video_open_failed", "video_metadata_invalid"):
|
| 64 |
+
findings.append(self.finding(
|
| 65 |
+
ctx, FAIL, stream=stream.key,
|
| 66 |
+
message=f"video cannot be opened or has invalid metadata ({reason})",
|
| 67 |
+
metrics={"reason": reason,
|
| 68 |
+
"error": str(report.get("error") or "")[:200]},
|
| 69 |
+
))
|
| 70 |
+
continue
|
| 71 |
+
|
| 72 |
+
count = int(report.get("decode_error_count") or 0)
|
| 73 |
+
max_errors = int(self.param(ctx, "max_errors", 0))
|
| 74 |
+
if count <= max_errors:
|
| 75 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 76 |
+
metrics={"decode_error_count": count}))
|
| 77 |
+
continue
|
| 78 |
+
|
| 79 |
+
fps = float(report.get("fps") or stream.fps or ctx.fps or 30.0)
|
| 80 |
+
frames = [int(item) for item in (report.get("decode_errors") or [])
|
| 81 |
+
if isinstance(item, (int, float))]
|
| 82 |
+
# 每个失败帧给一个零长度区间 —— 时间轴上就是一个可点击的红点
|
| 83 |
+
marks = tuple(
|
| 84 |
+
Range(start_sec=frame / fps, end_sec=frame / fps,
|
| 85 |
+
code=self.slug, severity=FAIL,
|
| 86 |
+
stream=stream.key, channel="rgb",
|
| 87 |
+
start_frame=frame, end_frame=frame,
|
| 88 |
+
rule=self.slug, message=f"frame {frame} failed to decode")
|
| 89 |
+
for frame in frames[:20]
|
| 90 |
+
)
|
| 91 |
+
findings.append(self.finding(
|
| 92 |
+
ctx, FAIL, stream=stream.key,
|
| 93 |
+
message=f"{count} frames failed to decode"
|
| 94 |
+
+ (f"(示例:{frames[:5]})" if frames else ""),
|
| 95 |
+
metrics={"decode_error_count": count,
|
| 96 |
+
"sample_frames": frames[:20]},
|
| 97 |
+
ranges=marks,
|
| 98 |
+
))
|
| 99 |
+
return findings
|
processor/app/processing/cleaning/checks/video/frame_drop.py
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""丢帧 / 帧数对齐 —— 视频帧数与 parquet 行数不一致。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
两者不一致时,按帧号取动作会整体错位;漏帧则会让模型学到"同一帧对应两个
|
| 6 |
+
动作"。真实的采集侧表现是视频比数据少几帧(编码器丢帧)或多几帧(尾部多录)。
|
| 7 |
+
|
| 8 |
+
判定口径
|
| 9 |
+
--------
|
| 10 |
+
容差取 ``max(tolerance_min, data_rows × tolerance_ratio)``:
|
| 11 |
+
小批次按绝对帧数兜底(默认 2 帧),大批次按比例(默认 0.5%)。
|
| 12 |
+
超出容差 5 倍判 FAIL,否则 WARN —— 差几帧和差几百帧不是一回事。
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
from app.processing.cleaning.contract import VIDEO_CAPABLE_MODALITIES
|
| 18 |
+
from app.processing.cleaning.checks import (
|
| 19 |
+
CheckContext, CleaningCheck, register_check,
|
| 20 |
+
Finding, PASS, WARN, FAIL,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
@register_check
|
| 25 |
+
class VideoFrameDropCheck(CleaningCheck):
|
| 26 |
+
slug = "video.frame_drop"
|
| 27 |
+
label = "Frame Drop / Count Alignment"
|
| 28 |
+
version = "1.0"
|
| 29 |
+
description = "Video frame count differs from data row count"
|
| 30 |
+
|
| 31 |
+
requires_channels = ("rgb",)
|
| 32 |
+
|
| 33 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 34 |
+
|
| 35 |
+
device_cards = VIDEO_CAPABLE_MODALITIES
|
| 36 |
+
|
| 37 |
+
# 阈值默认值 —— 节点配置可覆盖。
|
| 38 |
+
# 方案要求「阈值不要照搬」:这是平台起点值,必须用人工确认过的好/坏数据验证。
|
| 39 |
+
default_params = {"tolerance_ratio": 0.005, "tolerance_min": 2}
|
| 40 |
+
default_severity = WARN
|
| 41 |
+
|
| 42 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 43 |
+
findings: list[Finding] = []
|
| 44 |
+
for stream in ctx.streams_of("rgb"):
|
| 45 |
+
report = stream.video or {}
|
| 46 |
+
if not report:
|
| 47 |
+
continue
|
| 48 |
+
# 打不开/读不了元数据的流不算"丢帧问题",交给 video.decode_error
|
| 49 |
+
if report.get("reason") in ("video_open_failed", "video_metadata_invalid"):
|
| 50 |
+
continue
|
| 51 |
+
|
| 52 |
+
video_frames = int(report.get("frame_count") or 0)
|
| 53 |
+
data_rows = int(stream.rows or ctx.data_rows or 0)
|
| 54 |
+
if not video_frames or not data_rows:
|
| 55 |
+
continue
|
| 56 |
+
|
| 57 |
+
diff = abs(video_frames - data_rows)
|
| 58 |
+
tolerance = max(
|
| 59 |
+
int(self.param(ctx, "tolerance_min", 2)),
|
| 60 |
+
int(data_rows * float(self.param(ctx, "tolerance_ratio", 0.005))),
|
| 61 |
+
)
|
| 62 |
+
metrics = {
|
| 63 |
+
"video_frames": video_frames,
|
| 64 |
+
"data_rows": data_rows,
|
| 65 |
+
"diff": diff,
|
| 66 |
+
"tolerance": tolerance,
|
| 67 |
+
}
|
| 68 |
+
|
| 69 |
+
if diff <= tolerance:
|
| 70 |
+
findings.append(self.finding(ctx, PASS, stream=stream.key,
|
| 71 |
+
metrics=metrics))
|
| 72 |
+
continue
|
| 73 |
+
|
| 74 |
+
findings.append(self.finding(
|
| 75 |
+
ctx,
|
| 76 |
+
FAIL if diff > tolerance * 5 else WARN,
|
| 77 |
+
stream=stream.key,
|
| 78 |
+
message=f"video has {video_frames} frames / data has {data_rows} rows"
|
| 79 |
+
f",差 {diff} 帧(容差 {tolerance})",
|
| 80 |
+
metrics=metrics,
|
| 81 |
+
))
|
| 82 |
+
return findings
|
processor/app/processing/cleaning/checks/video/freeze.py
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""冻结 —— 画面连续多帧几乎不变。
|
| 2 |
+
|
| 3 |
+
为什么重要
|
| 4 |
+
----------
|
| 5 |
+
冻结段通常是采集程序卡住、编码器 stall、或 USB 带宽不足导致丢帧后补重复帧。
|
| 6 |
+
它和黑屏不同:画面是"正常的样子",但时间已经不动了。模型会以为任务停滞。
|
| 7 |
+
|
| 8 |
+
注意与"合理的静止"区分
|
| 9 |
+
----------------------
|
| 10 |
+
人拿着夹爪停在物体上方思考时,画面也接近静止。所以本项默认判 **WARN 而非
|
| 11 |
+
FAIL**,并在 metrics 里带上每段时长供人工/AI 复核 —— 方案明确要求
|
| 12 |
+
「阈值不要照搬」,静止多久算异常要按任务节奏定。
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
from app.processing.cleaning.contract import VIDEO_CAPABLE_MODALITIES
|
| 18 |
+
from app.processing.cleaning.checks import (
|
| 19 |
+
CheckContext, CleaningCheck, register_check,
|
| 20 |
+
Finding, Range, WARN, FAIL,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
@register_check
|
| 25 |
+
class VideoFreezeCheck(CleaningCheck):
|
| 26 |
+
slug = "video.freeze"
|
| 27 |
+
label = "Frozen Frame"
|
| 28 |
+
version = "1.0"
|
| 29 |
+
description = "Frame nearly unchanged across consecutive frames"
|
| 30 |
+
|
| 31 |
+
requires_channels = ("rgb",)
|
| 32 |
+
|
| 33 |
+
# 属于哪些设备卡片 —— 前端设置面板按它分 tab
|
| 34 |
+
|
| 35 |
+
device_cards = VIDEO_CAPABLE_MODALITIES
|
| 36 |
+
|
| 37 |
+
default_params = {"min_sec": 2.0, "fail_sec": 10.0}
|
| 38 |
+
default_severity = WARN
|
| 39 |
+
|
| 40 |
+
def run(self, ctx: CheckContext) -> list[Finding]:
|
| 41 |
+
findings: list[Finding] = []
|
| 42 |
+
for stream in ctx.streams_of("rgb"):
|
| 43 |
+
report = stream.video or {}
|
| 44 |
+
ranges = report.get("freeze_ranges") or []
|
| 45 |
+
if not ranges:
|
| 46 |
+
continue
|
| 47 |
+
|
| 48 |
+
fps = float(report.get("fps") or stream.fps or ctx.fps or 30.0)
|
| 49 |
+
min_sec = float(self.param(ctx, "min_sec", 2.0))
|
| 50 |
+
fail_sec = float(self.param(ctx, "fail_sec", 10.0))
|
| 51 |
+
|
| 52 |
+
converted: list[Range] = []
|
| 53 |
+
longest = 0.0
|
| 54 |
+
worst_status = WARN
|
| 55 |
+
for item in ranges:
|
| 56 |
+
try:
|
| 57 |
+
start, end = float(item[0]), float(item[1])
|
| 58 |
+
except (TypeError, ValueError, IndexError):
|
| 59 |
+
continue
|
| 60 |
+
span = end - start
|
| 61 |
+
if span < min_sec:
|
| 62 |
+
continue
|
| 63 |
+
longest = max(longest, span)
|
| 64 |
+
status = FAIL if span >= fail_sec else WARN
|
| 65 |
+
if status == FAIL:
|
| 66 |
+
worst_status = FAIL
|
| 67 |
+
converted.append(Range(
|
| 68 |
+
start_sec=start, end_sec=end,
|
| 69 |
+
code=self.slug, severity=status,
|
| 70 |
+
stream=stream.key, channel="rgb",
|
| 71 |
+
start_frame=int(start * fps), end_frame=int(end * fps),
|
| 72 |
+
rule=self.slug,
|
| 73 |
+
message=f"frozen for {span:.1f}s",
|
| 74 |
+
))
|
| 75 |
+
|
| 76 |
+
if not converted:
|
| 77 |
+
continue
|
| 78 |
+
|
| 79 |
+
findings.append(self.finding(
|
| 80 |
+
ctx, worst_status, stream=stream.key,
|
| 81 |
+
message=f"{len(converted)} frozen stretches, longest {longest:.1f}s",
|
| 82 |
+
metrics={"segments": len(converted),
|
| 83 |
+
"longest_sec": round(longest, 3)},
|
| 84 |
+
ranges=converted,
|
| 85 |
+
))
|
| 86 |
+
return findings
|
processor/app/processing/cleaning/collect.py
ADDED
|
@@ -0,0 +1,647 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""证据采集 —— 从批次目录读出检查项需要的一切。
|
| 2 |
+
|
| 3 |
+
检查项**不碰磁盘**(见 ``checks.CheckContext`` 的 IO 注入设计)。所有读取集中在
|
| 4 |
+
这里,好处有两个:
|
| 5 |
+
|
| 6 |
+
* 检查项本体是纯逻辑,测试喂假证据即可,不需要真 mp4 / parquet / cv2;
|
| 7 |
+
* "读什么、怎么读"只有一处实现,不会出现两个检查项对同一列用不同口径。
|
| 8 |
+
|
| 9 |
+
**本模块只读** —— 不写任何文件、不改任何数据。质检出结论,不出数据。
|
| 10 |
+
"""
|
| 11 |
+
|
| 12 |
+
from __future__ import annotations
|
| 13 |
+
|
| 14 |
+
import glob
|
| 15 |
+
import re
|
| 16 |
+
from dataclasses import dataclass, field as dc_field
|
| 17 |
+
from pathlib import Path
|
| 18 |
+
from typing import Any, Iterable
|
| 19 |
+
|
| 20 |
+
import numpy as np
|
| 21 |
+
|
| 22 |
+
from app.processing.cleaning.checks import StreamEvidence
|
| 23 |
+
from app.processing.cleaning.contract import (
|
| 24 |
+
pose_is_valid,
|
| 25 |
+
channels_for, normalize_device_modality,
|
| 26 |
+
)
|
| 27 |
+
|
| 28 |
+
# SLAM 列的命名:``observation.slam_*`` 是默认相机;双目/多把第二路存成
|
| 29 |
+
# ``observation.<相机名>_slam_*``。这里按**相机**分组,每组里 trajectory 优先。
|
| 30 |
+
#
|
| 31 |
+
# 为什么不能写死两个列名:实测 Test94/episode_000003 **只有** gripper_2 前缀的
|
| 32 |
+
# 两列,没有不带前缀的 —— 那种集有 SLAM 数据、却一个 SLAM 检查都不跑,且不报
|
| 33 |
+
# 任何错。同一项目里不同集的列名不一致是常态(见 MODALITY_CHANNELS 的告诫)。
|
| 34 |
+
#
|
| 35 |
+
# 两个**不要**匹配进去的近亲:
|
| 36 |
+
# observation.slam_trajectory_ns —— 时间戳伴生列,不是位姿
|
| 37 |
+
# processing.umi_slam_action.data_4.action —— 处理缓存列,名字里恰好含 slam
|
| 38 |
+
_SLAM_COLUMN_RE = re.compile(r"^observation\.(?:(.+)_)?slam_(trajectory|pose)$")
|
| 39 |
+
|
| 40 |
+
# 列名 → 信道。值为**可能的列名**(含历史名)。同一信道的每个存在的列都会
|
| 41 |
+
# 登记成一条流(左右两路力就是两条),流名取列名最后一段。
|
| 42 |
+
#
|
| 43 |
+
# ★ 这里**不能**用 `break` 只取第一个 —— 那样左右两路力只剩一路。
|
| 44 |
+
# 要与别名列去重的话看 ``_COLUMN_ALIASES``,别在这里提前退出。
|
| 45 |
+
#
|
| 46 |
+
# slam 不在下表里 —— 它按相机分组解析,见 ``_slam_columns``。
|
| 47 |
+
_COLUMN_CHANNELS: dict[str, tuple[str, ...]] = {
|
| 48 |
+
"slam": (
|
| 49 |
+
"observation.slam_trajectory", # 高频样本缓冲(优先)
|
| 50 |
+
"observation.slam_pose", # 低频快照(旧批才有)
|
| 51 |
+
),
|
| 52 |
+
"gripper_state": ("observation.gripper_state",),
|
| 53 |
+
"force": (
|
| 54 |
+
"observation.gripper_left_force",
|
| 55 |
+
"observation.gripper_right_force",
|
| 56 |
+
"observation.gripper_force",
|
| 57 |
+
),
|
| 58 |
+
"tactile": (
|
| 59 |
+
"observation.left_glove",
|
| 60 |
+
"observation.right_glove",
|
| 61 |
+
"observation.glove",
|
| 62 |
+
),
|
| 63 |
+
"device_status": (
|
| 64 |
+
"status.left_glove",
|
| 65 |
+
"status.right_glove",
|
| 66 |
+
"status.glove",
|
| 67 |
+
),
|
| 68 |
+
"depth": ("observation.images.depth",),
|
| 69 |
+
"action": ("action",),
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
# 别名列 → 协议列名。**同一份数据的第二套命名**,不新增信道,只是改名。
|
| 73 |
+
#
|
| 74 |
+
# 触觉数据有两套名字:
|
| 75 |
+
# 采集协议写 observation.left_glove / right_glove
|
| 76 |
+
# 导出时统一成 observation.tactile.left / right
|
| 77 |
+
# (见 lerobot_export._TACTILE_RENAME,hdf5_export / ai_annotation 读的是后者)
|
| 78 |
+
#
|
| 79 |
+
# 原始 session 目录里两套可能同时在(导出产物被合并回 canonical);而**只跑过
|
| 80 |
+
# 导出的树里只有后者**。此前只认协议名 —— 那种树上手套信道直接为空,所有手套
|
| 81 |
+
# 检查静默跳过,不报任何错。
|
| 82 |
+
_COLUMN_ALIASES: dict[str, str] = {
|
| 83 |
+
"observation.tactile.left": "observation.left_glove",
|
| 84 |
+
"observation.tactile.right": "observation.right_glove",
|
| 85 |
+
}
|
| 86 |
+
|
| 87 |
+
# 各信道的向量宽度 —— 用于把列值切成定长元组。
|
| 88 |
+
#
|
| 89 |
+
# ★ 不在表里的信道**不截断**(见 ``vector_series``)。默认截断到某个数字是
|
| 90 |
+
# 危险的:触觉阵列是 16×16=256 维,此前因为这里没有 ``tactile`` 项而落到
|
| 91 |
+
# 3 的兜底值,256 个感应点被静默截成 3 个 —— 检查项只看得到 1% 的阵列。
|
| 92 |
+
# 换手套(或加新阵列通道)时同样会静默丢数据,所以默认必须是"不截断"。
|
| 93 |
+
_CHANNEL_WIDTH: dict[str, int] = {
|
| 94 |
+
"gripper_state": 3,
|
| 95 |
+
"force": 3,
|
| 96 |
+
"action": 7,
|
| 97 |
+
}
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
def _apply_column_aliases(frame):
|
| 101 |
+
"""把别名列归一成协议列名。
|
| 102 |
+
|
| 103 |
+
**只在协议名缺失时改** —— 两套都在时保留协议名:它们是同一份数据,
|
| 104 |
+
登记成两条流会让同一处缺陷被报两次,也让"左右手"的流名对不上。
|
| 105 |
+
(实测 S80C / D435 两套都在时逐元素完全相同。)
|
| 106 |
+
"""
|
| 107 |
+
rename = {alias: canonical for alias, canonical in _COLUMN_ALIASES.items()
|
| 108 |
+
if alias in frame.columns and canonical not in frame.columns}
|
| 109 |
+
return frame.rename(columns=rename) if rename else frame
|
| 110 |
+
|
| 111 |
+
_DEFAULT_FPS = 30.0
|
| 112 |
+
_MAX_VIDEO_PROBES = 4
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
@dataclass(frozen=True)
|
| 116 |
+
class Evidence:
|
| 117 |
+
"""一个 episode 的全部证据 —— engine 交给检查项的那一份。"""
|
| 118 |
+
|
| 119 |
+
batch_dir: Path
|
| 120 |
+
streams: tuple[StreamEvidence, ...] = ()
|
| 121 |
+
channels: frozenset[str] = frozenset()
|
| 122 |
+
modalities: tuple[str, ...] = ()
|
| 123 |
+
fps: float = _DEFAULT_FPS
|
| 124 |
+
rows: int = 0
|
| 125 |
+
columns: tuple[str, ...] = ()
|
| 126 |
+
parquet: str = ""
|
| 127 |
+
notes: dict[str, Any] = dc_field(default_factory=dict)
|
| 128 |
+
|
| 129 |
+
@property
|
| 130 |
+
def duration_sec(self) -> float:
|
| 131 |
+
return (self.rows / self.fps) if self.fps else 0.0
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
# ── 读文件 ────────────────────────────────────────────────────
|
| 135 |
+
|
| 136 |
+
def _glob(pattern: str, root: Path) -> list[Path]:
|
| 137 |
+
return [Path(item) for item in sorted(glob.glob(str(root / pattern)))]
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
def _file_holding(parquets: list[Path], episode_index: int) -> Path | None:
|
| 141 |
+
"""哪个文件里装着这一集 —— 只读 ``episode_index`` 一列,不整份加载。
|
| 142 |
+
|
| 143 |
+
文件名对不上时的兜底(历史命名、手工搬过的目录)。返回 None 表示都没有。
|
| 144 |
+
"""
|
| 145 |
+
import pyarrow.parquet as pq
|
| 146 |
+
|
| 147 |
+
for path in parquets:
|
| 148 |
+
try:
|
| 149 |
+
column = pq.ParquetFile(path).read(columns=["episode_index"])
|
| 150 |
+
except Exception:
|
| 151 |
+
continue
|
| 152 |
+
values = column.column("episode_index").to_pylist()
|
| 153 |
+
if episode_index in values:
|
| 154 |
+
return path
|
| 155 |
+
return None
|
| 156 |
+
|
| 157 |
+
|
| 158 |
+
def resolve_episode_data(batch_dir: Path,
|
| 159 |
+
episode_index: int | None = None) -> tuple[Path | None, bool]:
|
| 160 |
+
"""定位这一集的数据文件。返回 ``(parquet, 还要不要按行过滤)``。
|
| 161 |
+
|
| 162 |
+
两种布局的粒度**不一样**,混了就会读出空帧:
|
| 163 |
+
|
| 164 |
+
canonical ``data/chunk-*/episode_NNNNNN.parquet`` —— 一集一文件。
|
| 165 |
+
选对**文件**即可,行全是这一集的。
|
| 166 |
+
导出产物 ``data/chunk-*/file-NNN.parquet`` —— 一文件多集,
|
| 167 |
+
必须再按 ``episode_index`` 列筛行。
|
| 168 |
+
|
| 169 |
+
★ 这里曾经是「取 ``parquets[0]`` 再按列过滤」。canonical 布局下第 0 个文件
|
| 170 |
+
只装第 0 集,于是传任何**别的** ``episode_index`` 都筛出空帧 —— 不报错,
|
| 171 |
+
只是报告变成 ``empty_report``(``episode.status = ERROR``)。在"数据不对
|
| 172 |
+
就拦回人工审核"的链路里,表现是**除第 0 集外每一集都被卡住**。
|
| 173 |
+
"""
|
| 174 |
+
root = Path(batch_dir)
|
| 175 |
+
|
| 176 |
+
canonical = _glob("data/chunk-*/episode_*.parquet", root)
|
| 177 |
+
if canonical:
|
| 178 |
+
if episode_index is None:
|
| 179 |
+
return canonical[0], False
|
| 180 |
+
# 文件名就是集号(LeRobot canonical 约定,注意编号可能有空洞)
|
| 181 |
+
wanted = f"episode_{int(episode_index):06d}.parquet"
|
| 182 |
+
for path in canonical:
|
| 183 |
+
if path.name == wanted:
|
| 184 |
+
return path, False
|
| 185 |
+
# 文件名对不上 —— 按列找,别静默返回空
|
| 186 |
+
return _file_holding(canonical, int(episode_index)), False
|
| 187 |
+
|
| 188 |
+
multi = _glob("data/chunk-*/*.parquet", root)
|
| 189 |
+
if not multi:
|
| 190 |
+
return None, False
|
| 191 |
+
if episode_index is None:
|
| 192 |
+
return multi[0], False
|
| 193 |
+
return _file_holding(multi, int(episode_index)), True
|
| 194 |
+
|
| 195 |
+
|
| 196 |
+
def load_rows(parquet: Path, filter_episode_index: int | None = None):
|
| 197 |
+
"""读 parquet;给了 ``filter_episode_index`` 就只取那一集的行。
|
| 198 |
+
|
| 199 |
+
只在**一文件多集**的布局下才该传它 —— canonical 的一集一文件布局下传了会
|
| 200 |
+
把这一集自己的行全筛掉。用 ``resolve_episode_data`` 拿到的第二个返回值决定。
|
| 201 |
+
"""
|
| 202 |
+
import pyarrow.parquet as pq
|
| 203 |
+
|
| 204 |
+
frame = pq.ParquetFile(parquet).read().to_pandas()
|
| 205 |
+
if filter_episode_index is not None and "episode_index" in frame.columns:
|
| 206 |
+
frame = frame[frame["episode_index"].values == filter_episode_index]
|
| 207 |
+
return frame.reset_index(drop=True)
|
| 208 |
+
|
| 209 |
+
|
| 210 |
+
# ── 列 → 序列 ─────────────────────────────────────────────────
|
| 211 |
+
|
| 212 |
+
def slam_valid_counts(values) -> tuple[int, ...]:
|
| 213 |
+
"""每帧的有效位姿样本数 —— 空帧即为 0。
|
| 214 |
+
|
| 215 |
+
slam_trajectory 是扁平的 ``[t,x,y,z,qx,qy,qz,qw]×N``。导出产物会把它补齐到
|
| 216 |
+
定长(补 NaN),所以"有效"= 非 NaN 的 8 元组个数。
|
| 217 |
+
"""
|
| 218 |
+
counts: list[int] = []
|
| 219 |
+
for item in values:
|
| 220 |
+
if item is None:
|
| 221 |
+
counts.append(0)
|
| 222 |
+
continue
|
| 223 |
+
array = np.asarray(item, dtype=float).ravel()
|
| 224 |
+
valid = 0
|
| 225 |
+
for index in range(array.size // 8):
|
| 226 |
+
if np.isfinite(array[index * 8:index * 8 + 8]).all():
|
| 227 |
+
valid += 1
|
| 228 |
+
counts.append(valid)
|
| 229 |
+
return tuple(counts)
|
| 230 |
+
|
| 231 |
+
|
| 232 |
+
def slam_poses(values, counts) -> tuple[tuple, ...]:
|
| 233 |
+
"""逐帧取【第一个有效样本】的位姿(位置 + 四元数)。
|
| 234 |
+
|
| 235 |
+
★ 刻意不重用 ``umi_slam_action.poses_from_trajectory`` 的插值重建 ——
|
| 236 |
+
那是生产口径(含时间基拟合与插值)。连续性检查要看的是**原始落盘数据**
|
| 237 |
+
有没有跳变;用插值后的平滑轨迹会掩盖真实跳变���
|
| 238 |
+
"""
|
| 239 |
+
poses: list[tuple] = []
|
| 240 |
+
for item, count in zip(values, counts):
|
| 241 |
+
if not count:
|
| 242 |
+
poses.append(())
|
| 243 |
+
continue
|
| 244 |
+
array = np.asarray(item, dtype=float).ravel()
|
| 245 |
+
poses.append(tuple(float(v) for v in array[1:8]))
|
| 246 |
+
return tuple(poses)
|
| 247 |
+
|
| 248 |
+
|
| 249 |
+
def _slam_columns(columns) -> list[str]:
|
| 250 |
+
"""按**相机**挑出 SLAM 列,每个相机一个:trajectory 优先,回落到 pose。
|
| 251 |
+
|
| 252 |
+
相机名 = ``observation.`` 与 ``_slam_`` 之间那段,默认相机没有前缀:
|
| 253 |
+
|
| 254 |
+
observation.slam_trajectory → 相机 ""
|
| 255 |
+
observation.gripper_2_slam_trajectory → 相机 "gripper_2"
|
| 256 |
+
|
| 257 |
+
每个相机只取一个 —— trajectory 是高频样本缓冲、pose 是低频快照,
|
| 258 |
+
两个都有时用前者(与旧行为一致,只是现在**每个相机各自**取一个,
|
| 259 |
+
而不是全局只取一个)。
|
| 260 |
+
"""
|
| 261 |
+
by_camera: dict[str, dict[str, str]] = {}
|
| 262 |
+
for column in sorted(str(name) for name in columns):
|
| 263 |
+
match = _SLAM_COLUMN_RE.match(column)
|
| 264 |
+
if not match:
|
| 265 |
+
continue
|
| 266 |
+
camera, kind = match.group(1) or "", match.group(2)
|
| 267 |
+
by_camera.setdefault(camera, {})[kind] = column
|
| 268 |
+
|
| 269 |
+
picked: list[str] = []
|
| 270 |
+
for camera in sorted(by_camera):
|
| 271 |
+
kinds = by_camera[camera]
|
| 272 |
+
picked.append(kinds.get("trajectory") or kinds["pose"])
|
| 273 |
+
return picked
|
| 274 |
+
|
| 275 |
+
|
| 276 |
+
def _column_series(values, channel: str, width: int | None = None) -> tuple:
|
| 277 |
+
"""一列 → 序列。**字符串列原样保留,不试着数值化。**
|
| 278 |
+
|
| 279 |
+
``vector_series`` 对字符串会走 ``np.asarray(..., dtype=float)`` 抛
|
| 280 |
+
``ValueError``,被 ``except`` 吞掉后**每帧都变成空元组** —— 检查项拿到的是
|
| 281 |
+
"这一列没有数据",看起来像"检查通过"。设备状态列(``status.left_glove``)
|
| 282 |
+
正是字符串,所以这里必须先分流。
|
| 283 |
+
"""
|
| 284 |
+
if len(values) and isinstance(values.iloc[0], str):
|
| 285 |
+
return tuple("" if item is None else str(item) for item in values)
|
| 286 |
+
if width is None:
|
| 287 |
+
width = _CHANNEL_WIDTH.get(channel)
|
| 288 |
+
return vector_series(values.tolist(), width)
|
| 289 |
+
|
| 290 |
+
|
| 291 |
+
def vector_series(values, width: int | None = None) -> tuple[tuple, ...]:
|
| 292 |
+
"""把一列向量值转成元组序列;空/坏的写成空元组。
|
| 293 |
+
|
| 294 |
+
``width=None``(默认)**不截断**。截断是静默的 —— 调用方只有显式声明了
|
| 295 |
+
正确宽度时才该发生(如 action 固定 7 维);未知列宽一律保留整条,
|
| 296 |
+
免得"少了一半感应点"这种问题要等到训练效果不对才发现。
|
| 297 |
+
"""
|
| 298 |
+
out: list[tuple] = []
|
| 299 |
+
for item in values:
|
| 300 |
+
if item is None:
|
| 301 |
+
out.append(())
|
| 302 |
+
continue
|
| 303 |
+
try:
|
| 304 |
+
array = np.asarray(item, dtype=float).ravel()
|
| 305 |
+
if width is not None:
|
| 306 |
+
array = array[:width]
|
| 307 |
+
out.append(tuple(float(v) for v in array))
|
| 308 |
+
except (TypeError, ValueError):
|
| 309 |
+
out.append(())
|
| 310 |
+
return tuple(out)
|
| 311 |
+
|
| 312 |
+
|
| 313 |
+
def _video_files(video_dir: Path) -> list[Path]:
|
| 314 |
+
"""列出视频目录下的 mp4(按名字排序,稳定可预期)。"""
|
| 315 |
+
root = Path(video_dir)
|
| 316 |
+
if not root.is_dir():
|
| 317 |
+
return []
|
| 318 |
+
return sorted(path for path in root.rglob("*.mp4") if path.is_file())
|
| 319 |
+
|
| 320 |
+
|
| 321 |
+
def videos_for_episode(root: Path, episode_index: int | None) -> list[Path]:
|
| 322 |
+
"""**这一集自己的**视频文件。取不到就返回空 —— 绝不猜。
|
| 323 |
+
|
| 324 |
+
★ 不能"把 videos/ 下的 mp4 都拿来用":v3.0 导出把 52 集的视频全放在
|
| 325 |
+
``videos/chunk-000/<流名>/file-NNN.mp4`` 同一个目录里,随便取几个会拿
|
| 326 |
+
别的集的画面去比对这一集的数据行数,得出荒谬的丢帧结论(实测出现过
|
| 327 |
+
"视频 9164 帧 / 数据 819 行")。
|
| 328 |
+
|
| 329 |
+
两种布局各有映射来源:
|
| 330 |
+
* canonical —— ``videos/<流名>/chunk-XXX/episode_XXXXXX.mp4``,
|
| 331 |
+
目录本身就对应这一集;有 ``project_dataset.episode_files`` 时用它。
|
| 332 |
+
* 导出产物 —— ``meta/episodes`` 里逐流记着 ``chunk_index``/``file_index``。
|
| 333 |
+
"""
|
| 334 |
+
root = Path(root)
|
| 335 |
+
if episode_index is None:
|
| 336 |
+
# 一集一文件的布局:videos/ 下就是这一集的全部视频
|
| 337 |
+
return _video_files(root / "videos")
|
| 338 |
+
|
| 339 |
+
# canonical:走既有索引
|
| 340 |
+
try:
|
| 341 |
+
from app.project_dataset import episode_files
|
| 342 |
+
entry = episode_files(root, episode_index) or {}
|
| 343 |
+
found: list[Path] = []
|
| 344 |
+
for item in entry.get("videos") or []:
|
| 345 |
+
# episode_files 返回 (source_key, path) 的形式
|
| 346 |
+
path = item[1] if isinstance(item, (tuple, list)) and len(item) > 1 else item
|
| 347 |
+
candidate = Path(path)
|
| 348 |
+
if candidate.is_file():
|
| 349 |
+
found.append(candidate)
|
| 350 |
+
if found:
|
| 351 |
+
return found
|
| 352 |
+
except Exception:
|
| 353 |
+
pass
|
| 354 |
+
|
| 355 |
+
# 导出产物:查 meta/episodes 的逐流索引
|
| 356 |
+
return _videos_from_episode_meta(root, episode_index)
|
| 357 |
+
|
| 358 |
+
|
| 359 |
+
def _videos_from_episode_meta(root: Path, episode_index: int) -> list[Path]:
|
| 360 |
+
"""从 ``meta/episodes`` 的逐流 chunk/file index 拼出视频路径。"""
|
| 361 |
+
import pyarrow.parquet as pq
|
| 362 |
+
|
| 363 |
+
for path in sorted(glob.glob(str(root / "meta" / "episodes" / "**" / "*.parquet"),
|
| 364 |
+
recursive=True)):
|
| 365 |
+
try:
|
| 366 |
+
frame = pq.ParquetFile(path).read().to_pandas()
|
| 367 |
+
except Exception:
|
| 368 |
+
continue
|
| 369 |
+
if "episode_index" not in frame.columns:
|
| 370 |
+
continue
|
| 371 |
+
rows = frame[frame["episode_index"].values == episode_index]
|
| 372 |
+
if rows.empty:
|
| 373 |
+
continue
|
| 374 |
+
|
| 375 |
+
# 列名形如 videos/<流名>/chunk_index 与 videos/<流名>/file_index
|
| 376 |
+
columns = {str(name) for name in frame.columns}
|
| 377 |
+
streams = sorted({
|
| 378 |
+
name[len("videos/"):-len("/chunk_index")]
|
| 379 |
+
for name in columns
|
| 380 |
+
if name.startswith("videos/") and name.endswith("/chunk_index")
|
| 381 |
+
})
|
| 382 |
+
row = rows.iloc[0]
|
| 383 |
+
found: list[Path] = []
|
| 384 |
+
for stream in streams:
|
| 385 |
+
try:
|
| 386 |
+
chunk = int(row[f"videos/{stream}/chunk_index"])
|
| 387 |
+
file_index = int(row[f"videos/{stream}/file_index"])
|
| 388 |
+
except (KeyError, TypeError, ValueError):
|
| 389 |
+
continue
|
| 390 |
+
candidate = (root / "videos" / f"chunk-{chunk:03d}" / stream
|
| 391 |
+
/ f"file-{file_index:03d}.mp4")
|
| 392 |
+
if candidate.is_file():
|
| 393 |
+
found.append(candidate)
|
| 394 |
+
if found:
|
| 395 |
+
return found
|
| 396 |
+
return []
|
| 397 |
+
|
| 398 |
+
|
| 399 |
+
def _is_depth_key(name: str) -> bool:
|
| 400 |
+
"""这个流名是不是纯深度流。
|
| 401 |
+
|
| 402 |
+
复用 ``lerobot_v21.is_depth_source`` —— 深度口径在这个仓库里有好几处实现
|
| 403 |
+
(``device_naming.is_depth_only_key`` 是同一套),但权威定义在那儿。
|
| 404 |
+
导入失败时回落到保守判断:名字含 depth 且不含 rgb/color。
|
| 405 |
+
"""
|
| 406 |
+
try:
|
| 407 |
+
from app.lerobot_v21 import is_depth_source
|
| 408 |
+
return bool(is_depth_source(name))
|
| 409 |
+
except Exception:
|
| 410 |
+
low = str(name or "").lower()
|
| 411 |
+
return "depth" in low and not any(
|
| 412 |
+
token in low for token in ("rgb", "color", "video"))
|
| 413 |
+
|
| 414 |
+
|
| 415 |
+
# ── 主入口 ────────────────────────────────────────────────────
|
| 416 |
+
|
| 417 |
+
def gather_evidence(batch_dir: Path, *,
|
| 418 |
+
episode_index: int | None = None,
|
| 419 |
+
probe_video: bool = False,
|
| 420 |
+
modality_hint: str | Iterable[str] | None = None) -> Evidence:
|
| 421 |
+
"""读一个 episode 的全部证据。
|
| 422 |
+
|
| 423 |
+
``probe_video=False``(默认)只读 parquet,毫秒级;视频检查要解码抽样,
|
| 424 |
+
一集几秒到几十秒,按需开。
|
| 425 |
+
|
| 426 |
+
``modality_hint`` 用于告诉采集层"这台设备是什么"(来自工作流连线的设备
|
| 427 |
+
卡片)。**可以是多个** —— 一个质检节点上游同时连相机和夹爪时就有多个。
|
| 428 |
+
给不出时按数据特征推断;两个来源取并集,理由见 ``_detect_modalities``。
|
| 429 |
+
|
| 430 |
+
★ ``batch_dir`` 必须是**单个 episode 的目录**,且 ``episode_index`` 必须
|
| 431 |
+
与它匹配。``localstore.get_episode()`` 返回的 ``path`` 是**项目目录**
|
| 432 |
+
(几十集共享),直接传进来会拿第 0 集的 parquet 去比全部集的视频
|
| 433 |
+
—— 实测过,会得出"视频 9164 帧 / 数据 819 行"。见 ``videos_for_episode``。
|
| 434 |
+
"""
|
| 435 |
+
root = Path(batch_dir)
|
| 436 |
+
parquet, filter_rows = resolve_episode_data(root, episode_index)
|
| 437 |
+
if parquet is None:
|
| 438 |
+
return Evidence(batch_dir=root, notes={"reason": "no_parquet"})
|
| 439 |
+
|
| 440 |
+
# ★ 按行过滤只在"一文件多集"的布局下做。canonical 是一集一文件,文件已经
|
| 441 |
+
# 选对了,再按 episode_index 列筛会把这一集自己的行全筛空 —— 结果是不报
|
| 442 |
+
# 任何错的 empty_report。见 resolve_episode_data。
|
| 443 |
+
frame = load_rows(parquet, episode_index if filter_rows else None)
|
| 444 |
+
if frame.empty:
|
| 445 |
+
return Evidence(batch_dir=root, parquet=str(parquet.relative_to(root)),
|
| 446 |
+
notes={"reason": "empty_episode"})
|
| 447 |
+
|
| 448 |
+
# 别名列先归一成协议名,再算 columns —— 必须在下面所有列名判断之前,
|
| 449 |
+
# 否则 observation.tactile.* 会被当成"这列不存在"
|
| 450 |
+
frame = _apply_column_aliases(frame)
|
| 451 |
+
columns = {str(name) for name in frame.columns}
|
| 452 |
+
fps = _detect_fps(root)
|
| 453 |
+
rows = len(frame)
|
| 454 |
+
streams: list[StreamEvidence] = []
|
| 455 |
+
channels: set[str] = {"time"}
|
| 456 |
+
|
| 457 |
+
def _relative(path: Path) -> str:
|
| 458 |
+
try:
|
| 459 |
+
return str(path.relative_to(root))
|
| 460 |
+
except ValueError:
|
| 461 |
+
return str(path)
|
| 462 |
+
|
| 463 |
+
# SLAM —— **每个相机一路**,各取 trajectory 优先(见 _slam_columns)。
|
| 464 |
+
# 这里曾经是 `break`:全局只登记一路,双目集第二路的 SLAM 从来没被采集过。
|
| 465 |
+
for column in _slam_columns(columns):
|
| 466 |
+
values = frame[column].tolist()
|
| 467 |
+
series: dict[str, Any] = {}
|
| 468 |
+
if column.endswith("trajectory"):
|
| 469 |
+
counts = slam_valid_counts(values)
|
| 470 |
+
series["valid_count"] = counts
|
| 471 |
+
series["pose"] = slam_poses(values, counts)
|
| 472 |
+
# 覆盖率必须走生产口径 —— "插值帧"的定义就是 poses_from_trajectory
|
| 473 |
+
# 里本地与相邻格子都够不到的那些,只读原始数据推不出来。
|
| 474 |
+
try:
|
| 475 |
+
from app.umi_slam_action import poses_from_trajectory
|
| 476 |
+
stamps = (frame["timestamp"].to_numpy(dtype=float)
|
| 477 |
+
if "timestamp" in columns
|
| 478 |
+
else np.arange(rows) / fps)
|
| 479 |
+
diag: dict[str, Any] = {}
|
| 480 |
+
poses_from_trajectory(values, stamps, rows, diagnostics=diag)
|
| 481 |
+
series["coverage"] = diag
|
| 482 |
+
except Exception as exc:
|
| 483 |
+
series["coverage_error"] = str(exc)[:200]
|
| 484 |
+
else:
|
| 485 |
+
poses = vector_series(values, 7)
|
| 486 |
+
series["valid_count"] = tuple(
|
| 487 |
+
1 if pose_is_valid(item) else 0 for item in poses)
|
| 488 |
+
series["pose"] = poses
|
| 489 |
+
# 流名取列名最后一段 —— 多相机时是 "gripper_2_slam_trajectory",
|
| 490 |
+
# 单相机时是 "slam_trajectory",两条流不会撞名。
|
| 491 |
+
streams.append(StreamEvidence(
|
| 492 |
+
key=column.rsplit(".", 1)[-1],
|
| 493 |
+
channel="slam", path=_relative(parquet),
|
| 494 |
+
columns=(column,), rows=rows, fps=fps, series=series,
|
| 495 |
+
))
|
| 496 |
+
channels.add("slam")
|
| 497 |
+
|
| 498 |
+
# 其余标量/向量列
|
| 499 |
+
#
|
| 500 |
+
# ★ series 的键一律用【信道名】。检查项按信道取序列(``series["force"]``),
|
| 501 |
+
# 如果这里用列名(``series["gripper_left_force"]``),检查项就静默地取不到
|
| 502 |
+
# 数据、全部判通过 —— 而且不报任何错。左右两路力共用 ``force`` 键,
|
| 503 |
+
# 靠 stream.key 区分。
|
| 504 |
+
for channel, candidates in _COLUMN_CHANNELS.items():
|
| 505 |
+
if channel == "slam":
|
| 506 |
+
continue
|
| 507 |
+
for column in candidates:
|
| 508 |
+
if column not in columns:
|
| 509 |
+
continue
|
| 510 |
+
streams.append(StreamEvidence(
|
| 511 |
+
key=column.rsplit(".", 1)[-1], channel=channel,
|
| 512 |
+
path=_relative(parquet), columns=(column,), rows=rows, fps=fps,
|
| 513 |
+
# 走 _column_series:宽度取不到就**不截断**(兜底成某个数字会
|
| 514 |
+
# 静默丢数据),字符串列原样保留(数值化会变成全空元组)
|
| 515 |
+
series={channel: _column_series(frame[column], channel)},
|
| 516 |
+
))
|
| 517 |
+
channels.add(channel)
|
| 518 |
+
|
| 519 |
+
# 视频 —— ★ 无论检不检查,都要登记成一条流。
|
| 520 |
+
#
|
| 521 |
+
# 只登记不检查时 ``video`` 是空 dict(视频检查项会自行跳过),但信道
|
| 522 |
+
# ``rgb`` 必须出现 —— 否则报告会写成"未采集 rgb",而实际上视频就在那儿,
|
| 523 |
+
# 只是这一轮没去解码。那会让"训练用途"给出错误的结论。
|
| 524 |
+
video_root = root / "videos"
|
| 525 |
+
for path in videos_for_episode(root, episode_index)[:_MAX_VIDEO_PROBES]:
|
| 526 |
+
report: dict[str, Any] = {}
|
| 527 |
+
if probe_video:
|
| 528 |
+
try:
|
| 529 |
+
from app.video_quality import _check_stream
|
| 530 |
+
report = _check_stream(path, rows, fps)
|
| 531 |
+
except Exception as exc:
|
| 532 |
+
report = {"passed": False, "reason": "video_check_failed",
|
| 533 |
+
"error": str(exc)[:200]}
|
| 534 |
+
# 流名 = 流名目录那一层(observation.images.xxx),**不是** chunk-XXX。
|
| 535 |
+
# 两种布局的层级顺序相反,所以不能写死取第几段:
|
| 536 |
+
# canonical/v2.1 videos/<流名>/chunk-000/episode_000000.mp4
|
| 537 |
+
# v3.0 videos/chunk-000/<流名>/file-000.mp4
|
| 538 |
+
# 取"路径里第一个不是 chunk-* 的目录段",两边都成立。
|
| 539 |
+
# 写错的话所有视频流会同名(都叫 chunk-000),报告分不清哪路相机,
|
| 540 |
+
# 且下游按 stream 去重时会互相覆盖。
|
| 541 |
+
try:
|
| 542 |
+
relative = str(path.relative_to(root))
|
| 543 |
+
segments = path.relative_to(video_root).parts[:-1] # 去掉文件名
|
| 544 |
+
key = next((seg for seg in segments
|
| 545 |
+
if not seg.startswith("chunk-")), path.stem)
|
| 546 |
+
except ValueError:
|
| 547 |
+
relative, key = str(path), path.stem
|
| 548 |
+
|
| 549 |
+
# ★ 深度流不能算作 rgb 信道。之前一律按 rgb 登记,结果 D435 项目的
|
| 550 |
+
# 深度视频引出了"视频 9164 帧 / 数据 819 行"这种荒谬的丢帧结论 ——
|
| 551 |
+
# 那是拿深度流跟 rgb 数据行数比。判定复用 lerobot_v21 的既有口径,
|
| 552 |
+
# 不另起一套字符串匹配。
|
| 553 |
+
channel = "depth" if _is_depth_key(key) else "rgb"
|
| 554 |
+
streams.append(StreamEvidence(
|
| 555 |
+
key=key, channel=channel, path=relative,
|
| 556 |
+
rows=rows, fps=fps, video=report,
|
| 557 |
+
))
|
| 558 |
+
channels.add(channel)
|
| 559 |
+
|
| 560 |
+
modalities = _detect_modalities(channels, modality_hint)
|
| 561 |
+
return Evidence(
|
| 562 |
+
batch_dir=root, streams=tuple(streams), channels=frozenset(channels),
|
| 563 |
+
modalities=modalities, fps=fps, rows=rows,
|
| 564 |
+
columns=tuple(sorted(columns)), parquet=_relative(parquet),
|
| 565 |
+
)
|
| 566 |
+
|
| 567 |
+
|
| 568 |
+
def _detect_fps(root: Path) -> float:
|
| 569 |
+
"""从 meta/info.json 取 fps;拿不到就用默认 30。"""
|
| 570 |
+
import json
|
| 571 |
+
|
| 572 |
+
for relative in ("meta/info.json", "info.json"):
|
| 573 |
+
candidate = root / relative
|
| 574 |
+
if not candidate.is_file():
|
| 575 |
+
continue
|
| 576 |
+
try:
|
| 577 |
+
data = json.loads(candidate.read_text(encoding="utf-8"))
|
| 578 |
+
value = float(data.get("fps") or 0)
|
| 579 |
+
if 0 < value < 1000:
|
| 580 |
+
return value
|
| 581 |
+
except Exception:
|
| 582 |
+
continue
|
| 583 |
+
return _DEFAULT_FPS
|
| 584 |
+
|
| 585 |
+
|
| 586 |
+
def hint_values(hint: str | Iterable[str] | None) -> list[str]:
|
| 587 |
+
"""把 hint 归一成字符串列表 —— 调用方有单数(旧调用)也有复数(工作流连线)。"""
|
| 588 |
+
if hint is None:
|
| 589 |
+
return []
|
| 590 |
+
if isinstance(hint, str):
|
| 591 |
+
return [hint] if hint.strip() else []
|
| 592 |
+
try:
|
| 593 |
+
return [str(item) for item in hint]
|
| 594 |
+
except TypeError:
|
| 595 |
+
return [str(hint)]
|
| 596 |
+
|
| 597 |
+
|
| 598 |
+
def _infer_modalities(channels: set[str]) -> list[str]:
|
| 599 |
+
"""只看数据特征能推断出什么。
|
| 600 |
+
|
| 601 |
+
★ 永远给不出 ``stereo_rgb`` / ``stereo_rgbd_camera`` —— "立体"不是列名里
|
| 602 |
+
能看见的特征。这两个模态**只能**由工作流连线给出。所以它们是
|
| 603 |
+
``_detect_modalities`` 取并集、而不是二选一的主要理由。
|
| 604 |
+
"""
|
| 605 |
+
if "slam" in channels or "gripper_state" in channels:
|
| 606 |
+
return ["gripper_device"]
|
| 607 |
+
if "tactile" in channels:
|
| 608 |
+
return ["glove_sensor"]
|
| 609 |
+
if "depth" in channels:
|
| 610 |
+
return ["rgbd_camera"]
|
| 611 |
+
if "rgb" in channels:
|
| 612 |
+
return ["mono_rgb"]
|
| 613 |
+
return []
|
| 614 |
+
|
| 615 |
+
|
| 616 |
+
def _detect_modalities(channels: set[str],
|
| 617 |
+
hint: str | Iterable[str] | None) -> tuple[str, ...]:
|
| 618 |
+
"""这台设备是什么 —— hint(工作流连线)与数据特征取**并集**。
|
| 619 |
+
|
| 620 |
+
两个来源各有各的盲区,缺一个都会让检查静默不跑:
|
| 621 |
+
|
| 622 |
+
只有 hint → 按数据模态声明的检查项不跑(将来接 ``depth.*`` 时,
|
| 623 |
+
D435 数据在但模态被写成 stereo_rgb 就查不到)
|
| 624 |
+
只有推断 → 工作流里那张卡片的 tab 键查不到,**用户在设置面板里配的
|
| 625 |
+
阈值被静默忽略、一律走默认值**(推断不出 stereo_*)
|
| 626 |
+
|
| 627 |
+
并集对"配置查找"和"检查项筛除"都更宽 —— 而本项目反复踩的坑恰恰是
|
| 628 |
+
"检查项静默不跑",所以宁可宽。
|
| 629 |
+
|
| 630 |
+
hint 在前:``engine.resolve_config`` 按这个顺序合并各设备卡片的配置,
|
| 631 |
+
"先出现的优先",工作流连线的顺序才是用户预期的顺序。
|
| 632 |
+
"""
|
| 633 |
+
from app.processing.cleaning.contract import DEVICE_MODALITIES
|
| 634 |
+
|
| 635 |
+
ordered: dict[str, None] = {}
|
| 636 |
+
for raw in hint_values(hint):
|
| 637 |
+
canonical = normalize_device_modality(raw)
|
| 638 |
+
if canonical in DEVICE_MODALITIES:
|
| 639 |
+
ordered.setdefault(canonical, None)
|
| 640 |
+
for modality in _infer_modalities(channels):
|
| 641 |
+
ordered.setdefault(modality, None)
|
| 642 |
+
return tuple(ordered)
|
| 643 |
+
|
| 644 |
+
|
| 645 |
+
def channels_from_modalities(modalities) -> tuple[str, ...]:
|
| 646 |
+
"""设备模态 → 该设备可能提供的信道(转调 contract,供 engine 用)。"""
|
| 647 |
+
return channels_for(modalities)
|
processor/app/processing/cleaning/contract.py
ADDED
|
@@ -0,0 +1,274 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""清洗契约 —— 模态、信道、严重度的公共定义。
|
| 2 |
+
|
| 3 |
+
这个文件把"平台里同一个东西有好几个名字"这件事收敛到一处:
|
| 4 |
+
|
| 5 |
+
device_naming.DEVICE_LABELS mono_rgb / stereo_rgb / ...
|
| 6 |
+
workflow_dispatch.semantic_type 同上(同一套值,只是没人敢这么说)
|
| 7 |
+
api/projects._device_input_sources mono_camera / stereo_camera ← 名字不一样
|
| 8 |
+
处理模块 slug mono_camera / stereo_camera
|
| 9 |
+
|
| 10 |
+
**本模块不新建第 7 套枚举** —— DEVICE_MODALITIES 就是 device_naming 那 6 个值的
|
| 11 |
+
权威声明,其余几套通过 ``normalize_device_modality()`` 单向映射进来。新增设备类型时
|
| 12 |
+
只需改这一处,测试会锁住映射完整性。
|
| 13 |
+
|
| 14 |
+
信道(channel) 是比设备模态更细的粒度: PDF 里"IMU 可选""SLAM 必须"这类判断
|
| 15 |
+
是**信道级**的,硬塞进设备模态会毁掉它。所以设备模态 → 信道的推导放在这里。
|
| 16 |
+
"""
|
| 17 |
+
|
| 18 |
+
from __future__ import annotations
|
| 19 |
+
|
| 20 |
+
from dataclasses import dataclass, field as dc_field
|
| 21 |
+
from typing import Any, Iterable
|
| 22 |
+
|
| 23 |
+
# ── 设备模态 ──────────────────────────────────────────────────
|
| 24 |
+
# 与 app/device_naming.py:15 DEVICE_LABELS 的键**逐字一致**。
|
| 25 |
+
# 改这里必须同步改那里,tests/test_cleaning_contract.py 会断言两边相等。
|
| 26 |
+
DEVICE_MODALITIES: tuple[str, ...] = (
|
| 27 |
+
"mono_rgb",
|
| 28 |
+
"stereo_rgb",
|
| 29 |
+
"rgbd_camera",
|
| 30 |
+
"stereo_rgbd_camera",
|
| 31 |
+
"glove_sensor",
|
| 32 |
+
"gripper_device",
|
| 33 |
+
)
|
| 34 |
+
|
| 35 |
+
# 「摄像机类」模态 —— 判定 collection_type 时要用
|
| 36 |
+
CAMERA_MODALITIES: frozenset[str] = frozenset({
|
| 37 |
+
"mono_rgb", "stereo_rgb", "rgbd_camera", "stereo_rgbd_camera",
|
| 38 |
+
})
|
| 39 |
+
|
| 40 |
+
# 会产出 RGB 视频的**全部**设备卡片 —— ``video.*`` 类检查适用于它们每一个。
|
| 41 |
+
# 比 CAMERA_MODALITIES 多一个 gripper_device:UMI 夹爪自带鱼眼相机,
|
| 42 |
+
# 视频检查对同样它成立。
|
| 43 |
+
VIDEO_CAPABLE_MODALITIES: tuple[str, ...] = (
|
| 44 |
+
"mono_rgb", "stereo_rgb", "rgbd_camera", "stereo_rgbd_camera",
|
| 45 |
+
"gripper_device",
|
| 46 |
+
)
|
| 47 |
+
|
| 48 |
+
# ── 信道 ──────────────────────────────────────────────────────
|
| 49 |
+
# 检查项按信道声明需求,而不是按设备模态 —— 这样一条"SLAM 连续性"检查
|
| 50 |
+
# 既能被 UMI 夹爪触发,将来也能被别的带 SLAM 的设备复用。
|
| 51 |
+
CHANNELS: tuple[str, ...] = (
|
| 52 |
+
"time", # 时间戳 / 帧号
|
| 53 |
+
"rgb", # RGB 视频
|
| 54 |
+
"depth", # 深度
|
| 55 |
+
"slam", # SLAM 位姿/轨迹
|
| 56 |
+
"gripper_state", # 夹爪开合
|
| 57 |
+
"force", # 力 / 触觉(夹爪力)
|
| 58 |
+
"tactile", # 触觉阵列(手套)
|
| 59 |
+
"imu", # 惯性单元
|
| 60 |
+
"keypoints", # 手部关键点
|
| 61 |
+
"action", # 动作(派生列)
|
| 62 |
+
# 设备的连接状态(``status.left_glove`` 这类**字符串**列)。
|
| 63 |
+
# 单独算一个信道而不是塞进 tactile 的 series:它是"设备这一帧在不在",
|
| 64 |
+
# 与阵列读数正交 —— 手套掉线时阵列会静默保持最后一个值,看起来完全正常。
|
| 65 |
+
"device_status",
|
| 66 |
+
)
|
| 67 |
+
|
| 68 |
+
# 设备模态 → 该设备**可能**提供的信道。
|
| 69 |
+
# 注意这是"可能"不是"必然" —— 实际有哪些信道由 collect.py 从真实列名推导,
|
| 70 |
+
# 这里只用来回答"这个模态需不需要某类检查"。
|
| 71 |
+
MODALITY_CHANNELS: dict[str, tuple[str, ...]] = {
|
| 72 |
+
"mono_rgb": ("rgb", "time"),
|
| 73 |
+
"stereo_rgb": ("rgb", "time"),
|
| 74 |
+
"rgbd_camera": ("rgb", "depth", "time"),
|
| 75 |
+
"stereo_rgbd_camera": ("rgb", "depth", "time"),
|
| 76 |
+
"glove_sensor": ("tactile", "device_status", "imu", "time"),
|
| 77 |
+
"gripper_device": ("rgb", "slam", "gripper_state", "force", "action", "time"),
|
| 78 |
+
}
|
| 79 |
+
|
| 80 |
+
# 非设备来源的信道 —— 由处理节点产出,不属于任何输入设备
|
| 81 |
+
DERIVED_CHANNELS: frozenset[str] = frozenset({"keypoints", "action"})
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def channels_for(modalities: Iterable[str]) -> tuple[str, ...]:
|
| 85 |
+
"""一组设备模态覆盖的全部信道(去重、保序)。纯函数。"""
|
| 86 |
+
seen: dict[str, None] = {}
|
| 87 |
+
for modality in modalities:
|
| 88 |
+
for channel in MODALITY_CHANNELS.get(str(modality), ()):
|
| 89 |
+
seen.setdefault(channel, None)
|
| 90 |
+
return tuple(seen)
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
def pose_is_valid(item) -> bool:
|
| 94 |
+
"""位姿能不能用 —— 空元组和**全零占位**都不算。
|
| 95 |
+
|
| 96 |
+
★ 全零占位长得像合法位姿:长度 7、能索引、能相减。但它的四元数模是 0,
|
| 97 |
+
任何涉及它的差分都会算出"瞬移":
|
| 98 |
+
|
| 99 |
+
两个零四元数的点积 = 0 → acos(0) = π/2 → 夹角 180°
|
| 100 |
+
|
| 101 |
+
30fps 下就是 180°×30 = **94.25 rad/s** —— 一个物理上荒谬的角速度。
|
| 102 |
+
实测 Test94 的 ``observation.slam_pose`` 有 437/599 行是全零,SLAM 连续性
|
| 103 |
+
因此报出 465 处"跳变"(599 帧里几乎每一帧),把整集假判成 FAIL 并拦住批次。
|
| 104 |
+
|
| 105 |
+
判据只看四元数模:**位置可以合法地是 (0,0,0)**(原点就是合法位姿),
|
| 106 |
+
四元数模为 0 则一定不是合法朝向。
|
| 107 |
+
|
| 108 |
+
放在 contract 而不是 collect:检查项要用它,而 collect 依赖 checks
|
| 109 |
+
(检查项自动发现),collect ← checks 的导入会成环。
|
| 110 |
+
"""
|
| 111 |
+
if not item or len(item) < 7:
|
| 112 |
+
return False
|
| 113 |
+
try:
|
| 114 |
+
return sum(float(value) ** 2 for value in item[3:7]) > 1e-12
|
| 115 |
+
except (TypeError, ValueError):
|
| 116 |
+
return False
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
def normalize_device_modality(value: object) -> str | None:
|
| 120 |
+
"""把别处的设备/输入类型名映射到 canonical 设备模态。
|
| 121 |
+
|
| 122 |
+
覆盖两套现存命名:
|
| 123 |
+
* ``stereo_camera`` / ``mono_camera``(api/projects、processing 模块 slug)
|
| 124 |
+
* ``fisheye_camera`` / ``rgb_camera``(历史工作流节点类型)
|
| 125 |
+
认不出来返回 ``None``(调用方应跳过该值,而不是终止)。
|
| 126 |
+
"""
|
| 127 |
+
text = str(value or "").strip()
|
| 128 |
+
if not text:
|
| 129 |
+
return None
|
| 130 |
+
if text in DEVICE_MODALITIES:
|
| 131 |
+
return text
|
| 132 |
+
return _ALIASES.get(text)
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
# 别名表 —— 只列真实存在于代码里的历史/并行命名。
|
| 136 |
+
# 新增别名时同步加进 tests/test_cleaning_contract.py 的用例。
|
| 137 |
+
_ALIASES: dict[str, str] = {
|
| 138 |
+
# api/projects._device_input_sources 与处理模块 slug 用的是 camera 后缀
|
| 139 |
+
"mono_camera": "mono_rgb",
|
| 140 |
+
"stereo_camera": "stereo_rgb",
|
| 141 |
+
"fisheye_camera": "mono_rgb", # 历史节点,镜头类型对清洗不可见
|
| 142 |
+
"rgb_camera": "mono_rgb", # 历史节点
|
| 143 |
+
"rgbd": "rgbd_camera",
|
| 144 |
+
"stereo_rgbd": "stereo_rgbd_camera",
|
| 145 |
+
# 中文/展示名兜底
|
| 146 |
+
"umi": "gripper_device",
|
| 147 |
+
"umi_gripper": "gripper_device",
|
| 148 |
+
"glove": "glove_sensor",
|
| 149 |
+
}
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
# ── 严重度 ────────────────────────────────────────────────────
|
| 153 |
+
PASS = "pass"
|
| 154 |
+
WARN = "warn"
|
| 155 |
+
FAIL = "fail"
|
| 156 |
+
PENDING = "pending_review"
|
| 157 |
+
ERROR = "error"
|
| 158 |
+
|
| 159 |
+
# 越靠前越严重 —— worst() 与排序都用它
|
| 160 |
+
SEVERITY_ORDER: tuple[str, ...] = (ERROR, FAIL, PENDING, WARN, PASS)
|
| 161 |
+
|
| 162 |
+
# 面向用户的展示文案(报告里的 ``status_label``)。界面是英文的,这里保持一致。
|
| 163 |
+
SEVERITY_LABELS: dict[str, str] = {
|
| 164 |
+
PASS: "Pass",
|
| 165 |
+
WARN: "Warning",
|
| 166 |
+
FAIL: "Fail",
|
| 167 |
+
PENDING: "Pending review",
|
| 168 |
+
ERROR: "Error",
|
| 169 |
+
}
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
def worst(statuses: Iterable[str]) -> str:
|
| 173 |
+
"""取一组状态里最严重的;空集合视为 PASS。纯函数。"""
|
| 174 |
+
rank = {status: index for index, status in enumerate(SEVERITY_ORDER)}
|
| 175 |
+
result = PASS
|
| 176 |
+
best = len(SEVERITY_ORDER)
|
| 177 |
+
for status in statuses:
|
| 178 |
+
index = rank.get(str(status), len(SEVERITY_ORDER))
|
| 179 |
+
if index < best:
|
| 180 |
+
best = index
|
| 181 |
+
result = str(status)
|
| 182 |
+
return result
|
| 183 |
+
|
| 184 |
+
|
| 185 |
+
def is_blocking(status: str) -> bool:
|
| 186 |
+
"""该状态是否应当拦截(退回人工审核)。"""
|
| 187 |
+
return str(status) in (FAIL, ERROR)
|
| 188 |
+
|
| 189 |
+
|
| 190 |
+
# ── 结构 ──────────────────────────────────────────────────────
|
| 191 |
+
@dataclass(frozen=True)
|
| 192 |
+
class Range:
|
| 193 |
+
"""一段时间区间 —— PDF §5「第二层结果」。"""
|
| 194 |
+
|
| 195 |
+
start_sec: float
|
| 196 |
+
end_sec: float
|
| 197 |
+
code: str
|
| 198 |
+
severity: str
|
| 199 |
+
stream: str = ""
|
| 200 |
+
channel: str = ""
|
| 201 |
+
start_frame: int | None = None
|
| 202 |
+
end_frame: int | None = None
|
| 203 |
+
rule: str = ""
|
| 204 |
+
message: str = ""
|
| 205 |
+
evidence: dict[str, Any] = dc_field(default_factory=dict)
|
| 206 |
+
|
| 207 |
+
def to_dict(self) -> dict[str, Any]:
|
| 208 |
+
return {
|
| 209 |
+
"start_sec": round(float(self.start_sec), 3),
|
| 210 |
+
"end_sec": round(float(self.end_sec), 3),
|
| 211 |
+
"code": self.code,
|
| 212 |
+
"severity": self.severity,
|
| 213 |
+
"stream": self.stream,
|
| 214 |
+
"channel": self.channel,
|
| 215 |
+
"start_frame": self.start_frame,
|
| 216 |
+
"end_frame": self.end_frame,
|
| 217 |
+
"rule": self.rule,
|
| 218 |
+
"message": self.message,
|
| 219 |
+
"evidence": dict(self.evidence or {}),
|
| 220 |
+
}
|
| 221 |
+
|
| 222 |
+
|
| 223 |
+
@dataclass(frozen=True)
|
| 224 |
+
class StreamResult:
|
| 225 |
+
"""一条数据流的检查结果 —— PDF §5「第一层结果」。"""
|
| 226 |
+
|
| 227 |
+
stream: str
|
| 228 |
+
modality: str = ""
|
| 229 |
+
label: str = ""
|
| 230 |
+
channels: tuple[str, ...] = ()
|
| 231 |
+
status: str = PASS
|
| 232 |
+
reasons: tuple[str, ...] = ()
|
| 233 |
+
metrics: dict[str, Any] = dc_field(default_factory=dict)
|
| 234 |
+
ranges: tuple[Range, ...] = ()
|
| 235 |
+
|
| 236 |
+
def to_dict(self) -> dict[str, Any]:
|
| 237 |
+
return {
|
| 238 |
+
"stream": self.stream,
|
| 239 |
+
"modality": self.modality,
|
| 240 |
+
"label": self.label,
|
| 241 |
+
"channels": list(self.channels),
|
| 242 |
+
"status": self.status,
|
| 243 |
+
"reasons": list(self.reasons),
|
| 244 |
+
"metrics": dict(self.metrics or {}),
|
| 245 |
+
"ranges": [item.to_dict() for item in self.ranges],
|
| 246 |
+
}
|
| 247 |
+
|
| 248 |
+
|
| 249 |
+
@dataclass(frozen=True)
|
| 250 |
+
class Finding:
|
| 251 |
+
"""一次检查产出的��论 —— 检查项 run() 的返回值单元。"""
|
| 252 |
+
|
| 253 |
+
rule: str
|
| 254 |
+
version: str
|
| 255 |
+
status: str
|
| 256 |
+
scope: str # stream | episode | cross_modal
|
| 257 |
+
stream: str = ""
|
| 258 |
+
message: str = ""
|
| 259 |
+
metrics: dict[str, Any] = dc_field(default_factory=dict)
|
| 260 |
+
ranges: tuple[Range, ...] = ()
|
| 261 |
+
elapsed_ms: int = 0
|
| 262 |
+
|
| 263 |
+
def to_dict(self) -> dict[str, Any]:
|
| 264 |
+
return {
|
| 265 |
+
"rule": self.rule,
|
| 266 |
+
"version": self.version,
|
| 267 |
+
"status": self.status,
|
| 268 |
+
"scope": self.scope,
|
| 269 |
+
"stream": self.stream,
|
| 270 |
+
"message": self.message,
|
| 271 |
+
"metrics": dict(self.metrics or {}),
|
| 272 |
+
"ranges": [item.to_dict() for item in self.ranges],
|
| 273 |
+
"elapsed_ms": self.elapsed_ms,
|
| 274 |
+
}
|
processor/app/processing/cleaning/engine.py
ADDED
|
@@ -0,0 +1,302 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""质检引擎 —— 把检查项串成一次运行,产出四层报告。
|
| 2 |
+
|
| 3 |
+
流程::
|
| 4 |
+
|
| 5 |
+
collect.gather_evidence() 读批次目录 → Evidence
|
| 6 |
+
│
|
| 7 |
+
▼
|
| 8 |
+
applicable_checks() 按实际信道/模态选检查项
|
| 9 |
+
│
|
| 10 |
+
▼
|
| 11 |
+
逐项 run(ctx) 产出 Finding
|
| 12 |
+
│
|
| 13 |
+
▼
|
| 14 |
+
rollup 成 StreamResult 每条流一个结论
|
| 15 |
+
│
|
| 16 |
+
▼
|
| 17 |
+
report.build_report() 四层结果
|
| 18 |
+
|
| 19 |
+
**只出结论,不产出数据**:不写文件、不改 episode 状态、不做任何清理动作。
|
| 20 |
+
报告由调用方决定怎么用(落盘 / 展示 / 过滤导出)。
|
| 21 |
+
|
| 22 |
+
设计取舍
|
| 23 |
+
--------
|
| 24 |
+
* **检查项失败不拖垮整轮** —— 单项抛异常记成 ``ERROR`` 结论继续跑,
|
| 25 |
+
否则一个 bug 会让整集没有报告。
|
| 26 |
+
* **不设 `on_failure`** —— 按用户要求简化:清洗只记录,不碰 episode 状态流转。
|
| 27 |
+
需要拦截时由调用方读报告的 ``passed`` / ``verdict`` 自行决定。
|
| 28 |
+
"""
|
| 29 |
+
|
| 30 |
+
from __future__ import annotations
|
| 31 |
+
|
| 32 |
+
import hashlib
|
| 33 |
+
import time
|
| 34 |
+
from pathlib import Path
|
| 35 |
+
from typing import Any, Iterable
|
| 36 |
+
|
| 37 |
+
from app.processing.cleaning import collect
|
| 38 |
+
from app.processing.cleaning.checks import (
|
| 39 |
+
CheckContext, applicable_checks, all_checks,
|
| 40 |
+
)
|
| 41 |
+
from app.processing.cleaning.contract import (
|
| 42 |
+
ERROR, PASS, SEVERITY_LABELS, StreamResult, worst,
|
| 43 |
+
)
|
| 44 |
+
from app.processing.cleaning.report import build_report, empty_report
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def ruleset_revision() -> str:
|
| 48 |
+
"""规则集指纹 —— 任何检查项的 slug/version 变了它就变。
|
| 49 |
+
|
| 50 |
+
用途:报告里带上这个值,规则更新后能识别出"这份报告是用旧规则跑的"。
|
| 51 |
+
与 ``workflow_dispatch.workflow_revision()`` 同款思路 —— 用内容哈希而不是
|
| 52 |
+
自增版本号,省掉版本分配和迁移。
|
| 53 |
+
"""
|
| 54 |
+
parts = sorted(f"{check.slug}@{check.version}" for check in all_checks())
|
| 55 |
+
return hashlib.sha256("|".join(parts).encode("utf-8")).hexdigest()[:24]
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def resolve_config(evidence: collect.Evidence,
|
| 59 |
+
config: dict | None) -> dict[str, dict]:
|
| 60 |
+
"""把节点配置解析成 ``{检查项 slug: {"enabled": bool, "params": {...}}}``。
|
| 61 |
+
|
| 62 |
+
节点配置结构(见 data_quality 节点)::
|
| 63 |
+
|
| 64 |
+
{
|
| 65 |
+
"devices": {
|
| 66 |
+
"gripper_device": {
|
| 67 |
+
"checks": {
|
| 68 |
+
"umi.slam_continuity": {"enabled": true,
|
| 69 |
+
"params": {"max_linear_mps": 3.0}},
|
| 70 |
+
"video.freeze": {"enabled": false}
|
| 71 |
+
}
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
本 episode 有多个设备卡片时,按 ``evidence.modalities`` 的顺序合并;
|
| 77 |
+
**先出现的优先**(后面的不覆盖前面已设的值)。实际采集里一个 episode
|
| 78 |
+
基本只对应一张卡片,多卡片是罕见情况,规则简单可预期即可。
|
| 79 |
+
|
| 80 |
+
没配到的检查项走默认:启用、用检查项自己声明的 ``default_params``。
|
| 81 |
+
"""
|
| 82 |
+
resolved: dict[str, dict] = {}
|
| 83 |
+
devices = (config or {}).get("devices") or {}
|
| 84 |
+
for modality in evidence.modalities:
|
| 85 |
+
card = devices.get(str(modality)) or {}
|
| 86 |
+
for slug, item in (card.get("checks") or {}).items():
|
| 87 |
+
if slug in resolved:
|
| 88 |
+
continue # 先出现的优先
|
| 89 |
+
entry = item if isinstance(item, dict) else {}
|
| 90 |
+
resolved[slug] = {
|
| 91 |
+
"enabled": bool(entry.get("enabled", True)),
|
| 92 |
+
"params": dict(entry.get("params") or {}),
|
| 93 |
+
}
|
| 94 |
+
return resolved
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def _context_for(evidence: collect.Evidence, cfg: dict[str, dict]) -> CheckContext:
|
| 98 |
+
"""按证据构造检查上下文。阈值按"当前检查项"逐个注入。"""
|
| 99 |
+
return CheckContext(
|
| 100 |
+
batch_dir=evidence.batch_dir,
|
| 101 |
+
channels=evidence.channels,
|
| 102 |
+
modalities=evidence.modalities,
|
| 103 |
+
data_rows=evidence.rows,
|
| 104 |
+
data_columns=evidence.columns,
|
| 105 |
+
duration_sec=evidence.duration_sec,
|
| 106 |
+
fps=evidence.fps,
|
| 107 |
+
streams=evidence.streams,
|
| 108 |
+
)
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
def _stream_results(evidence: collect.Evidence,
|
| 112 |
+
findings: Iterable) -> list[StreamResult]:
|
| 113 |
+
"""把 findings 归拢成"每条流一个结论" —— 报告的第一层。"""
|
| 114 |
+
by_stream: dict[str, list] = {}
|
| 115 |
+
for item in findings:
|
| 116 |
+
by_stream.setdefault(item.stream or "", []).append(item)
|
| 117 |
+
|
| 118 |
+
results: list[StreamResult] = []
|
| 119 |
+
seen: set[str] = set()
|
| 120 |
+
for stream in evidence.streams:
|
| 121 |
+
items = by_stream.get(stream.key) or []
|
| 122 |
+
seen.add(stream.key)
|
| 123 |
+
status = worst(item.status for item in items) if items else PASS
|
| 124 |
+
results.append(StreamResult(
|
| 125 |
+
stream=stream.key,
|
| 126 |
+
modality=",".join(evidence.modalities),
|
| 127 |
+
label=stream.key,
|
| 128 |
+
channels=(stream.channel,) if stream.channel else (),
|
| 129 |
+
status=status,
|
| 130 |
+
reasons=tuple(item.rule for item in items if item.status != PASS),
|
| 131 |
+
metrics={k: v for item in items
|
| 132 |
+
for k, v in (item.metrics or {}).items()
|
| 133 |
+
if isinstance(v, (int, float, str, bool))},
|
| 134 |
+
ranges=tuple(r for item in items for r in (item.ranges or ())),
|
| 135 |
+
))
|
| 136 |
+
|
| 137 |
+
# 没有对应证据流的 finding(如整集级结论)也要留在第一层里
|
| 138 |
+
for key, items in by_stream.items():
|
| 139 |
+
if key in seen:
|
| 140 |
+
continue
|
| 141 |
+
results.append(StreamResult(
|
| 142 |
+
stream=key or "episode",
|
| 143 |
+
status=worst(item.status for item in items),
|
| 144 |
+
reasons=tuple(item.rule for item in items if item.status != PASS),
|
| 145 |
+
ranges=tuple(r for item in items for r in (item.ranges or ())),
|
| 146 |
+
))
|
| 147 |
+
return results
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
def run_checks(batch_dir: Path, *,
|
| 151 |
+
episode_id: str = "",
|
| 152 |
+
episode_index: int | None = None,
|
| 153 |
+
project: str = "",
|
| 154 |
+
config: dict | None = None,
|
| 155 |
+
probe_video: bool = False,
|
| 156 |
+
modality_hint: str | Iterable[str] | None = None) -> dict:
|
| 157 |
+
"""跑一个 episode 的全部适用检查,返回四层报告(纯 dict,不含副作用)。
|
| 158 |
+
|
| 159 |
+
``modality_hint`` 是设备模态(可多个,来自工作流连线的设备卡片)。给了它,
|
| 160 |
+
``resolve_config`` 才能查到用户在设置面板里按设备卡片配的阈值;给不出就
|
| 161 |
+
走数据推断(推断不出 ``stereo_*``,那些 tab 的配置会被静默忽略)。
|
| 162 |
+
|
| 163 |
+
``episode_index`` 只在【一个文件装多集】的布局下需要(LeRobot v3.0 导出
|
| 164 |
+
产物就是这种);canonical 布局一集一文件,留空即可。**传错会导致把整份
|
| 165 |
+
文件当成一集读**,所有指标因此失真 —— 而 ``localstore.get_episode()``
|
| 166 |
+
返回的 ``path`` 是**项目目录**(多集共享),所以工作流路径**必须**带上它。
|
| 167 |
+
|
| 168 |
+
``probe_video=False``(默认)只读 parquet,毫秒级;视频检查要解码抽样,
|
| 169 |
+
每集约几秒到几十秒,按需开。
|
| 170 |
+
"""
|
| 171 |
+
started = time.perf_counter()
|
| 172 |
+
evidence = collect.gather_evidence(
|
| 173 |
+
batch_dir, episode_index=episode_index,
|
| 174 |
+
probe_video=probe_video, modality_hint=modality_hint)
|
| 175 |
+
|
| 176 |
+
if not evidence.streams:
|
| 177 |
+
return empty_report(
|
| 178 |
+
episode_id=episode_id, project=project,
|
| 179 |
+
reason=str(evidence.notes.get("reason") or "no_data"))
|
| 180 |
+
|
| 181 |
+
resolved = resolve_config(evidence, config)
|
| 182 |
+
ctx = _context_for(evidence, resolved)
|
| 183 |
+
|
| 184 |
+
# ★ 配置是【黑名单】,**不是白名单** —— 不传 ``enabled``。
|
| 185 |
+
#
|
| 186 |
+
# ``applicable_checks`` 的 ``enabled`` 是白名单语义(``slug not in allow``
|
| 187 |
+
# 就跳过)。而前端设置面板只在用户**碰过**的检查项上写 entry(见
|
| 188 |
+
# DeviceQualityModal 的 setEntry,未触碰的项不落进 config),所以
|
| 189 |
+
# ``resolved`` 里通常只有一两条。把它当白名单 = 「用户在弹窗里改了一个
|
| 190 |
+
# 阈值 → 其余检查项全部静默停跑」:实测只配 umi.gripper_range 时,
|
| 191 |
+
# 9 项变 1 项,且不报任何错。
|
| 192 |
+
#
|
| 193 |
+
# 检查项没有"默认关闭"的概念(CleaningCheck 无 default_enabled),
|
| 194 |
+
# 适用即跑;配置唯一的语义就是**关掉某项**。
|
| 195 |
+
#
|
| 196 |
+
# (曾经写成 `{...enabled} or None`:那个 ``or None`` 只修了空集那半边
|
| 197 |
+
# ——"空集=一个都不跑"——非空的白名单照样是错的。整条白名单都不该有。)
|
| 198 |
+
deny = {slug for slug, item in resolved.items() if not item["enabled"]} or None
|
| 199 |
+
|
| 200 |
+
checks = applicable_checks(
|
| 201 |
+
channels=evidence.channels,
|
| 202 |
+
modalities=evidence.modalities,
|
| 203 |
+
cross_modal_active=len(set(evidence.modalities)) >= 2
|
| 204 |
+
or len({item.channel for item in evidence.streams}) >= 2,
|
| 205 |
+
disabled=deny,
|
| 206 |
+
)
|
| 207 |
+
|
| 208 |
+
findings: list = []
|
| 209 |
+
failed_checks: list[str] = []
|
| 210 |
+
for check in checks:
|
| 211 |
+
entry = resolved.get(check.slug) or {}
|
| 212 |
+
ctx.params = dict(entry.get("params") or {})
|
| 213 |
+
try:
|
| 214 |
+
findings.extend(check.run(ctx) or [])
|
| 215 |
+
except Exception as exc:
|
| 216 |
+
failed_checks.append(f"{check.slug}: {exc}")
|
| 217 |
+
findings.append(_error_finding(check, exc))
|
| 218 |
+
|
| 219 |
+
# 视频流登记了却没检查 —— 必须显式记一笔,但**不能混进 findings**。
|
| 220 |
+
#
|
| 221 |
+
# 视频检查项"适用"(rgb 信道存在)不等于"查过":``probe_video=False`` 时
|
| 222 |
+
# 它们拿到的是空报告,内部直接 continue,一条结论都不产出。报告若就此收尾,
|
| 223 |
+
# 会得出 ``passed=True 全部通过`` —— 而实际上视频根本没看。
|
| 224 |
+
#
|
| 225 |
+
# ★ 但它不该混进 findings 的严重度排序:``PENDING`` 排在 ``WARN`` 之前,
|
| 226 |
+
# 一条"视频未检查"会把真实的力漂移警告盖掉(实测 52 集全被标成"待人工
|
| 227 |
+
# 确认")。"覆盖度"和"数据有多坏"是两个维度 —— 分开报。
|
| 228 |
+
unchecked: list[dict] = []
|
| 229 |
+
if not probe_video:
|
| 230 |
+
rgb_streams = [item for item in evidence.streams if item.channel == "rgb"]
|
| 231 |
+
if rgb_streams:
|
| 232 |
+
unchecked.append({
|
| 233 |
+
"channel": "rgb",
|
| 234 |
+
"reason": "probe_video_disabled",
|
| 235 |
+
"message": f"{len(rgb_streams)} 路视频未检查(本轮只读了 parquet)",
|
| 236 |
+
"streams": [item.key for item in rgb_streams],
|
| 237 |
+
})
|
| 238 |
+
|
| 239 |
+
elapsed_ms = int((time.perf_counter() - started) * 1000)
|
| 240 |
+
report = build_report(
|
| 241 |
+
_stream_results(evidence, findings),
|
| 242 |
+
findings,
|
| 243 |
+
episode_id=episode_id,
|
| 244 |
+
project=project,
|
| 245 |
+
modalities=_modality_summary(evidence, [c.slug for c in checks]),
|
| 246 |
+
channels=evidence.channels,
|
| 247 |
+
ruleset_revision=ruleset_revision(),
|
| 248 |
+
template_source=modality_hint or "",
|
| 249 |
+
elapsed_ms=elapsed_ms,
|
| 250 |
+
)
|
| 251 |
+
|
| 252 |
+
report["unchecked"] = unchecked
|
| 253 |
+
|
| 254 |
+
if not checks:
|
| 255 |
+
# 一个检查项都没跑 —— 报告在说"没问题",而实际是"没查"。
|
| 256 |
+
# 与 ``empty_report`` 同一条原则:没有结论不等于结论合格(fail closed)。
|
| 257 |
+
report["no_checks_ran"] = True
|
| 258 |
+
|
| 259 |
+
# ``episode`` 只反映【数据本身】的结论;``passed`` 额外要求没有未检查项。
|
| 260 |
+
#
|
| 261 |
+
# 分开的理由:前者是"数据有多坏",后者是"我们看全了没有"。混在一起会让
|
| 262 |
+
# "视频没检查"盖过"力漂移",也会让"数据确实没问题但没看全"看起来像数据坏了。
|
| 263 |
+
if unchecked or not checks:
|
| 264 |
+
report["passed"] = False
|
| 265 |
+
|
| 266 |
+
if failed_checks:
|
| 267 |
+
# 检查项本身出错不改判定 —— 但要让人看见,否则会误以为"这集没问题"
|
| 268 |
+
report["check_errors"] = failed_checks
|
| 269 |
+
return report
|
| 270 |
+
|
| 271 |
+
|
| 272 |
+
def _modality_summary(evidence: collect.Evidence,
|
| 273 |
+
checks_run: Iterable[str]) -> dict:
|
| 274 |
+
"""PDF §1 的「数据清单」—— 这台设备是什么、有哪些信道、**实际跑了**哪些检查。
|
| 275 |
+
|
| 276 |
+
``checks_run`` 必须是【真正执行了】的检查项(``applicable_checks`` 的结果),
|
| 277 |
+
不是配置里写的那份。两者可能差很远:设备卡片闸门会把不属于本设备的检查项
|
| 278 |
+
滤掉,而报告如果只报配置,就分不清"检查了没问题"和"压根没检查"。
|
| 279 |
+
"""
|
| 280 |
+
return {
|
| 281 |
+
"device_cards": list(evidence.modalities),
|
| 282 |
+
"channels": sorted(evidence.channels),
|
| 283 |
+
"rows": evidence.rows,
|
| 284 |
+
"fps": evidence.fps,
|
| 285 |
+
"parquet": evidence.parquet,
|
| 286 |
+
"checks_run": sorted(checks_run),
|
| 287 |
+
}
|
| 288 |
+
|
| 289 |
+
|
| 290 |
+
def _error_finding(check, exc: Exception):
|
| 291 |
+
"""检查项抛异常时的占位结论。
|
| 292 |
+
|
| 293 |
+
用 ``ERROR``(处理失败)而不是 ``FAIL`` —— 这是"没检查成",
|
| 294 |
+
不是"数据坏了"。混为一谈会让数据质量问题看起来比实际多。
|
| 295 |
+
"""
|
| 296 |
+
from app.processing.cleaning.contract import Finding
|
| 297 |
+
|
| 298 |
+
return Finding(
|
| 299 |
+
rule=check.slug, version=check.version, status=ERROR,
|
| 300 |
+
scope="episode",
|
| 301 |
+
message=f"check failed to run: {type(exc).__name__}: {exc}",
|
| 302 |
+
)
|
processor/app/processing/cleaning/report.py
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""清洗报告组装 —— PDF §5 的「四层结果」。
|
| 2 |
+
|
| 3 |
+
第一层 数据流 哪条流好、哪条流坏
|
| 4 |
+
第二层 时间区间 问题发生在哪几秒、哪些帧
|
| 5 |
+
第三层 episode 这一集整体什么结论
|
| 6 |
+
第四层 训练用途 能用于什么训练、不能用于什么
|
| 7 |
+
|
| 8 |
+
全部是纯函数 —— 输入是检查项产出的 ``StreamResult`` / ``Finding``,
|
| 9 |
+
输出是可 JSON 序列化的 dict。不碰磁盘、不碰 cv2,因此完全可单测。
|
| 10 |
+
"""
|
| 11 |
+
|
| 12 |
+
from __future__ import annotations
|
| 13 |
+
|
| 14 |
+
from datetime import datetime, timezone
|
| 15 |
+
from typing import Any, Iterable
|
| 16 |
+
|
| 17 |
+
from app.processing.cleaning.contract import (
|
| 18 |
+
Finding, Range, StreamResult,
|
| 19 |
+
PASS, WARN, FAIL, PENDING, ERROR,
|
| 20 |
+
SEVERITY_ORDER, SEVERITY_LABELS, worst, is_blocking,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
SCHEMA_VERSION = 2
|
| 24 |
+
|
| 25 |
+
# 「训练用途」判定表 —— 声明式,不是 if-else 堆出来的。
|
| 26 |
+
# requires 里的信道必须全部具备,该用途才成立。
|
| 27 |
+
# PDF §7 要求"说明 episode 适合纯视觉、触觉或联合训练",这张表就是那个说明。
|
| 28 |
+
TRAINING_USE_RULES: tuple[dict[str, Any], ...] = (
|
| 29 |
+
{"use": "pure_vision", "label": "Pure vision",
|
| 30 |
+
"requires": ("rgb", "time")},
|
| 31 |
+
{"use": "vision_gripper_joint", "label": "Vision + gripper",
|
| 32 |
+
"requires": ("rgb", "gripper_state", "time")},
|
| 33 |
+
{"use": "slam_trajectory", "label": "SLAM trajectory",
|
| 34 |
+
"requires": ("slam", "time")},
|
| 35 |
+
{"use": "vision_tactile_joint", "label": "Vision + tactile",
|
| 36 |
+
"requires": ("rgb", "tactile", "time")},
|
| 37 |
+
{"use": "bimanual_tactile", "label": "Bimanual tactile",
|
| 38 |
+
"requires": ("tactile", "time")},
|
| 39 |
+
)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def _utcnow() -> str:
|
| 43 |
+
return datetime.now(timezone.utc).isoformat()
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def build_timeline(streams: Iterable[StreamResult | dict]) -> list[dict]:
|
| 47 |
+
"""把各条流里的区间拍平成一条时间轴,按时间排序去重。
|
| 48 |
+
|
| 49 |
+
详情页要一次拉完整条时间轴,而 streams 是嵌套的 —— 所以顶层冗余一份。
|
| 50 |
+
单一来源、单一方向(从 streams 派生),有测试保证一致。
|
| 51 |
+
"""
|
| 52 |
+
items: list[dict] = []
|
| 53 |
+
for stream in streams:
|
| 54 |
+
if isinstance(stream, StreamResult):
|
| 55 |
+
for item in stream.ranges:
|
| 56 |
+
items.append(_range_row(item.to_dict(), stream.stream))
|
| 57 |
+
elif isinstance(stream, dict):
|
| 58 |
+
for item in stream.get("ranges") or []:
|
| 59 |
+
if isinstance(item, dict):
|
| 60 |
+
items.append(_range_row(item, str(stream.get("stream") or "")))
|
| 61 |
+
items.sort(key=lambda row: (row["start_sec"], row["end_sec"], row["code"]))
|
| 62 |
+
deduped: list[dict] = []
|
| 63 |
+
for row in items:
|
| 64 |
+
if deduped and all(
|
| 65 |
+
row[key] == deduped[-1][key]
|
| 66 |
+
for key in ("start_sec", "end_sec", "code", "stream")
|
| 67 |
+
):
|
| 68 |
+
continue
|
| 69 |
+
deduped.append(row)
|
| 70 |
+
return deduped
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def _range_row(item: dict, stream: str) -> dict:
|
| 74 |
+
return {
|
| 75 |
+
"stream": item.get("stream") or stream,
|
| 76 |
+
"channel": item.get("channel") or "",
|
| 77 |
+
"start_sec": float(item.get("start_sec") or 0.0),
|
| 78 |
+
"end_sec": float(item.get("end_sec") or 0.0),
|
| 79 |
+
"start_frame": item.get("start_frame"),
|
| 80 |
+
"end_frame": item.get("end_frame"),
|
| 81 |
+
"code": str(item.get("code") or ""),
|
| 82 |
+
"severity": str(item.get("severity") or WARN),
|
| 83 |
+
"rule": str(item.get("rule") or ""),
|
| 84 |
+
"message": str(item.get("message") or ""),
|
| 85 |
+
}
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
def rollup_episode(streams: Iterable[StreamResult | dict],
|
| 89 |
+
findings: Iterable[Finding | dict] = ()) -> dict:
|
| 90 |
+
"""第三层:整集结论。纯函数。
|
| 91 |
+
|
| 92 |
+
取所有流 + 所有 finding 里最严重的那个状态。
|
| 93 |
+
"""
|
| 94 |
+
statuses: list[str] = []
|
| 95 |
+
failed: list[str] = []
|
| 96 |
+
warned: list[str] = []
|
| 97 |
+
pending: list[str] = []
|
| 98 |
+
|
| 99 |
+
for stream in streams:
|
| 100 |
+
if isinstance(stream, StreamResult):
|
| 101 |
+
name, status = stream.stream, stream.status
|
| 102 |
+
else:
|
| 103 |
+
name, status = str(stream.get("stream") or ""), str(stream.get("status") or PASS)
|
| 104 |
+
statuses.append(status)
|
| 105 |
+
if is_blocking(status):
|
| 106 |
+
failed.append(name)
|
| 107 |
+
elif status == PENDING:
|
| 108 |
+
pending.append(name)
|
| 109 |
+
elif status == WARN:
|
| 110 |
+
warned.append(name)
|
| 111 |
+
|
| 112 |
+
for finding in findings:
|
| 113 |
+
status = finding.status if isinstance(finding, Finding) else str(finding.get("status") or PASS)
|
| 114 |
+
statuses.append(status)
|
| 115 |
+
|
| 116 |
+
overall = worst(statuses) if statuses else PASS
|
| 117 |
+
pieces: list[str] = []
|
| 118 |
+
if failed:
|
| 119 |
+
pieces.append(f"{len(failed)} stream(s) failed")
|
| 120 |
+
if warned:
|
| 121 |
+
pieces.append(f"{len(warned)} stream(s) with warnings")
|
| 122 |
+
if pending:
|
| 123 |
+
pieces.append(f"{len(pending)} stream(s) pending review")
|
| 124 |
+
return {
|
| 125 |
+
"status": overall,
|
| 126 |
+
"status_label": SEVERITY_LABELS.get(overall, overall),
|
| 127 |
+
"failed_streams": failed,
|
| 128 |
+
"warn_streams": warned,
|
| 129 |
+
"pending_streams": pending,
|
| 130 |
+
"summary": "; ".join(pieces) if pieces else "All checks passed",
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
def rollup_training_use(channels: Iterable[str],
|
| 135 |
+
episode_status: str = PASS) -> list[dict]:
|
| 136 |
+
"""第四层:这个 episode 能用于哪些训练。纯函数。
|
| 137 |
+
|
| 138 |
+
信道不具备 → 该用途判 fail("未采集"),这是**用途维度**的失败,
|
| 139 |
+
不代表数据本身坏 —— 所以 reason 写清楚是"未采集"而不是"质量差"。
|
| 140 |
+
"""
|
| 141 |
+
have = {str(item) for item in channels}
|
| 142 |
+
rows: list[dict] = []
|
| 143 |
+
for rule in TRAINING_USE_RULES:
|
| 144 |
+
required = tuple(rule["requires"])
|
| 145 |
+
missing = [channel for channel in required if channel not in have]
|
| 146 |
+
if missing:
|
| 147 |
+
rows.append({
|
| 148 |
+
"use": rule["use"],
|
| 149 |
+
"label": rule["label"],
|
| 150 |
+
"status": FAIL,
|
| 151 |
+
"reason": f"not collected: {', '.join(missing)}",
|
| 152 |
+
"requires": list(required),
|
| 153 |
+
"missing": missing,
|
| 154 |
+
})
|
| 155 |
+
else:
|
| 156 |
+
rows.append({
|
| 157 |
+
"use": rule["use"],
|
| 158 |
+
"label": rule["label"],
|
| 159 |
+
"status": episode_status,
|
| 160 |
+
"reason": "" if episode_status == PASS else "data quality problem",
|
| 161 |
+
"requires": list(required),
|
| 162 |
+
"missing": [],
|
| 163 |
+
})
|
| 164 |
+
return rows
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
def passed_for(status: str) -> bool:
|
| 168 |
+
"""该状态是否算"通过"(只有 PASS 算通过,WARN 不算)。"""
|
| 169 |
+
return status == PASS
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
def build_report(streams: Iterable[StreamResult],
|
| 173 |
+
findings: Iterable[Finding] = (),
|
| 174 |
+
*,
|
| 175 |
+
episode_id: str = "",
|
| 176 |
+
project: str = "",
|
| 177 |
+
modalities: dict[str, Any] | None = None,
|
| 178 |
+
channels: Iterable[str] = (),
|
| 179 |
+
ruleset_revision: str = "",
|
| 180 |
+
template_source: str = "",
|
| 181 |
+
elapsed_ms: int = 0) -> dict:
|
| 182 |
+
"""组装完整报告。纯函数 —— 唯一的副作用是 ``_utcnow()``。"""
|
| 183 |
+
stream_list = list(streams)
|
| 184 |
+
finding_list = list(findings)
|
| 185 |
+
episode = rollup_episode(stream_list, finding_list)
|
| 186 |
+
use_channels = {str(item) for item in channels}
|
| 187 |
+
for stream in stream_list:
|
| 188 |
+
use_channels.update(stream.channels)
|
| 189 |
+
|
| 190 |
+
return {
|
| 191 |
+
"schema_version": SCHEMA_VERSION,
|
| 192 |
+
"episode_id": episode_id,
|
| 193 |
+
"project": project,
|
| 194 |
+
"computed_at": _utcnow(),
|
| 195 |
+
"elapsed_ms": int(elapsed_ms),
|
| 196 |
+
"ruleset_revision": ruleset_revision,
|
| 197 |
+
"template_source": template_source,
|
| 198 |
+
|
| 199 |
+
# 模态清单 —— PDF §1 的「数据清单」
|
| 200 |
+
"modalities": dict(modalities or {}),
|
| 201 |
+
|
| 202 |
+
# 判定总入口:只有 PASS 算通过
|
| 203 |
+
"passed": passed_for(episode["status"]),
|
| 204 |
+
|
| 205 |
+
# 四层结果
|
| 206 |
+
"streams": [
|
| 207 |
+
item.to_dict() if isinstance(item, StreamResult) else dict(item)
|
| 208 |
+
for item in stream_list
|
| 209 |
+
],
|
| 210 |
+
"ranges": build_timeline(stream_list),
|
| 211 |
+
"episode": episode,
|
| 212 |
+
"training_use": rollup_training_use(use_channels, episode["status"]),
|
| 213 |
+
|
| 214 |
+
"findings": [
|
| 215 |
+
item.to_dict() if isinstance(item, Finding) else dict(item)
|
| 216 |
+
for item in finding_list
|
| 217 |
+
],
|
| 218 |
+
}
|
| 219 |
+
|
| 220 |
+
|
| 221 |
+
def empty_report(*, episode_id: str = "", project: str = "",
|
| 222 |
+
reason: str = "no_data") -> dict:
|
| 223 |
+
"""没有任何数据可检查时的占位报告。
|
| 224 |
+
|
| 225 |
+
``passed`` 恒为 False —— 没有数据不等于数据合格(fail closed)。
|
| 226 |
+
"""
|
| 227 |
+
return {
|
| 228 |
+
"schema_version": SCHEMA_VERSION,
|
| 229 |
+
"episode_id": episode_id,
|
| 230 |
+
"project": project,
|
| 231 |
+
"computed_at": _utcnow(),
|
| 232 |
+
"elapsed_ms": 0,
|
| 233 |
+
"ruleset_revision": "",
|
| 234 |
+
"template_source": "",
|
| 235 |
+
"modalities": {},
|
| 236 |
+
"passed": False,
|
| 237 |
+
"streams": [],
|
| 238 |
+
"ranges": [],
|
| 239 |
+
"episode": {
|
| 240 |
+
"status": ERROR,
|
| 241 |
+
"status_label": SEVERITY_LABELS[ERROR],
|
| 242 |
+
"failed_streams": [],
|
| 243 |
+
"warn_streams": [],
|
| 244 |
+
"pending_streams": [],
|
| 245 |
+
"summary": reason,
|
| 246 |
+
},
|
| 247 |
+
"training_use": [],
|
| 248 |
+
"findings": [],
|
| 249 |
+
"reason": reason,
|
| 250 |
+
}
|
processor/app/processing/cleaning/store.py
ADDED
|
@@ -0,0 +1,189 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""质检报告落盘 —— 写进 episode state 的 ``cleaning_report`` 字段。
|
| 2 |
+
|
| 3 |
+
**报告与状态流转是同一次写入**:判 FAIL/ERROR 的批次会被拦回
|
| 4 |
+
``to_review``(人工审核),并留下 ``cleaning_summary.blocking`` 这个粘性标记。
|
| 5 |
+
|
| 6 |
+
只写报告、不碰状态的方案看着更"干净",但挡住不坏数据 —— 自动批准散在
|
| 7 |
+
``ai_annotation`` / ``video_quality`` 两处门禁里,它们各看各的报告,
|
| 8 |
+
谁都不会读质检结论。判 FAIL 的集只要视频干净,照样被自动批准放行。
|
| 9 |
+
|
| 10 |
+
另外**不动** ``deleted`` / ``rejected`` 这类状态:质检不能把用户删掉或否决过的
|
| 11 |
+
批次复活。
|
| 12 |
+
|
| 13 |
+
放在 ``state/episode_states/<episode_id>.json`` 而不是项目数据集里:
|
| 14 |
+
后者是给 LeRobot 消费的干净树(``project_dataset`` 明确不建 ``processed/``、
|
| 15 |
+
``meta/processing/``),而质检报告是 episode 级的可变状态 —— ``state/`` 正是
|
| 16 |
+
它的既有归属,``ai_quality_report`` / ``video_quality_report`` 也都在这。
|
| 17 |
+
|
| 18 |
+
读走 ``localstore.read_episode_state``(带内存缓存),写走
|
| 19 |
+
``write_episode_state``(原子替换 + 同步更新缓存)—— 两条路径都自带锁,
|
| 20 |
+
不要自己读写那个文件。
|
| 21 |
+
"""
|
| 22 |
+
|
| 23 |
+
from __future__ import annotations
|
| 24 |
+
|
| 25 |
+
from typing import Any
|
| 26 |
+
|
| 27 |
+
from app.processing.cleaning.contract import is_blocking
|
| 28 |
+
|
| 29 |
+
# episode state 里放质检报告的字段名。
|
| 30 |
+
REPORT_FIELD = "cleaning_report"
|
| 31 |
+
# 只放摘要,完整报告另存 —— 报告里可能有几十条区间,全塞进 state 会让
|
| 32 |
+
# 每次 ``read_episode_state`` 都要解析几百 KB。完整报告见 ``reports_dir()``。
|
| 33 |
+
SUMMARY_FIELD = "cleaning_summary"
|
| 34 |
+
|
| 35 |
+
# 完整报告单独存一个目录,避免污染 episode state
|
| 36 |
+
_REPORTS_SUBDIR = "quality_reports"
|
| 37 |
+
|
| 38 |
+
# 允许被质检拦回人工审核的状态。
|
| 39 |
+
#
|
| 40 |
+
# **不含 ``to_review``**(本来就在那儿,再写一次是空操作)、也不含 ``deleted``
|
| 41 |
+
# 与 ``rejected`` —— 质检不能把用户删掉/否决过的批次复活。
|
| 42 |
+
_BLOCKABLE_STATUSES = frozenset({
|
| 43 |
+
"processing", "completed", "reviewed", "approved", "failed",
|
| 44 |
+
})
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def _reports_dir():
|
| 48 |
+
from app.localstore import STATE_ROOT
|
| 49 |
+
return STATE_ROOT / _REPORTS_SUBDIR
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def summary_of(report: dict) -> dict[str, Any]:
|
| 53 |
+
"""从完整报告里抽一个轻量摘要 —— 列表页/徽标只读这个。
|
| 54 |
+
|
| 55 |
+
``blocking`` 是**给自动批准路径看的**闸门(见 ``is_approval_blocked``),
|
| 56 |
+
不是展示字段:判 FAIL/ERROR 才为真,WARN 只是警告、不拦。
|
| 57 |
+
"""
|
| 58 |
+
episode = report.get("episode") or {}
|
| 59 |
+
return {
|
| 60 |
+
"schema_version": report.get("schema_version"),
|
| 61 |
+
"passed": bool(report.get("passed")),
|
| 62 |
+
"blocking": is_blocking(str(episode.get("status") or "")),
|
| 63 |
+
"status": episode.get("status"),
|
| 64 |
+
"status_label": episode.get("status_label"),
|
| 65 |
+
"summary": episode.get("summary"),
|
| 66 |
+
"counts": {
|
| 67 |
+
"fail": len(episode.get("failed_streams") or []),
|
| 68 |
+
"warn": len(episode.get("warn_streams") or []),
|
| 69 |
+
"pending": len(episode.get("pending_streams") or []),
|
| 70 |
+
},
|
| 71 |
+
"ranges": len(report.get("ranges") or []),
|
| 72 |
+
"ruleset_revision": report.get("ruleset_revision"),
|
| 73 |
+
"computed_at": report.get("computed_at"),
|
| 74 |
+
"reason": report.get("reason"),
|
| 75 |
+
}
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def is_approval_blocked(state: dict) -> bool:
|
| 79 |
+
"""该集的质检结论是否禁止**自动批准**。
|
| 80 |
+
|
| 81 |
+
自动批准(``to_review`` → ``reviewed``)散在好几处:AI 标注的质量门禁
|
| 82 |
+
(``ai_annotation._set_ai_quality_state``)、视频门禁
|
| 83 |
+
(``video_quality._apply_video_quality_result``)。它们只看**自己的**
|
| 84 |
+
报告,不看数据质检的结论 —— 于是质检判 FAIL、视频门禁判 PASS 时,坏数据
|
| 85 |
+
会被自动放行。
|
| 86 |
+
|
| 87 |
+
★ 为什么不能只靠"把 status 推回 to_review":
|
| 88 |
+
几条门禁是**并发**跑的(都在 run 完成回调里 create_task),写的先后没有
|
| 89 |
+
保证。若质检先把状态推回 to_review、视频门禁随后又置成 reviewed,
|
| 90 |
+
拦截就白做了。所以质检除了推状态,还留下这个**粘性标记**,让所有自动
|
| 91 |
+
批准路径都先看一眼。
|
| 92 |
+
|
| 93 |
+
标记不需要手工清除:下一次质检会整份覆盖 ``cleaning_summary``,
|
| 94 |
+
数据修好后 ``blocking`` 自然变回 False。
|
| 95 |
+
"""
|
| 96 |
+
summary = (state or {}).get(SUMMARY_FIELD) or {}
|
| 97 |
+
return bool(summary.get("blocking"))
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
def save_report(episode_id: str, report: dict) -> bool:
|
| 101 |
+
"""把报告写进 episode state。返回是否写入成功。
|
| 102 |
+
|
| 103 |
+
**不改 ``status``** —— 只设 ``cleaning_report`` / ``cleaning_summary`` /
|
| 104 |
+
``cleaning_at`` 三个字段。
|
| 105 |
+
|
| 106 |
+
episode 不存在时静默跳过(外键不存在就别凭空造一条 state)。
|
| 107 |
+
"""
|
| 108 |
+
from app.localstore import mutate_episode_state, _write_json
|
| 109 |
+
|
| 110 |
+
key = str(episode_id)
|
| 111 |
+
if not key:
|
| 112 |
+
return False
|
| 113 |
+
|
| 114 |
+
# 完整报告先单独落盘 —— 它可能有几��条区间,塞进 episode state 会让每次
|
| 115 |
+
# read_episode_state 都要解析几百 KB。放在下面临界区**外面**:这是个大文件
|
| 116 |
+
# 写入(在 NAS 上可能是几十毫秒),不该占着状态锁。
|
| 117 |
+
try:
|
| 118 |
+
_write_json(_reports_dir() / f"{key}.json", report)
|
| 119 |
+
except Exception:
|
| 120 |
+
pass # 摘要照样进 state,完整报告失败不影响
|
| 121 |
+
|
| 122 |
+
summary = summary_of(report)
|
| 123 |
+
|
| 124 |
+
def _apply(state: dict) -> bool:
|
| 125 |
+
state[REPORT_FIELD] = report
|
| 126 |
+
state[SUMMARY_FIELD] = summary
|
| 127 |
+
state["cleaning_at"] = report.get("computed_at")
|
| 128 |
+
|
| 129 |
+
# 数据不对 → 拦回人工审核。与写报告**同一次锁内 RMW**:分成两次写的话
|
| 130 |
+
# 中间那个窗口里,另一条并发的门禁可能把状态改成 reviewed。
|
| 131 |
+
if summary.get("blocking"):
|
| 132 |
+
current = str(state.get("status") or "")
|
| 133 |
+
if current in _BLOCKABLE_STATUSES:
|
| 134 |
+
state["status"] = "to_review"
|
| 135 |
+
state["approved_at"] = None
|
| 136 |
+
return True
|
| 137 |
+
|
| 138 |
+
# 走锁内 RMW:跑完工作流后 video_quality 的门禁任务与本任务是并发写的,
|
| 139 |
+
# 手写 read→write 会让其中一方的字段被另一方的旧快照覆盖(详见
|
| 140 |
+
# localstore.mutate_episode_state 的注释)。episode 不存在时返回 None,
|
| 141 |
+
# 保持"不凭空造 state"的既有行为。
|
| 142 |
+
return bool(mutate_episode_state(key, _apply))
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def load_report(episode_id: str) -> dict | None:
|
| 146 |
+
"""读完整报告(不存在返回 None)。"""
|
| 147 |
+
from app.localstore import _read_json
|
| 148 |
+
|
| 149 |
+
return _read_json(_reports_dir() / f"{episode_id}.json")
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
def load_summary(episode_id: str) -> dict | None:
|
| 153 |
+
"""读摘要(从 episode state 取,快)。"""
|
| 154 |
+
from app.localstore import read_episode_state
|
| 155 |
+
|
| 156 |
+
return (read_episode_state(str(episode_id)) or {}).get(SUMMARY_FIELD)
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def list_reports(limit: int = 0) -> list[dict]:
|
| 160 |
+
"""列出全部摘要,按时间倒序 —— 给质量列表页用。
|
| 161 |
+
|
| 162 |
+
只扫 ``quality_reports/`` 目录,不 join ``scan_sessions()`` —— 后者在
|
| 163 |
+
SSHFS 上冷读极慢,而报告是自包含的(项目名就在里面)。
|
| 164 |
+
"""
|
| 165 |
+
from app.localstore import _read_json
|
| 166 |
+
|
| 167 |
+
directory = _reports_dir()
|
| 168 |
+
if not directory.is_dir():
|
| 169 |
+
return []
|
| 170 |
+
items: list[dict] = []
|
| 171 |
+
for path in directory.glob("*.json"):
|
| 172 |
+
report = _read_json(path)
|
| 173 |
+
if not isinstance(report, dict):
|
| 174 |
+
continue
|
| 175 |
+
item = summary_of(report)
|
| 176 |
+
item["episode_id"] = report.get("episode_id") or path.stem
|
| 177 |
+
item["project"] = report.get("project")
|
| 178 |
+
items.append(item)
|
| 179 |
+
items.sort(key=lambda item: str(item.get("computed_at") or ""), reverse=True)
|
| 180 |
+
return items[:limit] if limit else items
|
| 181 |
+
|
| 182 |
+
|
| 183 |
+
def forget_report(episode_id: str) -> None:
|
| 184 |
+
"""删除报告(episode 被删除时调用)。"""
|
| 185 |
+
for path in (_reports_dir() / f"{episode_id}.json",):
|
| 186 |
+
try:
|
| 187 |
+
path.unlink(missing_ok=True)
|
| 188 |
+
except OSError:
|
| 189 |
+
pass
|
processor/app/processing/cleaning/trigger.py
ADDED
|
@@ -0,0 +1,225 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""质检触发 —— 把纯引擎接到运行时(工作流跑完之后)。
|
| 2 |
+
|
| 3 |
+
三层分工,别混:
|
| 4 |
+
|
| 5 |
+
engine.py 纯函数:读证据 → 跑检查 → 出报告。不写盘、不改状态。
|
| 6 |
+
store.py 落盘:写进 episode state 的 ``cleaning_report``,**不碰 status**。
|
| 7 |
+
trigger.py 本文件:从 run 快照里判断"该不该跑、按什么配置跑",
|
| 8 |
+
丢到线程池执行,再把报告交给 store。
|
| 9 |
+
|
| 10 |
+
调用方是 ``api/worker.py`` 的 run 完成回调(与 ``video_quality`` 的门禁任务
|
| 11 |
+
同一个挂载点)。
|
| 12 |
+
|
| 13 |
+
★ 为什么这里要 `to_thread`
|
| 14 |
+
-------------------------
|
| 15 |
+
``engine.run_checks`` 是**同步**的,而且会读 parquet / 必要时解码视频。直接在
|
| 16 |
+
协程里调用会把整个事件循环卡住几秒到几十秒(存储根在 NAS 上时更甚,见
|
| 17 |
+
``routes/pages.py`` 里同样的告诫)。``asyncio.to_thread`` 把它挪到线程池。
|
| 18 |
+
|
| 19 |
+
★ 与 video_quality 门禁的关系
|
| 20 |
+
----------------------------
|
| 21 |
+
两者挂在同一张 Data Quality 卡片上,**可能同时触发**(相机类工作流就是):
|
| 22 |
+
|
| 23 |
+
video_quality_gate → 通过→reviewed / 失败→to_review
|
| 24 |
+
cleaning(本模块) → 判 FAIL/ERROR 才拦回 to_review;其余只记录
|
| 25 |
+
|
| 26 |
+
三条路都写同一份 episode state,所以读改写都必须走
|
| 27 |
+
``localstore.mutate_episode_state``(锁内 RMW),否则后写的一方会把另一方
|
| 28 |
+
刚写的字段用旧快照冲掉 —— 尤其是 ``status``,被冲掉意味着失败批次看起来
|
| 29 |
+
像已通过。
|
| 30 |
+
|
| 31 |
+
**光推状态还不够**:几条门禁是并发跑的,谁先写没有保证。质检除了推状态,
|
| 32 |
+
还在 ``cleaning_summary.blocking`` 上留一个粘性标记,让
|
| 33 |
+
``ai_annotation`` / ``video_quality`` 的**自动批准**路径主动让路 ——
|
| 34 |
+
否则质检刚推回 to_review,视频门禁随后又置成 reviewed。
|
| 35 |
+
"""
|
| 36 |
+
|
| 37 |
+
from __future__ import annotations
|
| 38 |
+
|
| 39 |
+
import asyncio
|
| 40 |
+
from pathlib import Path
|
| 41 |
+
|
| 42 |
+
from app.processing.cleaning import engine, store
|
| 43 |
+
from app.processing.cleaning.contract import normalize_device_modality
|
| 44 |
+
from app.workflow_types import canonical_node_type
|
| 45 |
+
|
| 46 |
+
# 质检节点在工作流里的 slug。合并前的两个旧 slug(ai_quality_review /
|
| 47 |
+
# data_cleaning)由 ``workflow_types.canonical_node_type`` 统一映射到这里。
|
| 48 |
+
_QUALITY_NODE = "data_quality"
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _nodes_by_id(graph: dict) -> dict[str, dict]:
|
| 52 |
+
nodes = (graph or {}).get("nodes") or []
|
| 53 |
+
return {str(node.get("id")): node for node in nodes if node.get("id") is not None}
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _incoming_edges(graph: dict) -> dict[str, list[str]]:
|
| 57 |
+
"""``{目标节点 id: [源节点 id, ...]}``。"""
|
| 58 |
+
incoming: dict[str, list[str]] = {}
|
| 59 |
+
for edge in (graph or {}).get("edges") or []:
|
| 60 |
+
source, target = edge.get("source"), edge.get("target")
|
| 61 |
+
if source is None or target is None:
|
| 62 |
+
continue
|
| 63 |
+
incoming.setdefault(str(target), []).append(str(source))
|
| 64 |
+
return incoming
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def _node_type(node: dict | None) -> str:
|
| 68 |
+
return canonical_node_type((node or {}).get("data", {}).get("nodeType"))
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def _upstream_device_cards(node_id: str, by_id: dict[str, dict],
|
| 72 |
+
incoming: dict[str, list[str]]) -> tuple[str, ...]:
|
| 73 |
+
"""从质检节点往回找,收集上游的**设备卡片**模态。
|
| 74 |
+
|
| 75 |
+
往回找而不是只看直接上游,是因为中间可能夹着处理节点
|
| 76 |
+
(相机 → RGB-D_3D_BareHand → 质检),直接上游是个处理节点、认不出模态。
|
| 77 |
+
|
| 78 |
+
设备卡片就是链路源头,遇到就不再往回走 —— 否则会顺着"卡片A → 卡片B"
|
| 79 |
+
这种不合法的连线继续爬到别处。
|
| 80 |
+
"""
|
| 81 |
+
seen: set[str] = set()
|
| 82 |
+
queue = list(incoming.get(str(node_id), ()))
|
| 83 |
+
cards: list[str] = []
|
| 84 |
+
while queue:
|
| 85 |
+
current = queue.pop(0)
|
| 86 |
+
if current in seen:
|
| 87 |
+
continue
|
| 88 |
+
seen.add(current)
|
| 89 |
+
modality = normalize_device_modality(_node_type(by_id.get(current)))
|
| 90 |
+
if modality:
|
| 91 |
+
if modality not in cards:
|
| 92 |
+
cards.append(modality)
|
| 93 |
+
continue
|
| 94 |
+
queue.extend(incoming.get(current, ()))
|
| 95 |
+
return tuple(cards)
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def data_quality_target(graph: dict,
|
| 99 |
+
node_configs: dict) -> tuple[dict, tuple[str, ...]] | None:
|
| 100 |
+
"""找到**接在数据链路上**的 Data Quality 卡片。
|
| 101 |
+
|
| 102 |
+
返回 ``(卡片配置, 上游设备卡片模态)``;没有卡片、或卡片没接线 → ``None``
|
| 103 |
+
(画布上孤零零摆一张卡片只是配置,不该触发)。
|
| 104 |
+
|
| 105 |
+
★ 判据是"**有入边**",不是列举"哪些源节点算数"。
|
| 106 |
+
``ai_annotation.video_quality_gate_config`` 用的是后者,它的白名单里
|
| 107 |
+
**没有 ``gripper_device``** —— 于是 UMI 夹爪工作流接上质检卡片后静默
|
| 108 |
+
地什么都不跑。这种白名单还会继续漏:每加一种设备就漏一次,而且漏了
|
| 109 |
+
不报错。质检节点的语义本来就与上游是哪类设备无关 —— 它检查的是
|
| 110 |
+
"这个批次目录里实际有什么数据",跑哪些检查由引擎按真实信道决定。
|
| 111 |
+
"""
|
| 112 |
+
by_id = _nodes_by_id(graph)
|
| 113 |
+
incoming = _incoming_edges(graph)
|
| 114 |
+
|
| 115 |
+
# 排序保证多张卡片时结果确定,不受画布 JSON 顺序影响
|
| 116 |
+
for node_id in sorted(by_id):
|
| 117 |
+
node = by_id[node_id]
|
| 118 |
+
if _node_type(node) != _QUALITY_NODE:
|
| 119 |
+
continue
|
| 120 |
+
if not incoming.get(node_id):
|
| 121 |
+
continue # 没接线 —— 只是块配置,不触发
|
| 122 |
+
|
| 123 |
+
data = node.get("data") or {}
|
| 124 |
+
config = dict(data.get("config") or {})
|
| 125 |
+
config.update((node_configs or {}).get(node.get("id")) or {})
|
| 126 |
+
return config, _upstream_device_cards(node_id, by_id, incoming)
|
| 127 |
+
return None
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
async def run_cleaning_checks(episode_id: str, batch_dir: str | Path, *,
|
| 131 |
+
config: dict | None = None,
|
| 132 |
+
device_cards: tuple[str, ...] = (),
|
| 133 |
+
episode_index: int | None = None,
|
| 134 |
+
project: str = "",
|
| 135 |
+
probe_video: bool = True) -> dict:
|
| 136 |
+
"""跑一次质检并落盘,返回四层报告。
|
| 137 |
+
|
| 138 |
+
``episode_index`` 必传 —— 工作流给的 ``batch_dir`` 是**项目目录**,多集
|
| 139 |
+
共享(见 ``collect.gather_evidence`` 的告诫)。漏传的后果不是报错,而是
|
| 140 |
+
拿第 0 集的数据去比全部集的视频,得出一堆假的丢帧结论。
|
| 141 |
+
|
| 142 |
+
``probe_video=True`` 是刻意的:只读 parquet 时视频检查拿不到证据,
|
| 143 |
+
报告会被标 ``unchecked`` 并强制 ``passed=False`` —— 每一集都"未通过"的
|
| 144 |
+
门禁等于没有门禁。代价实测约 4.8s/集(单流 UMI,本机),跑在线程池里。
|
| 145 |
+
"""
|
| 146 |
+
report = await asyncio.to_thread(
|
| 147 |
+
engine.run_checks,
|
| 148 |
+
Path(batch_dir),
|
| 149 |
+
episode_id=str(episode_id),
|
| 150 |
+
episode_index=episode_index,
|
| 151 |
+
project=str(project or ""),
|
| 152 |
+
config=config,
|
| 153 |
+
probe_video=probe_video,
|
| 154 |
+
modality_hint=device_cards or None,
|
| 155 |
+
)
|
| 156 |
+
await asyncio.to_thread(store.save_report, str(episode_id), report)
|
| 157 |
+
return report
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
# 后台任务的强引用。
|
| 161 |
+
#
|
| 162 |
+
# ``asyncio.create_task`` 的返回值**必须有人持有**:只把它丢给事件循环的话,
|
| 163 |
+
# 任务可能在执行到一半时被 GC 回收(asyncio 文档原文:"Save a reference to
|
| 164 |
+
# the result of this function, to avoid a task disappearing mid-execution")。
|
| 165 |
+
# 这里持有到它跑完为止。
|
| 166 |
+
_BACKGROUND_TASKS: set[asyncio.Task] = set()
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
def spawn_cleaning_for_run(run: dict, episode: dict) -> bool:
|
| 170 |
+
"""worker 完成回调调用的**同步**入口。返回是否已调度。
|
| 171 |
+
|
| 172 |
+
先做图查找再建任务:卡片没接线时连任务都不建(同步、纯字典遍历,很快),
|
| 173 |
+
这样"没有质检节点"这条常见路径不会往事件循环里塞空任务。
|
| 174 |
+
"""
|
| 175 |
+
if data_quality_target((run or {}).get("graph") or {},
|
| 176 |
+
(run or {}).get("node_configs") or {}) is None:
|
| 177 |
+
return False
|
| 178 |
+
task = asyncio.create_task(trigger_cleaning_for_run(run, episode))
|
| 179 |
+
_BACKGROUND_TASKS.add(task)
|
| 180 |
+
task.add_done_callback(_BACKGROUND_TASKS.discard)
|
| 181 |
+
return True
|
| 182 |
+
|
| 183 |
+
|
| 184 |
+
async def trigger_cleaning_for_run(run: dict, episode: dict) -> dict | None:
|
| 185 |
+
"""跑一次质检的后台协程。卡片没接线时返回 None。
|
| 186 |
+
|
| 187 |
+
**永不抛异常**:这是后台任务,抛出去会变成 unhandled asyncio exception,
|
| 188 |
+
而调用方(worker 的完成响应)已经返回了,没人能处理。失败只记日志 ——
|
| 189 |
+
与 ``video_quality.run_video_quality_review`` 同样的取舍。
|
| 190 |
+
"""
|
| 191 |
+
episode_id = str((run or {}).get("episode_id") or (episode or {}).get("id") or "")
|
| 192 |
+
batch_dir = (episode or {}).get("path")
|
| 193 |
+
if not episode_id or not batch_dir:
|
| 194 |
+
return None
|
| 195 |
+
|
| 196 |
+
try:
|
| 197 |
+
target = data_quality_target(
|
| 198 |
+
(run or {}).get("graph") or {},
|
| 199 |
+
(run or {}).get("node_configs") or {},
|
| 200 |
+
)
|
| 201 |
+
if target is None:
|
| 202 |
+
return None
|
| 203 |
+
config, device_cards = target
|
| 204 |
+
|
| 205 |
+
report = await run_cleaning_checks(
|
| 206 |
+
episode_id, batch_dir,
|
| 207 |
+
config=config, device_cards=device_cards,
|
| 208 |
+
# ``get_episode`` 的记录里 episode_index 可能是 0,不能用 or 兜底
|
| 209 |
+
episode_index=(episode.get("episode_index")
|
| 210 |
+
if episode.get("episode_index") is not None else None),
|
| 211 |
+
project=str(episode.get("project") or ""),
|
| 212 |
+
)
|
| 213 |
+
except Exception as exc: # noqa: BLE001 —— 后台任务不抛
|
| 214 |
+
print(f"[Quality] Data quality checks failed for {episode_id}: "
|
| 215 |
+
f"{type(exc).__name__}: {exc}")
|
| 216 |
+
return None
|
| 217 |
+
|
| 218 |
+
summary = store.summary_of(report)
|
| 219 |
+
verdict = str(summary.get("status"))
|
| 220 |
+
if summary.get("blocking"):
|
| 221 |
+
verdict += " -> blocked, back to manual review"
|
| 222 |
+
print(f"[Quality] {episode_id}: {verdict} "
|
| 223 |
+
f"({summary.get('ranges', 0)} ranges, "
|
| 224 |
+
f"{len(device_cards)} device card(s))")
|
| 225 |
+
return report
|
processor/app/processing/modules/ai_quality_review.py
DELETED
|
@@ -1,36 +0,0 @@
|
|
| 1 |
-
"""AI Quality Review — lightweight media and annotation quality gate.
|
| 2 |
-
|
| 3 |
-
The actual check runs after the workflow result is persisted. It checks video
|
| 4 |
-
decode/frame continuity/black screens/freezes and, when connected after AI
|
| 5 |
-
Annotation, annotation coverage too. This node keeps the normal artifact
|
| 6 |
-
contract so existing worker graphs remain compatible. The service-level gate
|
| 7 |
-
moves failed/incomplete batches to the existing Reviewing queue and only
|
| 8 |
-
allows a passed result to become Approved.
|
| 9 |
-
"""
|
| 10 |
-
|
| 11 |
-
from app.processing import ProcessingModule, JobContext, ArtifactRef
|
| 12 |
-
from app.processing.registry import register
|
| 13 |
-
from app.processing.theme import REVIEW_COLOR
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
@register
|
| 17 |
-
class AIQualityReviewModule(ProcessingModule):
|
| 18 |
-
slug = "ai_quality_review"
|
| 19 |
-
version = "1.0"
|
| 20 |
-
category = "review"
|
| 21 |
-
label = "AI Quality Review"
|
| 22 |
-
icon = "ant-design:robot-outlined"
|
| 23 |
-
color = REVIEW_COLOR
|
| 24 |
-
inputs = ({"key": "data", "label": "Quality Review Target"},)
|
| 25 |
-
outputs = ({"key": "reviewed", "label": "Reviewed Data"},)
|
| 26 |
-
default_config = {"mode": "gate"}
|
| 27 |
-
execution_target = "server"
|
| 28 |
-
capabilities = ("quality_review",)
|
| 29 |
-
|
| 30 |
-
def run(self, ctx: JobContext) -> dict[str, ArtifactRef] | None:
|
| 31 |
-
# The workflow executor still receives the upstream artifact. The
|
| 32 |
-
# episode-level gate is applied by the AI annotation service after its
|
| 33 |
-
# persisted ranges have been validated.
|
| 34 |
-
if not ctx.incoming:
|
| 35 |
-
ctx.skip("No upstream data — skipped")
|
| 36 |
-
return dict(ctx.incoming)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
processor/app/processing/modules/data_quality.py
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Data Cleaning —— 数据清洗(质检):一个节点统管确定性与 AI 两类检查。
|
| 2 |
+
|
| 3 |
+
显示名是 Data Cleaning;slug 仍是 ``data_quality``(持久化契约,改它会让已保存
|
| 4 |
+
工作流的 nodeType 对不上)。两者不一致是有意的 —— 见下方端口 key 的同类说明。
|
| 5 |
+
|
| 6 |
+
本节点由两个旧节点合并而来,两者功能高度重叠:
|
| 7 |
+
|
| 8 |
+
ai_quality_review 视频检查(黑屏/冻结/丢帧/解码) + AI 标注覆盖率
|
| 9 |
+
data_cleaning 视频检查 + UMI/手套/EGO 检查(第 2 期接入)
|
| 10 |
+
|
| 11 |
+
历史原因:视频检查最早"寄居"在 ``ai_quality_review`` 里,好让纯视频工作流
|
| 12 |
+
也有质量门禁(见 ``ai_annotation.py`` 的 ``video_quality_review_node_config``
|
| 13 |
+
docstring)。现在合并成一张卡片,语义与配置都归一处。
|
| 14 |
+
|
| 15 |
+
**两个旧 slug 都作为别名保留**(``workflow_types.LEGACY_TO_CANONICAL``),
|
| 16 |
+
已保存的工作流打开时自动迁移,连线不受影响 —— 端口的 key 刻意保持不变:
|
| 17 |
+
|
| 18 |
+
输入 data 所有旧连线都用它
|
| 19 |
+
输出 reviewed 旧连线用它接 lerobot_export
|
| 20 |
+
|
| 21 |
+
检查分两层,与清洗方案 PDF §4 的两栏一一对应:
|
| 22 |
+
|
| 23 |
+
机器可直接判断(确定性、免费、可每集跑)
|
| 24 |
+
├─ 视频:解码 / 黑屏 / 冻结 / 丢帧
|
| 25 |
+
├─ UMI:帧对齐 / action / 夹爪量程 / SLAM 连续性
|
| 26 |
+
├─ 手套:全零 / 死点 / 漂移 / 饱和
|
| 27 |
+
└─ EGO:曝光 / 关键点缺失 / 左右手
|
| 28 |
+
|
| 29 |
+
需联合判断或复核(有成本、可选)
|
| 30 |
+
└─ AI 标注覆盖率;将来的 VLM 语义复核
|
| 31 |
+
|
| 32 |
+
检查项的算法实现与工作流适配器分开,与 ``app/processing/black_glove`` 同构:
|
| 33 |
+
|
| 34 |
+
app/processing/cleaning/ ← 契约 + 检查项 + 报告组装
|
| 35 |
+
app/processing/modules/data_quality.py ← 本文件:artifact 契约 + 端口 + 配置
|
| 36 |
+
|
| 37 |
+
新增检查项 = 在 ``app/processing/cleaning/checks/`` 下放一个 .py,本文件不用改。
|
| 38 |
+
|
| 39 |
+
**只出结论,不产出数据** —— 原始数据一字不改,也不生成 clean_v1 之类的副本。
|
| 40 |
+
区间信息完整保留在报告的 ``ranges`` 里,下游按 ``training_use`` 过滤即可。
|
| 41 |
+
|
| 42 |
+
数据不对时**卡在人工审核**:判 FAIL/ERROR 的批次会被推回 ``to_review``
|
| 43 |
+
(就是审核页的入口),并在 ``cleaning_summary.blocking`` 上留一个粘性标记,
|
| 44 |
+
让 AI 标注 / 视频门禁的自动批准路径让路。WARN 只记录、不拦。
|
| 45 |
+
"""
|
| 46 |
+
|
| 47 |
+
from __future__ import annotations
|
| 48 |
+
|
| 49 |
+
from app.processing import ProcessingModule, JobContext, ArtifactRef
|
| 50 |
+
from app.processing.registry import register
|
| 51 |
+
from app.processing.theme import REVIEW_COLOR
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
@register
|
| 55 |
+
class DataQualityModule(ProcessingModule):
|
| 56 |
+
slug = "data_quality"
|
| 57 |
+
version = "1.0"
|
| 58 |
+
category = "review"
|
| 59 |
+
label = "Data Cleaning"
|
| 60 |
+
# 必须取自 web/static/iconify-preload.js 的白名单 —— 图标是离线预载的,
|
| 61 |
+
# 不在表内的名字 iconify-icon 会去 api.iconify.design 拉,调色板一次渲染
|
| 62 |
+
# 全部节点,一个漏网图标就能让整块面板卡住。check-circle 表示"校验",
|
| 63 |
+
# 与同组 review 节点的 eye(human) 不冲突。
|
| 64 |
+
# 改图标后请跑 scripts/check_icon_preload.sh。
|
| 65 |
+
icon = "ant-design:check-circle-outlined"
|
| 66 |
+
color = REVIEW_COLOR
|
| 67 |
+
|
| 68 |
+
# 端口 key 是持久化契约 —— 取值必须与合并前的 ai_quality_review 逐字一致,
|
| 69 |
+
# 否则已保存工作流的连线会因 sourceHandle/targetHandle 对不上而失效。
|
| 70 |
+
# 详见 workflow_types.py 里前端对 ``out.key === inp.key`` 的校验。
|
| 71 |
+
inputs = ({"key": "data", "label": "Data"},)
|
| 72 |
+
outputs = ({"key": "reviewed", "label": "Reviewed Data"},)
|
| 73 |
+
|
| 74 |
+
# 配置结构(前端设置弹窗按设备卡片写入)::
|
| 75 |
+
#
|
| 76 |
+
# {"devices": {
|
| 77 |
+
# "gripper_device": {
|
| 78 |
+
# "checks": {
|
| 79 |
+
# "umi.slam_continuity": {"enabled": true,
|
| 80 |
+
# "params": {"max_linear_mps": 3.0}},
|
| 81 |
+
# "video.freeze": {"enabled": false}
|
| 82 |
+
# }}}}
|
| 83 |
+
#
|
| 84 |
+
# 引擎侧由 ``cleaning.engine.resolve_config()`` 解析。
|
| 85 |
+
default_config = {"devices": {}}
|
| 86 |
+
|
| 87 |
+
# ★ 刻意留空 —— 配置全部走专用弹窗(DeviceQualityModal),不走通用
|
| 88 |
+
# config_schema 渲染。
|
| 89 |
+
#
|
| 90 |
+
# 通用那套是**平铺字段列表**,表达不了"按设备卡片分组 + 每项带阈值"的
|
| 91 |
+
# 嵌套结构;硬塞会变成一长串 enable_xxx 开关。曾短暂用过那套,结果是
|
| 92 |
+
# 三个阈值在节点里和检查项里各存一份(frame_tolerance 与
|
| 93 |
+
# video.frame_drop.tolerance_ratio 同义),迟早不一致。
|
| 94 |
+
#
|
| 95 |
+
# 现在阈值只有一处定义:检查项的 ``default_params``;节点只存"覆盖值"。
|
| 96 |
+
config_schema = ()
|
| 97 |
+
|
| 98 |
+
# 真实检查在 run 完成回调里跑(那时才能读到合并后的 episode 数据),
|
| 99 |
+
# 与 human_review / 旧 ai_quality_review 的分工一致。
|
| 100 |
+
# 注:该字段当前无消费方,仅作语��标注(全仓 grep 只见 schema 与文档)。
|
| 101 |
+
execution_target = "server"
|
| 102 |
+
capabilities = ("data_quality", "quality_gate", "quality_review")
|
| 103 |
+
|
| 104 |
+
def run(self, ctx: JobContext) -> dict[str, ArtifactRef] | None:
|
| 105 |
+
"""透传上游 artifact。上游没连 → skipped。
|
| 106 |
+
|
| 107 |
+
**真正的检查不在 ``run()`` 里跑** —— 在 worker 的 run 完成回调里
|
| 108 |
+
(``api/worker.py`` 调 ``cleaning.trigger.spawn_cleaning_for_run``)。
|
| 109 |
+
|
| 110 |
+
为什么不在 ``run()``:DAG 执行期 worker 是在 staging 目录里干活、
|
| 111 |
+
产物要等 ``_publish_episode_result`` 才合并回批次目录。在这一步读
|
| 112 |
+
批次目录,拿到的是**上一轮**的数据 —— 尤其 ``action`` 列,它由
|
| 113 |
+
``umi_slam_action`` 派生产出,读早了会全是采集端的零占位。
|
| 114 |
+
完成回调时数据已经合并完毕,与 ``human_review`` / 旧
|
| 115 |
+
``ai_quality_review`` 的分工一致。
|
| 116 |
+
|
| 117 |
+
检查所需的数据**不靠端口传** —— 完成回调手上就有
|
| 118 |
+
``localstore.get_episode()`` 的记录(``path`` + ``episode_index``),
|
| 119 |
+
检查项直接读批次目录。
|
| 120 |
+
|
| 121 |
+
★ 进来的是 ``episode_index`` 不是可选的:``path`` 是**项目目录**,
|
| 122 |
+
几十集共享,漏传会拿第 0 集的数据去比全部集的视频。
|
| 123 |
+
"""
|
| 124 |
+
if not ctx.incoming:
|
| 125 |
+
ctx.skip("No upstream data — skipped")
|
| 126 |
+
return dict(ctx.incoming)
|
processor/app/project_dataset.py
CHANGED
|
@@ -102,6 +102,17 @@ def write_project_episode_index(project_root: Path,
|
|
| 102 |
# The legacy flat JSONL index is intentionally not generated anymore.
|
| 103 |
(project_root / "meta" / "episodes.jsonl").unlink(missing_ok=True)
|
| 104 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
def is_project_dataset(root: Path) -> bool:
|
| 107 |
root = Path(root)
|
|
@@ -1402,13 +1413,30 @@ def merge_normalized_episode(project_root: Path, episode_root: Path,
|
|
| 1402 |
}
|
| 1403 |
|
| 1404 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1405 |
def verify_project_dataset(project_root: Path) -> dict[str, Any]:
|
| 1406 |
root = Path(project_root)
|
| 1407 |
rows = project_episode_rows(root)
|
| 1408 |
errors: list[str] = []
|
| 1409 |
unexpected = sorted(
|
| 1410 |
path.name for path in root.iterdir()
|
| 1411 |
-
if path.name not in _CANONICAL_ROOTS and not path.name
|
| 1412 |
) if root.is_dir() else []
|
| 1413 |
errors.extend(f"unexpected project root entry: {name}" for name in unexpected)
|
| 1414 |
expected_meta = {"info.json", "stats.json", "tasks.json", "episodes"}
|
|
@@ -1416,7 +1444,7 @@ def verify_project_dataset(project_root: Path) -> dict[str, Any]:
|
|
| 1416 |
if meta_root.is_dir():
|
| 1417 |
unexpected_meta = sorted(
|
| 1418 |
path.name for path in meta_root.iterdir()
|
| 1419 |
-
if path.name not in expected_meta and not path.name
|
| 1420 |
)
|
| 1421 |
errors.extend(f"unexpected meta entry: {name}" for name in unexpected_meta)
|
| 1422 |
for row in rows:
|
|
@@ -2074,11 +2102,25 @@ def append_project_episode(project_root: Path, episode_root: Path,
|
|
| 2074 |
existing = next((row for row in rows
|
| 2075 |
if str(row.get("episode_id") or row.get("source_batch") or "") == str(episode_id)), None)
|
| 2076 |
if existing is None:
|
| 2077 |
-
|
| 2078 |
-
|
| 2079 |
-
|
| 2080 |
-
|
| 2081 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2082 |
chunk_index = episode_chunk_for_index(target_episode_index)
|
| 2083 |
else:
|
| 2084 |
if not replace:
|
|
@@ -2104,6 +2146,17 @@ def append_project_episode(project_root: Path, episode_root: Path,
|
|
| 2104 |
if project_existed and backup_root.exists():
|
| 2105 |
backup_root.rename(project_root)
|
| 2106 |
raise
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2107 |
result["verification"] = verify_project_dataset(project_root)
|
| 2108 |
result["backup"] = str(backup_root) if project_existed else None
|
| 2109 |
return result
|
|
|
|
| 102 |
# The legacy flat JSONL index is intentionally not generated anymore.
|
| 103 |
(project_root / "meta" / "episodes.jsonl").unlink(missing_ok=True)
|
| 104 |
|
| 105 |
+
# ★ 必须失效缓存 —— 本函数是**索引的唯一写入点**,而 project_episode_rows()
|
| 106 |
+
# 读的是带 60s TTL 的模块级缓存。不失效的话,同一次调用里"写完再读"会拿到
|
| 107 |
+
# 删除前的旧行。
|
| 108 |
+
#
|
| 109 |
+
# 实测后果(2026-09-11 引入缓存时漏掉这步,删除从此全坏):
|
| 110 |
+
# delete_project_episode: 读一次(预热缓存) → 删文件 → 写索引 → 校验
|
| 111 |
+
# 校验又调 project_episode_rows → 旧缓存 → 报 "missing data" → RuntimeError
|
| 112 |
+
# →「清空回收站」500。盘上的索引其实是对的,纯粹是缓存没跟上。
|
| 113 |
+
# (delete_episode 末尾也会 invalidate_session_cache(),但那要等异常之后。)
|
| 114 |
+
invalidate_project_cache(project_root)
|
| 115 |
+
|
| 116 |
|
| 117 |
def is_project_dataset(root: Path) -> bool:
|
| 118 |
root = Path(root)
|
|
|
|
| 1413 |
}
|
| 1414 |
|
| 1415 |
|
| 1416 |
+
# 人工/脚本留下的备份与临时文件 —— 不是删除或合并操作产生的,不该算到校验头上。
|
| 1417 |
+
#
|
| 1418 |
+
# ★ 为什么必须放过它们:``delete_episode`` 在**删完文件之后**才跑本校验,校验失败就
|
| 1419 |
+
# 抛 RuntimeError → 500,而「清空回收站」是循环处理的,一个项目失败整轮中止。
|
| 1420 |
+
# 实测踩过:两个 D435 项目的 meta/ 里各有一个 ``info.json.bak-20260908-144550``
|
| 1421 |
+
# (9-08 留下的),于是**每次清空都 500、什么都删不掉**,而前端没有错误分支,
|
| 1422 |
+
# 用户看到的是"点了没反应"。一个备份文件不该有这种杀伤力。
|
| 1423 |
+
_IGNORED_ENTRY_SUFFIXES = (".bak", ".orig", ".tmp", ".save", "~")
|
| 1424 |
+
|
| 1425 |
+
|
| 1426 |
+
def _is_ignorable_entry(name: str) -> bool:
|
| 1427 |
+
"""备份/临时文件,或隐藏文件 —— 校验一律不看。"""
|
| 1428 |
+
if name.startswith("."):
|
| 1429 |
+
return True
|
| 1430 |
+
return name.endswith(_IGNORED_ENTRY_SUFFIXES) or ".bak-" in name
|
| 1431 |
+
|
| 1432 |
+
|
| 1433 |
def verify_project_dataset(project_root: Path) -> dict[str, Any]:
|
| 1434 |
root = Path(project_root)
|
| 1435 |
rows = project_episode_rows(root)
|
| 1436 |
errors: list[str] = []
|
| 1437 |
unexpected = sorted(
|
| 1438 |
path.name for path in root.iterdir()
|
| 1439 |
+
if path.name not in _CANONICAL_ROOTS and not _is_ignorable_entry(path.name)
|
| 1440 |
) if root.is_dir() else []
|
| 1441 |
errors.extend(f"unexpected project root entry: {name}" for name in unexpected)
|
| 1442 |
expected_meta = {"info.json", "stats.json", "tasks.json", "episodes"}
|
|
|
|
| 1444 |
if meta_root.is_dir():
|
| 1445 |
unexpected_meta = sorted(
|
| 1446 |
path.name for path in meta_root.iterdir()
|
| 1447 |
+
if path.name not in expected_meta and not _is_ignorable_entry(path.name)
|
| 1448 |
)
|
| 1449 |
errors.extend(f"unexpected meta entry: {name}" for name in unexpected_meta)
|
| 1450 |
for row in rows:
|
|
|
|
| 2102 |
existing = next((row for row in rows
|
| 2103 |
if str(row.get("episode_id") or row.get("source_batch") or "") == str(episode_id)), None)
|
| 2104 |
if existing is None:
|
| 2105 |
+
used = {_episode_index(row.get("episode_index")) for row in rows
|
| 2106 |
+
if _episode_index(row.get("episode_index")) is not None}
|
| 2107 |
+
# ★ 优先沿用**名字里的编号**,让存储索引与公开 ID 一致。
|
| 2108 |
+
#
|
| 2109 |
+
# 此前一律 max(used)+1,于是"名字说 000005、索引是 3"这种对不上的
|
| 2110 |
+
# 情况会出现 —— 实测 119 个带编号批次里 **21 个**中招。它的直接
|
| 2111 |
+
# 危害在 _match_episode_row 的名字回退:按名字推出索引后会撞上
|
| 2112 |
+
# 别的批次(模拟:删 ..._000002 时若它自己的行已缺失,会去删
|
| 2113 |
+
# index=2 那个批次的数据和视频)。
|
| 2114 |
+
#
|
| 2115 |
+
# 编号允许空洞(见本模块 delete_project_episode 的说明
|
| 2116 |
+
# "gaps are valid"),所以直接采用名字里的编号是安全的。
|
| 2117 |
+
# 两种情况仍回落到 max+1:名字里没有编号;或该编号已被别的批次
|
| 2118 |
+
# 占用(撞号)。
|
| 2119 |
+
named = episode_index_from_name(episode_id)
|
| 2120 |
+
if named is not None and named not in used:
|
| 2121 |
+
target_episode_index = named
|
| 2122 |
+
else:
|
| 2123 |
+
target_episode_index = max(used, default=-1) + 1
|
| 2124 |
chunk_index = episode_chunk_for_index(target_episode_index)
|
| 2125 |
else:
|
| 2126 |
if not replace:
|
|
|
|
| 2146 |
if project_existed and backup_root.exists():
|
| 2147 |
backup_root.rename(project_root)
|
| 2148 |
raise
|
| 2149 |
+
# ★ 换入之后必须失效【project_root】的缓存,不能只靠
|
| 2150 |
+
# write_project_episode_index 里那次 —— 那次失效的是 **temp_root**
|
| 2151 |
+
# 的 key,而内容是靠 rename 换过来的。
|
| 2152 |
+
#
|
| 2153 |
+
# 不失效的后果有两条:
|
| 2154 |
+
# (1) 紧跟的 verify_project_dataset(project_root) 读到的还是**追加前**
|
| 2155 |
+
# 的旧行 —— 校验等于在验旧树,"通过"是空头支票(实测:连续三次
|
| 2156 |
+
# append 全部通过,而盘上索引早已是 [0,1,2,5,6,7]);
|
| 2157 |
+
# (2) 缓存有 60 秒的盲目 TTL(TTL 内连指纹都不查),上传后前端
|
| 2158 |
+
# 最长 60 秒看不到新批次。
|
| 2159 |
+
invalidate_project_cache(project_root)
|
| 2160 |
result["verification"] = verify_project_dataset(project_root)
|
| 2161 |
result["backup"] = str(backup_root) if project_existed else None
|
| 2162 |
return result
|
processor/app/routes/export.py
CHANGED
|
@@ -38,6 +38,9 @@ router = APIRouter(prefix="/api/v1/export", tags=["export"])
|
|
| 38 |
|
| 39 |
EXPORTS_DIR = STATE_ROOT / "exports"
|
| 40 |
EXPORT_TASKS_DIR = STATE_ROOT / "export_tasks"
|
|
|
|
|
|
|
|
|
|
| 41 |
# 导出页/批量导出的构建目录与 zip 放本地盘:同一份视频数据不再在
|
| 42 |
# NAS(SSHFS ~28MB/s)上读写三遍,只保留一次源文件读取。
|
| 43 |
LOCAL_EXPORTS_DIR = settings.upload_staging_root / "exports"
|
|
@@ -484,6 +487,38 @@ async def batch_download_sessions(request: Request):
|
|
| 484 |
|
| 485 |
# ── 导出任务(JSON,后台打包)─────────────────────────
|
| 486 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 487 |
def _load_task(job_id: str) -> dict | None:
|
| 488 |
return _read_json(EXPORT_TASKS_DIR / f"{job_id}.json", None)
|
| 489 |
|
|
@@ -573,8 +608,9 @@ def _run_export_task(job_id: str, episode_ids: list[str], export_format: str,
|
|
| 573 |
]
|
| 574 |
if blocked:
|
| 575 |
raise RuntimeError("Some episodes are still in Reviewing")
|
| 576 |
-
# 构建与 zip 都在本地盘进行(NAS 只读一次源文件)
|
| 577 |
-
#
|
|
|
|
| 578 |
dest = LOCAL_EXPORTS_DIR / f"{job_id}.zip"
|
| 579 |
dest.parent.mkdir(parents=True, exist_ok=True)
|
| 580 |
export_root = LOCAL_EXPORTS_DIR / f"{dataset_name}-{job_id[:8]}"
|
|
@@ -591,12 +627,17 @@ def _run_export_task(job_id: str, episode_ids: list[str], export_format: str,
|
|
| 591 |
progress_callback=_on_episode)
|
| 592 |
_set_progress(0.75)
|
| 593 |
_zip_directory(export_root, dest)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 594 |
_set_progress(0.95)
|
| 595 |
else:
|
| 596 |
_set_progress(0.2)
|
| 597 |
_zip_episode_selection([str(e) for e in episode_ids], dest)
|
| 598 |
_set_progress(0.95)
|
| 599 |
-
|
|
|
|
| 600 |
task = _load_task(job_id)
|
| 601 |
if task:
|
| 602 |
task["status"] = "completed"
|
|
@@ -788,7 +829,9 @@ _RE_TASKS: dict[str, dict] = {}
|
|
| 788 |
_RE_TASKS_DIR = STATE_ROOT / "re_export_tasks"
|
| 789 |
|
| 790 |
_PASSTHROUGH_TYPES = {
|
| 791 |
-
|
|
|
|
|
|
|
| 792 |
}
|
| 793 |
_CAMERA_TYPES = {"rgb_camera", "fisheye_camera", "rgbd_camera", "stereo_camera",
|
| 794 |
"stereo_rgbd_camera", "mono_camera"}
|
|
|
|
| 38 |
|
| 39 |
EXPORTS_DIR = STATE_ROOT / "exports"
|
| 40 |
EXPORT_TASKS_DIR = STATE_ROOT / "export_tasks"
|
| 41 |
+
# 可训练的数据集 —— 与 EXPORTS_DIR 分开:那边是按 run_id 命名的**处理产物**
|
| 42 |
+
# (每条一集),这边是**能拿去训练的成品**,训练页只列这个目录。
|
| 43 |
+
DATASETS_DIR = STATE_ROOT / "datasets"
|
| 44 |
# 导出页/批量导出的构建目录与 zip 放本地盘:同一份视频数据不再在
|
| 45 |
# NAS(SSHFS ~28MB/s)上读写三遍,只保留一次源文件读取。
|
| 46 |
LOCAL_EXPORTS_DIR = settings.upload_staging_root / "exports"
|
|
|
|
| 487 |
|
| 488 |
# ── 导出任务(JSON,后台打包)─────────────────────────
|
| 489 |
|
| 490 |
+
def _persist_trainable_dataset(export_root: Path, dataset_name: str) -> Path | None:
|
| 491 |
+
"""把导出好的数据集落到 ``state/datasets/<名字>/``,让它能当训练输入。
|
| 492 |
+
|
| 493 |
+
原来是打包完直接 ``rmtree``(省本地空间)—— 结果导出的成果只剩一个 zip,
|
| 494 |
+
磁盘上什么都没有,训练页根本无从列起。
|
| 495 |
+
|
| 496 |
+
**交换式替换**,不是"先删旧的再搬新的":先把新数据集搬进 ``.staging``,
|
| 497 |
+
再把旧的挪到 ``.old``,然后 rename 上位,最后删 ``.old``。任何一步失败都不会
|
| 498 |
+
让目标位置处于"半个数据集"的状态 —— 直接 rmtree 再 move 的话,搬到一半
|
| 499 |
+
失败就两头落空。
|
| 500 |
+
|
| 501 |
+
返回落位后的路径;``export_root`` 里没有 ``meta/info.json``(不是数据集)时
|
| 502 |
+
返回 None,调用方照旧删掉临时目录。
|
| 503 |
+
"""
|
| 504 |
+
if not (export_root / "meta" / "info.json").is_file():
|
| 505 |
+
return None
|
| 506 |
+
|
| 507 |
+
DATASETS_DIR.mkdir(parents=True, exist_ok=True)
|
| 508 |
+
target = DATASETS_DIR / dataset_name
|
| 509 |
+
staging = DATASETS_DIR / f".{dataset_name}.staging"
|
| 510 |
+
previous = DATASETS_DIR / f".{dataset_name}.old"
|
| 511 |
+
|
| 512 |
+
shutil.rmtree(staging, ignore_errors=True)
|
| 513 |
+
shutil.move(str(export_root), str(staging))
|
| 514 |
+
shutil.rmtree(previous, ignore_errors=True)
|
| 515 |
+
if target.exists():
|
| 516 |
+
target.rename(previous)
|
| 517 |
+
staging.rename(target) # 同盘 rename,原子
|
| 518 |
+
shutil.rmtree(previous, ignore_errors=True)
|
| 519 |
+
return target
|
| 520 |
+
|
| 521 |
+
|
| 522 |
def _load_task(job_id: str) -> dict | None:
|
| 523 |
return _read_json(EXPORT_TASKS_DIR / f"{job_id}.json", None)
|
| 524 |
|
|
|
|
| 608 |
]
|
| 609 |
if blocked:
|
| 610 |
raise RuntimeError("Some episodes are still in Reviewing")
|
| 611 |
+
# 构建与 zip 都在本地盘进行(NAS 只读一次源文件)。产物两样都留:
|
| 612 |
+
# zip 供下载,数据集目录(LeRobot 分支)供训练页选用 —— 后者走
|
| 613 |
+
# _persist_trainable_dataset 落到 state/datasets/,不再是"打完包就删"。
|
| 614 |
dest = LOCAL_EXPORTS_DIR / f"{job_id}.zip"
|
| 615 |
dest.parent.mkdir(parents=True, exist_ok=True)
|
| 616 |
export_root = LOCAL_EXPORTS_DIR / f"{dataset_name}-{job_id[:8]}"
|
|
|
|
| 627 |
progress_callback=_on_episode)
|
| 628 |
_set_progress(0.75)
|
| 629 |
_zip_directory(export_root, dest)
|
| 630 |
+
# 打包后**留下**数据集,不要删 —— 训练页的数据集列表就是从这里读的
|
| 631 |
+
# (state/datasets/)。只有它压根不是数据集时才清掉临时目录。
|
| 632 |
+
if _persist_trainable_dataset(export_root, dataset_name) is None:
|
| 633 |
+
shutil.rmtree(export_root, ignore_errors=True)
|
| 634 |
_set_progress(0.95)
|
| 635 |
else:
|
| 636 |
_set_progress(0.2)
|
| 637 |
_zip_episode_selection([str(e) for e in episode_ids], dest)
|
| 638 |
_set_progress(0.95)
|
| 639 |
+
# 非 LeRobot 分支产出的只是一个选集的 zip,没有数据集目录可留。
|
| 640 |
+
shutil.rmtree(export_root, ignore_errors=True)
|
| 641 |
task = _load_task(job_id)
|
| 642 |
if task:
|
| 643 |
task["status"] = "completed"
|
|
|
|
| 829 |
_RE_TASKS_DIR = STATE_ROOT / "re_export_tasks"
|
| 830 |
|
| 831 |
_PASSTHROUGH_TYPES = {
|
| 832 |
+
# data_quality 由 ai_quality_review + data_cleaning 合并而来;此处的 t 已过
|
| 833 |
+
# canonical_node_type(),旧 slug 不会出现。
|
| 834 |
+
"annotation", "human_review", "ai_annotation", "data_quality",
|
| 835 |
}
|
| 836 |
_CAMERA_TYPES = {"rgb_camera", "fisheye_camera", "rgbd_camera", "stereo_camera",
|
| 837 |
"stereo_rgbd_camera", "mono_camera"}
|
processor/app/routes/pages.py
CHANGED
|
@@ -21,10 +21,30 @@ _static_dir = STATIC_DIR
|
|
| 21 |
|
| 22 |
# 静态资源缓存键的唯一来源:模板里写 ``?v={{ asset_version }}``。
|
| 23 |
# 以前每个模板各写各的日期串,发版时靠人记得逐个改 —— 漏一个,那个页面
|
| 24 |
-
# 就会继续发浏览器缓存里的旧脚本。
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
|
| 30 |
@router.get("/", response_class=HTMLResponse)
|
|
@@ -68,6 +88,16 @@ async def trash_page(request: Request):
|
|
| 68 |
return templates.TemplateResponse("trash.html", {"request": request})
|
| 69 |
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
@router.get("/users", response_class=HTMLResponse)
|
| 72 |
async def users_page(request: Request, _: dict = Depends(require_roles("admin"))):
|
| 73 |
"""Account management is visible only to administrators."""
|
|
|
|
| 21 |
|
| 22 |
# 静态资源缓存键的唯一来源:模板里写 ``?v={{ asset_version }}``。
|
| 23 |
# 以前每个模板各写各的日期串,发版时靠人记得逐个改 —— 漏一个,那个页面
|
| 24 |
+
# 就会继续发浏览器缓存里的旧脚本。
|
| 25 |
+
#
|
| 26 |
+
# ★ 取的是**前端号**(web/VERSION),不是后端号。改前端只 bump 那一个文件即可,
|
| 27 |
+
# 提交信息里的「前端 X」就是它 —— 缓存键和版本记录是同一个数,不会对不上。
|
| 28 |
+
# (原来取后端号,于是每次改前端都要同时 bump 后端号和前端号两个地方。)
|
| 29 |
+
from app.web_version import frontend_version as _frontend_version
|
| 30 |
+
|
| 31 |
+
templates.env.globals["asset_version"] = _frontend_version()
|
| 32 |
+
|
| 33 |
+
# 服务端取词 —— 模板里写 ``{{ t('nav_home') }}``。语言由 app/i18n.py 的
|
| 34 |
+
# LanguageMiddleware 从 cookie 读进 ContextVar,这里是同一个上下文,读得到。
|
| 35 |
+
# 这样首屏渲染出来的就是正确语言,不再"先英文后由 JS 纠正"(那会闪一下)。
|
| 36 |
+
from app.i18n import get_current_lang as _get_lang
|
| 37 |
+
from app.i18n import t as _t
|
| 38 |
+
|
| 39 |
+
templates.env.globals["t"] = _t
|
| 40 |
+
# 模板要写 <html lang="…">,所以语言码也放进全局。
|
| 41 |
+
#
|
| 42 |
+
# ★ 注册的是**函数**,模板里必须写成 ``{{ current_lang() }}`` 带括号。
|
| 43 |
+
# Jinja 的全局是"每次渲染取一次这个对象" —— 值是固定的,而语言是**每请求**
|
| 44 |
+
# 才定的(存在 ContextVar 里),所以只能注册一个可调用的东西,渲染时调它。
|
| 45 |
+
# 少写括号不会报错,只会把函数的 repr 渲染进 HTML:
|
| 46 |
+
# <html lang="<function get_current_lang at 0x7f…>">
|
| 47 |
+
templates.env.globals["current_lang"] = _get_lang
|
| 48 |
|
| 49 |
|
| 50 |
@router.get("/", response_class=HTMLResponse)
|
|
|
|
| 88 |
return templates.TemplateResponse("trash.html", {"request": request})
|
| 89 |
|
| 90 |
|
| 91 |
+
@router.get("/train", response_class=HTMLResponse)
|
| 92 |
+
async def train_page(request: Request):
|
| 93 |
+
"""Model training — read-only view of checkpoints produced outside the platform.
|
| 94 |
+
|
| 95 |
+
Data comes from ``GET /api/v1/training/runs`` (see ``app/routes/training.py``).
|
| 96 |
+
Nothing on this page starts or stops training yet.
|
| 97 |
+
"""
|
| 98 |
+
return templates.TemplateResponse("train.html", {"request": request})
|
| 99 |
+
|
| 100 |
+
|
| 101 |
@router.get("/users", response_class=HTMLResponse)
|
| 102 |
async def users_page(request: Request, _: dict = Depends(require_roles("admin"))):
|
| 103 |
"""Account management is visible only to administrators."""
|
processor/app/routes/training.py
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Training API —— 路由层,薄薄一层。
|
| 2 |
+
|
| 3 |
+
业务逻辑在 ``app/training/`` 下(产物扫描 / GPU 读取 / 数据集发现),这里只负责
|
| 4 |
+
把它们挂到 URL 上。**没有任何训练执行** —— 见那个包的 docstring。
|
| 5 |
+
|
| 6 |
+
将来接执行时,新端点加在这里(创建 / 停止 / 日志),形状与现有的保持一致;
|
| 7 |
+
真正的子进程管理写在 ``app/training/runner.py``。
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
from fastapi import APIRouter, HTTPException
|
| 13 |
+
from fastapi.responses import FileResponse
|
| 14 |
+
|
| 15 |
+
from app.training import (
|
| 16 |
+
list_datasets, list_gpus, list_projects_progress, list_runs,
|
| 17 |
+
)
|
| 18 |
+
from app.training.runs import resolve_checkpoint_file
|
| 19 |
+
|
| 20 |
+
router = APIRouter(prefix="/api/v1/training", tags=["training"])
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
@router.get("/gpus")
|
| 24 |
+
def gpus():
|
| 25 |
+
"""本机 GPU(读 ``nvidia-smi``)。
|
| 26 |
+
|
| 27 |
+
**驱动不可用时靠 ``available: false`` 区分**,不要把它当成"利用率 0%" ——
|
| 28 |
+
详见 ``app/training/gpus.py``。
|
| 29 |
+
"""
|
| 30 |
+
return list_gpus()
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
@router.get("/datasets")
|
| 34 |
+
def datasets():
|
| 35 |
+
"""可训练的数据集 —— 审核页导出后落在 ``state/datasets/`` 的成品。
|
| 36 |
+
|
| 37 |
+
空列表是**正常**的(还没导出过),前端据此渲染引导而不是报错。
|
| 38 |
+
"""
|
| 39 |
+
return list_datasets()
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
@router.get("/runs")
|
| 43 |
+
def runs():
|
| 44 |
+
"""训练产物(checkpoints)。目录不存在时返回空列表,不是 404。"""
|
| 45 |
+
return list_runs()
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
@router.get("/ready")
|
| 49 |
+
def ready():
|
| 50 |
+
"""各项目离「可以导出」还有多远 —— 训练页「到点提醒」用。
|
| 51 |
+
|
| 52 |
+
判据:**已通过集数达到项目设置的采集上限**(``params.target_episodes``)。
|
| 53 |
+
到点之后只**提醒**,不自动导出 —— 理由见 ``app/training/projects.py``。
|
| 54 |
+
"""
|
| 55 |
+
return list_projects_progress()
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
@router.get("/runs/{run_name}/checkpoints/{step}/files/{name:path}")
|
| 59 |
+
def download_checkpoint_file(run_name: str, step: str, name: str):
|
| 60 |
+
"""下载 checkpoint 里的单个文件。
|
| 61 |
+
|
| 62 |
+
``{name:path}`` 而不是 ``{name}``:要下的文件在子目录里
|
| 63 |
+
(``pretrained_model/model.safetensors``),默认的路径参数不匹配斜杠。
|
| 64 |
+
|
| 65 |
+
**安全性由 ``resolve_checkpoint_file`` 保证** —— 它只在扫描出的真实文件清单里
|
| 66 |
+
按名精确匹配,不做路径拼接,所以构造不出清单外的路径(比如 ``../../etc/passwd``)。
|
| 67 |
+
详见那个函数的 docstring。
|
| 68 |
+
|
| 69 |
+
单个文件而不是打包 zip:一个 checkpoint 约 600MB,其中 ``training_state/``
|
| 70 |
+
(优化器状态,约 400MB)**只在续训时有用**,拿去推理/部署只需要
|
| 71 |
+
``pretrained_model/``(约 200MB)。让用户按需下载,比强制他拉一个 600MB 的包好。
|
| 72 |
+
"""
|
| 73 |
+
path = resolve_checkpoint_file(run_name, step, name)
|
| 74 |
+
if path is None or not path.is_file():
|
| 75 |
+
raise HTTPException(status_code=404, detail="No such checkpoint file")
|
| 76 |
+
# filename 只取最后一段 —— 让浏览器存成 model.safetensors 而不是把整条相对路径
|
| 77 |
+
# 当成文件名(带斜杠的文件名在多数系统上写不出来)。
|
| 78 |
+
return FileResponse(path, filename=path.name)
|