Ego Clinical Task

Research dataset containing synchronized egocentric RGB, camera-frame MANO hand sequences, and automatically generated left/right glove masks. It contains 26 recordings (15,946 frames), grouped by activity in session_index.csv. The recordings are glove demonstrations and object interactions; they do not claim to contain clinical subjects or clinical outcomes.

quality_exceptions.csv is a sparse index of currently known frame-level issues, inspired by HOT3D's independent quality masks. Frame ranges are zero-based and inclusive. component identifies the affected modality; status distinguishes a confirmed invalid annotation or image from a measured timing warning or a region awaiting MANO review. An absent row means not reviewed or not flagged, not a manual QA pass. Consumers can exclude only the modalities and ranges relevant to their training task. These entries do not alter the RGB, MANO, or mask files.

Each session contains:

• rgb.mp4: rectified ZED left-camera video.
• mano.npz: camera-frame MANO parameters aligned one-to-one with RGB frames.
• metadata.json: camera model, coordinate convention, hands present, and source range.
• hand_masks.npz: lossless, bit-packed left/right glove masks aligned one-to-one with RGB frames.
• mask_metadata.json: mask format, hand presence, model, prompt, and generation method.

The compact format is compact_rgb_mano_v3. Coordinates use the rectified ZED left-camera frame: +X right, +Y down, +Z forward. Translation is in meters. Missing left or right hands are represented by empty arrays. There is no extra project-specific scale parameter; hand shape is represented by the ten MANO betas coefficients.

To decode a mask frame, use np.unpackbits and the stored image width:

import numpy as np

with np.load("hand_masks.npz", allow_pickle=False) as masks:
    width = int(masks["width"])
    left = np.unpackbits(masks["left_packed"], axis=2, count=width,
                         bitorder=str(masks["bitorder"])).astype(bool)
    right = np.unpackbits(masks["right_packed"], axis=2, count=width,
                          bitorder=str(masks["bitorder"])).astype(bool)
    # left[i] and right[i] align with frame i of rgb.mp4 and mano.npz.

For an absent hand, the corresponding packed array and decoded array have zero frames. The masks were generated with SAM 3.1 using the text prompt black glove and forward/backward video tracking. Projected MANO geometry assigns each mask to the anatomical left or right hand and produces quality alerts; it does not clip the predicted mask. There is no wrist-distance cutoff. The masks are automatically generated research annotations, not pixel-accurate ground truth. Occlusion and contact can still cause missing glove pixels or included object pixels. Known reviewed exceptions are recorded in quality_exceptions.csv. Internal validation reports and debug overlays are not part of this delivery.

The corrupted source RGB frame in session_041 frame 365 must be excluded from training even if its mask were repaired. Session 040 has been withdrawn from this release because its MANO tracking requires further investigation.

Recordings remain separate rather than being concatenated so that each video, timestamp sequence, calibration record, and MANO sequence stays internally aligned. session_index.csv is the authoritative, continuously updated list of available sessions and activity groups.