# Dataset format (LeRobot v2.1, T-Rex force layout)

```
<root>/
  meta/info.json                       codebase_version "v2.1", fps, features, totals
  meta/episodes.jsonl                  {"episode_index", "tasks": [str], "length"}
  meta/tasks.jsonl                     {"task_index", "task"}
  meta/stats.json                      mean/std/min/max/q01/q99 per feature
  meta/relative_stats_dreamzero.json   {"eef62": {q01, q99, ...}} over 20 Hz delta-base action chunks
  meta/modality.json, embodiment.json  GR00T/DreamZero modality description (state/action/video keys)
  meta/dataset_ready.json              written by scripts/check_dataset_ready.py
  data/chunk-XXX/episode_XXXXXX.parquet
  videos/chunk-XXX/<view>/episode_XXXXXX.mp4
```

## Parquet columns used by FACT

| column | shape | description |
|---|---|---|
| `timestamp` | float | native frame time (any rate; FACT resamples to a 20 Hz logical grid by nearest timestamp) |
| `observation.state_eef62` | [62] | left arm `[xyz(3), rot6d(6)]`, left hand (22), right arm (9), right hand (22) |
| `action.eef62_absolute` | [62] | commanded absolute EEF pose / hand targets in the same layout |
| `observation.tactile_force` | [60] | 10 F6 sensors × (Fx, Fy, Fz, Mx, My, Mz), finger order: left thumb..pinky, right thumb..pinky |
| `observation.state`, `action` | [58] | raw joint-space state / command (kept for reference, not used by FACT) |

`rot6d` is the first two columns of the rotation matrix.  Delta-base actions
(`fact.data_utils.eef62_delta_base`): arm translation and rotation expressed in the frame of
the block reference state (`R_ref^T (t - t_ref)`, `R_ref^T R_t`); hand targets stay absolute.

## Videos

Three RGB views are required: `observation.images.head_left`, `observation.images.left_wrist`,
`observation.images.right_wrist`.  The offline tokenizer resizes each view to 160×320, tiles
them into a 2×2 grid `[head | right ; left | black]`, resizes back to 160×320 and encodes
single frames with the Wan2.2 VAE (`[48,10,20]` latent per frame).  Deformation (tactile
camera) videos are never decoded.

## Caches

```
<cache>/latents/episode_XXXXXX.npy   float16 [N,48,10,20], one row per parquet frame (memory-mapped)
<cache>/text_cache.npz               tokens [n_tasks,T,4096] float16, mask [n_tasks,T] bool, task_index [n_tasks]
```

Built by `scripts/build_cache_dataset.py --what latents|text` (automatically by
`scripts/train_full.sh`).

## Sharpa north_ces datasets

RoCo-TaskBoardAssembly, Robotic-Origami-Challenge and SharpaDex-v1.0 (Sharpa north_ces
humanoid, 65 joints, Sharpa Wave hands, 10-finger F6 tactile) were converted into this layout.
`observation.state_eef62` was obtained with a product-of-exponentials forward kinematics of the
torso + arm chain fitted to the recorded `tcp_pose` (0.00 mm residual; parameters in
`data_tools/north_ces_fk_params.json`), the 22 hand joints map one-to-one, and
`action.eef62_absolute` is the commanded `action.tcp_pose` / hand targets.  Episodes without
tactile force were skipped.  `data_tools/build_merged_dataset.py` merges such datasets with
the T-Rex data.