AI & ML interests
Persistent preprocessing, dataset composition and caching for large robotics datasets.
Recent Activity
What is apairo?
apairo is a unified loader for robotics sensor datasets — lidar, cameras,
poses, IMU, labels — with one chainable API across synchronous (KITTI-style)
and asynchronous (multi-rate) layouts. No bespoke glob + np.load loaders,
no hand-written timestamp matching: everything is a dataset you filter,
select, cache, synchronize, concat and feed straight to a PyTorch
DataLoader.
- numpy in, numpy out —
ds[i].data["lidar"] -> np.ndarray; convert to torch/tf at the edge, never inside the dataset. - everything is a dataset —
filter / select / cache / join / concat / synchronize / split / transformall return chainable, lazy views. .apairois the source of truth on disk — channels (raw + derived) are declared in a sidecar; expensive preprocess output is persisted and reloaded transparently.
import apairo
ds = (apairo.RawDataset("/data/mission", keys=["lidar", "image", "gps"])
.synchronize(reference="lidar", method="nearest", tolerance=0.05)
.filter(lambda s: s.data["lidar"].shape[0] > 1000))
ds[0].data # {"lidar": (N, 4), "image": (H, W, 3), "gps": (3,)}
# an apairo dataset *is* a torch Dataset -> DataLoader(ds, collate_fn=...), no adapter
Point RawDataset at a directory and it loads — no code. When a channel's clock
lives in its filenames, declare it right there and apairo reads it in memory,
never writing into your data. See
Bring your own dataset.
The ecosystem
Mechanisms live in the core; collections live in satellites. The core never gains a dependency beyond numpy + PyYAML.
| Repo | What it does |
|---|---|
| apairo | The core — load / synchronize / filter / cache / preprocess robotics datasets, one API for sync + async layouts. |
| apairo_transform | Access-time numpy transforms & augmentations (range/box filters, voxelization, rotations, interpolators). |
| apairo_preprocess | Heavy offline preprocessors, persisted as derived .apairo channels and reloaded transparently. |
| apairo_extractor | Turn ROS bags into the apairo / KITTI on-disk layout, with optional preprocessing. |
| apairo_rr | Rerun-based lidar / multi-sensor visualization of apairo datasets. |
| apairo_huggingface | Label apairo datasets and export them to the HuggingFace LeRobotDataset format. |
Getting started
pip install apairo # Python >= 3.11
pip install apairo[vision] # optional: image loading (Pillow)
- Documentation — quickstart, the
.apairoschema, synchronizing async sensors, preprocessing. - Bring your own dataset — YAML profiles, thin subclasses, filename-encoded keys.
- apairo-robotics/apairo — source, issues, contributing.