pi0.5 model - 🧪 gibbonbot training pipeline
- Dataset: AgentAppStore/Test_Data_Ex
- Wandb run id: None
Error Traceback
We faced an issue while training your model.
Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/openpi-source/src/openpi/training/data_loader.py", line 58, in __getitem__
return self._transform(self._dataset[index])
~~~~~~~~~~~~~^^^^^^^
File "/openpi-source/src/openpi/training/data_loader.py", line 58, in __getitem__
return self._transform(self._dataset[index])
~~~~~~~~~~~~~^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/lerobot/datasets/lerobot_dataset.py", line 1101, in __getitem__
video_frames = self._query_videos(query_timestamps, ep_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/lerobot/datasets/lerobot_dataset.py", line 1064, in _query_videos
frames = decode_video_frames(video_path, shifted_query_ts, self.tolerance_s, self.video_backend)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/lerobot/datasets/video_utils.py", line 150, in decode_video_frames
return decode_video_frames_torchcodec(video_path, timestamps, tolerance_s)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/lerobot/datasets/video_utils.py", line 334, in decode_video_frames_torchcodec
decoder = decoder_cache.get_decoder(str(video_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/lerobot/datasets/video_utils.py", line 270, in get_decoder
from torchcodec.decoders import VideoDecoder
File "/app/.venv/lib/python3.11/site-packages/torchcodec/__init__.py", line 12, in <module>
from . import decoders, encoders, samplers, transforms # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/torchcodec/decoders/__init__.py", line 7, in <module>
from .._core import AudioStreamMetadata, VideoStreamMetadata
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/__init__.py", line 8, in <module>
from ._metadata import (
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/_metadata.py", line 16, in <module>
from torchcodec._core.ops import (
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 109, in <module>
ffmpeg_major_version, core_library_path = load_torchcodec_shared_libraries()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 76, in load_torchcodec_shared_libraries
raise RuntimeError(
RuntimeError: Could not load libtorchcodec. Likely causes:
1. FFmpeg is not properly installed in your environment. We support
versions 4, 5, 6, 7, and 8, and we attempt to load libtorchcodec
for each of those versions. Errors for versions not installed on
your system are expected; only the error for your installed FFmpeg
version is relevant. On Windows, ensure you've installed the
"full-shared" version which ships DLLs.
2. The PyTorch version (2.7.1+cu126) is not compatible with
this version of TorchCodec. Refer to the version compatibility
table:
https://github.com/pytorch/torchcodec?tab=readme-ov-file#installing-torchcodec.
3. Another runtime dependency; see exceptions below.
The following exceptions were raised as we tried to load libtorchcodec:
[start of libtorchcodec loading traceback]
FFmpeg version 8:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 57, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
File "/app/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1392, in load_library
ctypes.CDLL(path)
File "/root/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libavutil.so.60: cannot open shared object file: No such file or directory
FFmpeg version 7:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 57, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
File "/app/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1392, in load_library
ctypes.CDLL(path)
File "/root/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libavutil.so.59: cannot open shared object file: No such file or directory
FFmpeg version 6:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 57, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
File "/app/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1392, in load_library
ctypes.CDLL(path)
File "/root/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libavutil.so.58: cannot open shared object file: No such file or directory
FFmpeg version 5:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 57, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
File "/app/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1392, in load_library
ctypes.CDLL(path)
File "/root/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libavutil.so.57: cannot open shared object file: No such file or directory
FFmpeg version 4:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/torchcodec/_core/ops.py", line 57, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
File "/app/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1392, in load_library
ctypes.CDLL(path)
File "/root/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/ctypes/__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /app/.venv/lib/python3.11/site-packages/torchcodec/libtorchcodec_core4.so: undefined symbol: _ZN3c1013MessageLogger6streamB5cxx11Ev
[end of libtorchcodec loading traceback].
Training parameters
{
"save_interval": 100,
"num_train_steps": 500,
"batch_size": 32,
"seed": 42,
"data.image_keys": [
"observation.images.main",
"observation.images.secondary_0"
]
}
📖 Get Started: docs.gibbon.bot
🤖 Get your robot: robots.gibbon.bot