maximellerbach/omx-multicubes-mujoco

OMX multicubes (MuJoCo)

A LeRobot EnvHub environment replicating the real setup of the maximellerbach/omx_multicubes and maximellerbach/omx_pickandplace datasets: a ROBOTIS OMX arm on a grey board fenced by wooden slats, painted wooden cubes and a blue tape square, seen by a top camera and a wrist camera.

Task (--robot.task)	Dataset	Cubes	Instruction	Episode length
omx_multicubes	omx_multicubes	red, green, blue, yellow	"pick all the cubes and place them one by one in the blue square"	3600 steps (2 min)
omx_pickandplace	omx_pickandplace	red	"Pick the red cube and place it in the blue square"	900 steps (30 s)

Observations, actions and units are the dataset's, so policies trained on it (for example maximellerbach/omx_multicubes_act) run in simulation unchanged, without a rename map.

Observation	pixels: top and wrist, 480×640 RGB; agent_pos: 6 joint positions
Action	6 absolute joint position targets: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll in -100..100 (100 = 180°) and gripper in 0..100 (50 = closed), the units of LeRobot's omx_follower with its default use_degrees=False and factory calibration
Control rate	30 fps
Reward	the fraction of the task's cubes resting in the square at each step; info["cubes_in_square"] gives the count
Episode end	success (terminated) when all the task's cubes are in the square; truncated after the task's episode length (episode_length overrides it)
Reset	the arm at the datasets' start pose, the task's cubes (2.9 cm) at seeded positions and yaws in the region they start from in its dataset

Calibrated on the dataset

Everything below was fitted to the dataset's images and joint states, not guessed:

• Joint units. The OMX follower's factory calibration maps encoder 2048 to 0 and ±100 to ±180°, as ROBOTIS's ROS 2 driver maps 2048 to 0 rad, with the same directions. On top of that, small per-joint zero offsets (up to 9°) come from the fit below; the shoulder gets 4° more so that replayed dataset grasps close at mid-cube height (the real arm reaches lower than its encoders report, through gear backlash and sag).
• Top camera. Its pose and field of view (0.425 m above the board, level, tilted 10° toward the arm) maximize the overlap between the rendered arm and the arm's silhouette in 36 dataset frames, rendered at the recorded joint states.
• Camera delay. The same fit finds the images about 2 frames (67 ms) behind the joint states. The environment reproduces it: each observation pairs the current joint state with the images of 2 steps earlier (image_delay).
• Wrist camera. Fitted the same way on the gripper fingers in the wrist images: 4.4 cm in front of the wrist roll joint, on the side opposite the fingers' opening, 77° field of view.
• Scene. The tape square and the fence are dataset pixels back-projected onto the board through the fitted top camera. A top photo of the real setup confirms them within about 1 cm (front slats about 0.38 m from the pan axis, tape square 12.8 x 15.3 cm) and places the top camera's mast. The board's near edge is back-projected too, and its sides sit 8 mm outside the 9 mm side slats, as on the real board. The cube size (2.9 cm) is fitted by rendering the red cube of omx_pickandplace against its top-camera mask; the cubes' start regions are where they start in each dataset.
• Gripper. The real fingers are Fin Ray style and bend around a cube: holding one, the gripper reads 52.8 (10° open), closer than rigid fingers allow. Each finger's collision hull is cut where its inner face reaches at that opening, so the fingers close parallel on the cube at the real reading, with stiff contacts instead of sinking into it. The gripper's torque is capped at the XL330's stall torque (0.5 N·m), like the real current-limited servo, and its inertia lets it follow the recorded gripper commands within about 1.5 units.

Replaying a dataset episode's recorded actions open loop, with the cubes at their measured start positions, the simulated joints follow the recorded states within 0.5 to 2.5 units on average. All 6 replayed omx_pickandplace episodes pick the red cube and drop it in the square; while it is held, the cube slips less than 2 mm in the gripper, the fingers press less than 1 mm into it, and the gripper reads 52.8 as on the real arm.

With the LeRobot simulator server

From a LeRobot checkout, start the server (the image installs requirements.txt; env.py declares the control rate and joint names in LEROBOT_SIM):

lerobot-env-server --docker --envhub=maximellerbach/omx-multicubes-mujoco

Then drive it as a robot, here with a policy:

lerobot-rollout --robot.type=sim --robot.endpoint=tcp/127.0.0.1:7448 \
  --robot.env.type=envhub --robot.env.hub_path=maximellerbach/omx-multicubes-mujoco --robot.env.task=omx_multicubes \
  --policy.path=maximellerbach/omx_multicubes_act --fps=30

Without --robot.endpoint, the same command runs the environment in-process (with --robot.env.trust_remote_code=true and the requirements installed next to LeRobot). lerobot-eval takes the same options as --env.*, e.g. --env.type=envhub --env.hub_path=maximellerbach/omx-multicubes-mujoco --env.task=omx_multicubes.

In-process

from lerobot.envs.factory import make_env

envs = make_env("maximellerbach/omx-multicubes-mujoco", n_envs=1, trust_remote_code=True)

make_env keyword arguments override the defaults of OmxMulticubesEnv in env.py (episode length, image size, instruction, initial joints, joint noise, image delay). Headless machines need MUJOCO_GL=egl.

Files

File	Content
env.py	the EnvHub entry point (make_env), the Gymnasium env, the calibration constants and LEROBOT_SIM
omx_multicubes.xml	the scene: board, fence, tape square, cubes, top camera, mast and the arm's mount site
robotis_omx/	ROBOTIS's OMX model, attached at the mount site (env.py adds the wrist camera and finger pads)
textures/wood.png	a seamless wood grain, tinted per cube

Licenses

• This environment is Apache-2.0, including textures/wood.png (generated procedurally for it).
• robotis_omx/ is the OMX model from ROBOTIS-GIT/robotis_mujoco_menagerie (commit d8344c0dbe7a00208d0301111523dde65efc174a, preview image left out), under the Apache-2.0 license in that folder.