k-valentin/unitree-g1-mujoco

Unitree G1 MuJoCo model for LeRobot

This fork of lerobot/unitree-g1-mujoco defaults to the 23dof (rev_1_0) body with the Pollen Robotics AmazingHand end effector and a D455 pan/tilt head (one published head camera). Body (29dof/23dof), end effector (rubber_hand, none, dex1, dex3, amazing_hand), head mount (fixed/pan_tilt) and head sensor (d435i/d455) combine freely: sim/mjcf/compose.py composes the selected scene at load time from a bare body and reusable parts files. This is the model repository loaded dynamically by LeRobot through env.py.

The existing make_env() entry point, body motor commands, body state DDS topics, simulation stepping and ZMQ image message format are preserved.

Files. Every model is one bare body plus parts files:

File	What	Origin
assets/g1_29dof.xml, assets/g1_23dof.xml	bodies without hands, bare wrist inertia	extracted from Unitree's MJCF
assets/end_effectors/rubber_hand.xml	Unitree's stock passive hands	extracted from Unitree's 29dof MJCF
assets/end_effectors/dex3.xml	Dex3-1 hands	extracted from Unitree's Dex3 MJCF + URDF
assets/end_effectors/dex1.xml	Dex1-1 grippers	generated by tools/build_dex1_model.py from reference/hiw500/
assets/end_effectors/amazing_hand.xml	AmazingHands	generated by tools/build_amazing_hand.py from assets/amazing_hand/
assets/heads/pan_tilt.xml	pan/tilt head mount (the fixed mount is the body's own)	from the lab URDF (g1_comp.urdf)
assets/sensors/d435i.xml	RealSense D435i head camera (stock G1)	D435i color stream optics
assets/sensors/d455.xml	RealSense D455 head camera	D455 color stream optics
assets/worlds/pick_cylinder.xml	tabletop world (table, free cylinder, table_view camera) merged into the scene by WORLD	this fork
assets/scene.xml	floor, lights and global camera every model is included into	upstream

An end-effector parts file defines left_ee_mount / right_ee_mount bodies in the 29dof wrist_yaw_link frame, with the meshes, defaults, actuators, sensors and equality constraints they use; a head mount defines a body tree in the 23dof torso_link frame with a head_sensor_mount, and a sensor file a head_sensor body in that mount's frame. The extracted files name their source; Unitree's full models are in this repo's history (commit 1801640). Generated files are checked by ToolsTests and must not be edited by hand.

Body variants

config.yaml defaults to BODY: 23dof, END_EFFECTOR: amazing_hand and HEAD_MOUNT: pan_tilt, HEAD_SENSOR: d455. The 23dof body models the rev_1_0 hardware, which has no waist_roll/waist_pitch and no wrist_pitch/wrist_yaw per arm (6 fewer joints than the 29dof body).

BODY	Base model	Body motors	mode_machine
23dof (default)	assets/g1_23dof.xml	23	4
29dof	assets/g1_29dof.xml	29	2

The DDS LowState_/LowCmd_ messages always carry 35 motor slots. The 23dof body publishes only slots 0-12, 15-19, 22-26; the 6 slots it has no joint for (13, 14, 20, 21, 27, 28, i.e. waist_roll/waist_pitch and each arm's wrist_pitch/wrist_yaw) are left at the SDK default q=dq=tau=0, and any command written to them is ignored. This mapping lives in sim/unitree_sdk2py_bridge.py's JOINT_SLOTS.

To use the 23dof body from LeRobot:

lerobot-record \
  --robot.type=unitree_g1 \
  --robot.is_simulation=true \
  --robot.sim_env_repo_id=k-valentin/unitree-g1-mujoco \
  ...

Direct users of this repository select it with make_env(body="23dof") (or BODY: 23dof in config.yaml).

Choose the end effectors

End effectors are named after the hardware; each one mounts on either body. config.yaml defaults to END_EFFECTOR: amazing_hand. Finger counts, effort limits and the published camera list follow the selection automatically.

Selection	Parts (from)	Actuated joints per side	Cameras
rubber_hand	Unitree's stock passive hand	none	head
none	bare wrist (no parts)	none	head
dex1	Dex1-1 gripper	2 fingers (DDS)	head + both wrists
dex3	Dex3-1 hand (with Dex1's wrist cameras)	7 (DDS)	head + both wrists
amazing_hand (default)	AmazingHand	8 servos (ZMQ bridge)	head

HEAD_MOUNT: pan_tilt adds the 2-DoF pan/tilt head (ZMQ bridge) to any of them, carrying any HEAD_SENSOR.

Mounting. The *_ee_mount bodies go inside *_wrist_yaw_link on the 29dof body, and inside *_wrist_roll_link at the wrist pitch+yaw offsets (+0.084 m) on the 23dof body, i.e. where the hand sits on a 29dof arm with its wrist pitch/yaw at zero. The bodies carry the bare wrist inertia: on 29dof the URDF's bare wrist_yaw_link; on 23dof Unitree's fused wrist-roll + rubber-hand link minus the rubber hand's mass (Unitree lumps it into the 29dof wrist_yaw_link, which gives it). Dex hands on the 23dof body and the AmazingHand on either body have no adapter CAD yet, so those mounts are placeholders. The composed MJCF is written to a temp directory (<tmp>/unitree_g1_mujoco/) with absolute mesh paths.

Direct users of this repository can also call:

from env import make_env

if __name__ == "__main__":
    env = make_env(end_effector="dex3")  # Omit the argument for dex1.
    try:
        env.reset()
        while True:
            env.step()  # Body commands continue to arrive over DDS.
    finally:
        env.close()

Pan/tilt head and AmazingHand

HEAD_MOUNT: pan_tilt adds a 2-DoF Dynamixel pan/tilt head (assets/heads/pan_tilt.xml), here with HEAD_SENSOR: d455, an Intel RealSense D455 (assets/sensors/d455.xml), and END_EFFECTOR: amazing_hand a Pollen Robotics AmazingHand on each wrist (assets/end_effectors/amazing_hand.xml). config.yaml defaults to BODY: 23dof, END_EFFECTOR: amazing_hand, HEAD_MOUNT: pan_tilt, HEAD_SENSOR: d455, CAMERAS: ["head_camera"].

• Head = mount + sensor: HEAD_MOUNT and HEAD_SENSOR are independent. The fixed mount is the body's own head_sensor_mount (the stock head, at the pose of the stock camera); pan_tilt replaces it with assets/heads/pan_tilt.xml. A sensor (assets/sensors/<name>.xml) is a head_sensor body placed in the mount's head_sensor_mount, looking along its +x with +z up, holding a head_camera and optionally its own geoms/inertia. A new camera is a new sensors file and its name in sim/mjcf/compose.py's HEAD_SENSORS.

• Camera intrinsics: each sensor's head_camera carries pinhole intrinsics for the recorded 640x480 color mode (resolution, focalpixel, principalpixel, sensorsize), so a 640x480 render has the real camera's geometry. They come from the Intel D400-series datasheet until each unit's calibration is read (rs-enumerate-devices -c); the derivation is in each sensor file. Lens distortion is not modelled.

  HEAD_SENSOR	Datasheet RGB FOV (H x V)	640x480 focal length	640x480 FOV (H x V)
  d435i (default, stock G1)	69 x 42 deg (1920x1080)	623.01 px	54.4 x 42.1 deg
  d455	90 x 65 deg (1280x800)	380.36 px	80.1 x 64.5 deg

• Mount chain: torso_link -> head_servo_link (fixed) -> xl330_link (head_yaw_joint, pan, range -0.7..0.7 rad) -> head_tilt_link (head_pitch_joint, tilt, range -1.5708..0.8 rad) -> head_sensor_mount (fixed, the front face of the tilt bracket's tip). Both joints are position actuators with kp=5.

• Head meshes: the links are primitive boxes (head_servo_link 20x20x20 mm, xl330_link 20x34x26 mm, head_tilt_link 90x25x25 mm); config.yaml's HEAD_MESHES: true (default) swaps in the lab's assets/meshes/head/{head_servo_link,xl330_link,head_tilt_link}.STL when all three are present (see THIRD_PARTY_NOTICES.md for their provenance).

• Hands: each AmazingHand contributes 8 actuated hinge joints named by servo ID (left_hand_motor11_joint .. left_hand_motor18_joint, right_hand_motor1_joint .. right_hand_motor8_joint; finger 1 servo 1 = the lowest ID), ctrlrange snapped to exactly +-pi/2.

• Joint names vs lerobot: head/hand joints and actuators use the G1 MJCF snake_case style, and sim/head_hand_sim.py's mjcf_joint_name maps lerobot's Unitree-style motor names onto them: kHeadYaw -> head_yaw_joint, kHeadPitch -> head_pitch_joint, kLeftHandMotor11 -> left_hand_motor11_joint. Link names (head_servo_link, xl330_link) keep the lab URDF's part names; its d455_link is head_tilt_link here, as it can carry any sensor. Every body/joint/geom/mesh/material/actuator from the source onshape-to-robot export is prefixed left_hand_/right_hand_, including the 24 passive ball/hinge joints per hand that close the finger's parallel four-bar linkage (<equality connect> constraints, carried over renamed). The source geoms are visual-only (contype="0" conaffinity="0"); tools/build_amazing_hand.py adds one capsule per phalanx (fitted to the mesh) and a box for the palm plate as collision geoms (group 3, hidden by default, contype="2" conaffinity="0", friction 1.5 0.02 0.001, condim 6), so a hand only touches world objects whose geoms have conaffinity bit 2 (e.g. pick_cylinder), never the robot, the floor or itself. The finger actuators are limited to the SCS0009 stall torque (forcerange +-0.23 Nm) and the passive linkage joints carry 0.005 Nm of friction loss instead of the export's 0.1 Nm, which would lock them against that torque. Meshes are copied from AHSimulation/AH_Left|Right/mjcf under assets/amazing_hand/{left,right}/; see THIRD_PARTY_NOTICES.md for the CC-BY-4.0 attribution.

• Mount transform: the Onshape connector transform from the G1 wrist to the AmazingHand mount is not available yet, so tools/build_amazing_hand.py uses a placeholder (pos="0.13 0 0", a quaternion rotating the hand's local +z, its finger-reach axis, onto the wrist's local +x, so the hand extends away from the elbow). Replace both the offset and the rotation once the CAD is available.

• DDS scope: NUM_MOTORS/motor_effort_limit_list stay body-only (23, matching JOINT_SLOTS["23dof"]); the D455 pan/tilt head and the 16 hand actuators are never wired to DDS. sim/base_sim.py's joint scan never adds joints to left_hand_index/right_hand_index when END_EFFECTOR == "amazing_hand" (is_dds_hand is False), so those two lists stay empty and dds_actuator_index covers only the 23 body actuators; the 24 passive linkage joints per hand also carry a left_hand_/right_hand_ prefix but are never actuated, so they would not match this scan even if it ran. mj_data.ctrl for the 18 non-DDS actuators (head + both hands) is left at whatever another writer sets, never overwritten by the body torque loop -- that other writer is the ZMQ bridge described next.

• Head/hand ZMQ bridge: whenever HEAD_MOUNT != "fixed" or END_EFFECTOR == "amazing_hand", env.py's make_env() starts sim/head_hand_sim.py's SimHeadHandDevice (a MuJoCo-backed stand-in for lerobot's real HeadHandDevice) behind lerobot.robots.unitree_g1.headhand_zmq.HeadHandServer, in a daemon thread bound to 127.0.0.1. Ports default to HEADHAND_STATE_PORT: 6003 / HEADHAND_CMD_PORT: 6002 in config.yaml, overridable with UNITREE_G1_MUJOCO_HEADHAND_STATE_PORT / UNITREE_G1_MUJOCO_HEADHAND_CMD_PORT. A lerobot.robots.unitree_g1.UnitreeG1 robot with is_simulation=True, head_mount=pan_tilt and end_effector=amazing_hand talks to this bridge exactly as it would to real Dynamixel/Feetech hardware, and passes its body/end_effector/head_mount/head_sensor to make_env as the UNITREE_G1_MUJOCO_* variables below. The server thread is stopped from env.close().

Worlds

WORLD merges an assets/worlds/<name>.xml fragment (table, objects, cameras) into the scene next to the floor; any world fits any body, end effector and head. The default ("") is the plain floor, unchanged.

Option (config.yaml / make_env keyword / UNITREE_G1_MUJOCO_<NAME>)	Default	Meaning
WORLD / world	""	"" (plain floor) or pick_cylinder
WORLD_GRASP / world_grasp	attach	attach (kinematic grasp) or physics (finger contacts only); only meaningful with a WORLD
WORLD_RANDOMIZE / world_randomize	0.0	metres of uniform xy jitter, plus +-15 deg yaw when > 0, of the world's free objects at start and on every reset

pick_cylinder is a table (top at 0.755 m, front edge 0.20 m ahead of the pelvis) with an upright cylinder on it (cylinder free joint, radius 0.025 m, height 0.15 m, 0.05 kg, condim 4, priority 2, friction 1.0 0.03 0.002, solref 0.008 1, solimp 0.97 0.995 0.001) at (0.328, -0.071, 0.83) in the world frame, i.e. 0.328 m in front of and 0.071 m to the right of a robot standing under the GR00T controller (pelvis 0.74 m). The world sets <option cone="elliptic" impratio="10" integrator="implicitfast"/>; the plain floor keeps the defaults. tools/build_amazing_hand.py gives each hand slim phalanx cores plus a fingertip capsule per finger. tools/grasp_diag.py runs the scripted side grasp (--physics is the tuned one, --sweep the +-2 cm spawn grid) with first-contact, orientation and contact-force diagnostics. With WORLD_GRASP=physics the right AmazingHand lifts the cylinder with finger contacts only: 25/25 on the grid, rise 8-9 cm, slip 1-2 mm, closed-stage normal forces of 10-30 N. It is a palm-and-thumb pinch with finger 3 at the cylinder's far side, not a wrap: the cylinder sits 7.5 cm along the fingers and 3 cm off the palm in the right_tcp frame, fingers 1-3 close to +-70 deg, the thumb from open (-80, -70) to (40, -70) deg over the last 60 % of the 1.2 s close, then a diagonal lift (+8 cm forward per 10 cm up) because the 5-DoF arm holds the palm level only on a slanted plane. The harness drives the arm through the bridge with LeRobot's default gains plus G1ArmIK gravity torques (the teleop's path) and tracks within 3 mm; the grasp TCP (0.33, -0.15, 0.86) is where the level palm clears the torso and the table (the world's spawn is 7.7 cm further forward). The rest of the closed hand's cage is larger than the cylinder, so grasps nearer the palm or the fingertips tilt it. Not confirmed on hardware or in a teleop session; the default stays attach. With WORLD_GRASP=attach the world also holds inactive right_grasp / left_grasp welds ({side}_ee_mount to cylinder; composed in only for this option and for end effectors that have a mount). sim_step estimates each AmazingHand's closure from the measured servo angles of fingers 1-3 (LeRobot's q_to_closure: open -35/+35 deg, closed +60/-60 deg) and, at closure >= 0.6 with the cylinder's centre inside the grasp zone (right hand, in the right_tcp frame: x 0.03..0.11, y 0.00..0.07, z -0.07..0.05; y mirrored for the left), activates the weld at the current relative pose (no jump) and stops hand-object contacts; at closure <= 0.4 it releases both. reset() clears it. env.grasp_state is {side: bool}. It makes pick-up data collectable without relying on the physical grasp (above); other end effectors never attach. sim_env.reset() puts the cylinder back at its spawn pose (jittered by WORLD_RANDOMIZE, and kept at least 5 cm from the table's edge) together with the robot. With a world, reset() also puts the AmazingHands in their open pose: their zero pose is curled, with the fingertips on the cylinder. The plain floor keeps it.

The world adds a fixed table_view camera (in front of the table, looking at the cylinder) to the published cameras, next to the end effector's own (head_camera, plus the wrist cameras of the Dex hands); it is kept when config.yaml's CAMERAS lists only some of them, and make_env(cameras=[...]) still selects exactly.

env = make_env(world="pick_cylinder", world_randomize=0.02)  # or set WORLD / WORLD_RANDOMIZE in config.yaml for run_sim.py

Headless / option overrides

LeRobot's make_env(repo_id) cannot forward keyword options, so every make_env option can also be set through the environment as UNITREE_G1_MUJOCO_<OPTION>, e.g. UNITREE_G1_MUJOCO_ONSCREEN=0 (no viewer window), UNITREE_G1_MUJOCO_PUBLISH_IMAGES=0, UNITREE_G1_MUJOCO_CAMERA_PORT=5556, UNITREE_G1_MUJOCO_BODY=29dof, UNITREE_G1_MUJOCO_END_EFFECTOR=dex1, UNITREE_G1_MUJOCO_WORLD=pick_cylinder, UNITREE_G1_MUJOCO_JOYSTICK_TYPE=xbox. Keyword arguments still win when calling make_env directly. Note that after disconnect() the Python process may not exit on its own because of CycloneDDS finalizers; end the session with Ctrl-C.

Keyboard (viewer window)

Key presses in the MuJoCo viewer window are forwarded to LeRobot's unitree_g1_keyboard teleoperator (sim/keyboard_forward.py), so it works on Wayland desktops where pynput cannot capture keystrokes: focus the viewer window and use its key map (arrows = head, q/e = hands, w a s d / i j k l = sticks, t/g = base height). Keys 7/8/9 still drive the elastic band.

Gamepad

The bridge reads a pygame joystick and publishes it as the robot's wireless_remote (JOYSTICK_TYPE in config.yaml, default dualshock4 for a Sony DualShock 4 under SDL's HIDAPI driver; xbox and switch as upstream). Plug the pad in before launching.

Cameras

All three streams are enabled by default on tcp://127.0.0.1:5555, with 640 × 480 images and the existing approximately 30 Hz publishing setting.

Stream name	Mount
head_camera	Existing head camera
left_wrist_cam	Left gripper base / wrist
right_wrist_cam	Right gripper base / wrist

Each stream is advertised through the same top-level JPEG key and nested images / timestamps entries as head_camera. The existing view_cameras_live.py discovers the names from those messages. They are also listed in env.camera_configs, env.camera_names and env.metadata["cameras"]. The wrist cameras move with their respective wrists, with 95° vertical field of view and the approximate extrinsics from the HIW-500 model builder.

LeRobot's ZMQ cameras require explicit client configuration, just as the head camera does. Pass this dictionary as UnitreeG1Config(..., cameras=cameras):

from lerobot.cameras.zmq.configuration_zmq import ZMQCameraConfig

cameras = {
    name: ZMQCameraConfig(
        server_address="127.0.0.1", port=5555, camera_name=name,
        width=640, height=480, fps=30,
    )
    for name in ("head_camera", "left_wrist_cam", "right_wrist_cam")
}

An equivalent camera configuration is in lerobot_cameras.json. make_env(cameras=["head_camera"]) selects a subset; publish_images=False disables publishing. onscreen=False disables the viewer for headless use.

Gripper control

Dex1 fingers use force actuators, driven by the existing bridge's external PD controller. They start open at 0.0245 m. To preserve the simulator's hand transport, the first two motor entries on rt/dex3/left/cmd and rt/dex3/right/cmd command fingers 1 and 2, with matching state topics. Gripper q is in metres and tau is in newtons; each finger is limited to 20 N. Command both fingers to the same position for symmetric opening or closing. Original hand mode retains all seven rotational command entries per side.

The simulation lower limit is -0.023 m, following HIW-500's mesh closure trim. The source URDF retains the official -0.020 m lower limit. Run python tools/build_dex1_model.py --official-limits to use that limit in MJCF too; it leaves approximately 5.88 mm between the supplied finger pads. This model update does not add finger actions to LeRobot's 29-body-motor UnitreeG1 action schema.

Development and checks

Install the existing Unitree SDK2 / CycloneDDS prerequisites, then python -m pip install -r requirements.txt. Regenerate the generated parts files with python tools/build_dex1_model.py and python tools/build_amazing_hand.py.

MUJOCO_GL=egl python -m unittest discover -s tests -v
MUJOCO_GL=egl python tests/smoke_live.py
MUJOCO_GL=egl python tests/smoke_live.py --end-effector dex3

The regression suite uses real MuJoCo for both variants, motor/observation mapping, force-controlled closure, wrist camera motion, rendering and camera publisher shared-memory buffers. It isolates DDS with a test double. smoke_live.py additionally requires the real Gymnasium, Unitree SDK2 and ZMQ/OpenCV dependencies and checks the live environment and transports. See VALIDATION.md for what was executed for this update.

Sources

• Base model/runtime: lerobot/unitree-g1-mujoco.
• Dex1-1 URDF, meshes and wrist-camera geometry: Hxxxz0/HIW-500-controoler.
• LeRobot loading and camera configuration checked against main at b6ec006.

The original Dex1 source URDF, Apache 2.0 license and attribution notice are in reference/hiw500/. Unitree mesh assets retain their upstream terms.