XYZPIT/vlash-random4-ee-gr00t-n1.6-160000

VLASH GR00T N1.6 — R1 Lite, EE-pose actions (random delay 0–4)

Finetune of Isaac GR00T N1.6 for the Galaxea R1 Lite embodiment that outputs end-effector pose actions instead of arm joint actions, trained with VLASH temporal delay augmentation: for every sample a delay k ~ U[0, 4] is drawn and the state/action windows are shifted by k while video and language stay at the base timestep.

video/language:  t                                   (stale observation)
state:           t + k
action chunk:    action[t + k : t + k + horizon]

The policy therefore predicts an action chunk that begins k steps after the observation it was given, which is what makes asynchronous chunk execution possible on the real robot: the next chunk can be computed while the current one is still executing.

Modalities

	keys	dim
Video	head_rgb, left_wrist_rgb, right_wrist_rgb	3 views, 1 frame
State	left_arm, right_arm, left_gripper, right_gripper, left_ee_pose, right_ee_pose	6+6+1+1+9+9 = 32
Action	left_gripper, right_gripper, left_ee_pose, right_ee_pose	1+1+9+9 = 20

EE poses are xyz + rot6d (ActionFormat.XYZ_ROT6D), where rot6d is the first two rows of the rotation matrix — 9 numbers per arm, not a 6-DoF pose vector. EE actions are relative (ActionRepresentation.RELATIVE, ActionType.EEF) against the corresponding EE state as the reference frame; grippers are absolute on a 0–100 scale.

Details

Base model	Isaac GR00T N1.6 (nvidia/GR00T-N1.6-3B)
Checkpoint	step 160000
Embodiment tag	new_embodiment (R1 Lite, bimanual + 2 grippers)
Action chunk (training)	32
max_delay_steps	4 (random per sample, k ~ U[0, 4])
use_state_ground_truth	true (delayed state is the measured state at t + k)
Global batch size / steps	32 / 160000
Dataset	R1 Lite foldhoodie, 50 episodes, 72474 frames @ 31 fps

Open-loop evaluation

Delay sweep with gr00t/eval/delayed_open_loop_eval.py, 3 trajectories × 800 steps, horizon 16. Unnormalized error, averaged over trajectories.

EE pose MSE (metres / rot6d units):

delay	0	1	2	3	4	5	6	7	8
left	5.0e-5	4.4e-5	4.3e-5	4.9e-5	4.8e-5	5.3e-5	6.0e-5	6.8e-5	7.2e-5
right	6.1e-5	6.1e-5	6.3e-5	7.3e-5	7.8e-5	8.5e-5	9.8e-5	1.04e-4	1.13e-4
mean	5.5e-5	5.2e-5	5.3e-5	6.1e-5	6.3e-5	6.9e-5	7.9e-5	8.6e-5	9.2e-5

EE MAE at the minimum is 3.2e-3 (left) / 4.3e-3 (right), i.e. a few millimetres against an xyz range of roughly 0.3–0.45 m.

EE error bottoms out at delay 1–2, which is the mean of the U[0, 4] training distribution, and rises monotonically out to delay 8 (about 1.8× the minimum). Gripper error is uncorrelated with delay — those are near-binary 0/100 transitions where a single mistimed frame swamps the metric, and the between-trajectory spread (right gripper: ~30 on one trajectory vs ~250 on another) is far larger than any delay effect. The concatenated 20-dim MSE is dominated by the grippers for the same reason, so use the EE rows to compare delays.

Caveats: the sweep was run on the training dataset, so these are train-set fit, not held-out generalization. Also, the EE action labels in this dataset are the measured EE pose at t + 1 (a state-shifted pseudo-action), not the controller's commanded target, so they carry no tracking error or actuation lag.

Files

Inference artifacts only — optimizer.pt, scheduler.pt, rng_state_*.pth, and the trainer state are not included, so this checkpoint cannot be used to resume training.

Usage

from gr00t.policy.gr00t_policy import Gr00tPolicy
from gr00t.data.embodiment_tags import EmbodimentTag

policy = Gr00tPolicy(
    embodiment_tag=EmbodimentTag.NEW_EMBODIMENT,
    model_path="XYZPIT/vlash-random4-ee-gr00t-n1.6-160000",
    device="cuda",
)
action_chunk, _ = policy.get_action(observation)
# action_chunk keys: left_gripper, right_gripper, left_ee_pose, right_ee_pose

Because of the delay augmentation, the returned chunk is meant to start executing a few steps after the observation timestamp rather than immediately. The EE outputs are absolute poses in the same frame as observation.state.*_ee_pose_9d; a downstream IK step is required to drive the arms, which the joint-action variant (XYZPIT/vlash-random4-gr00t-n1.6-160000) does not need.