YAM task suite

Scripted manipulation tasks for the YAM bimanual arm in Isaac Lab, laid out the way ManiSkill lays out its tasks: one registered class per task file, with the environment, the solvers and the motion planner as separate layers.

yam/
  motion/    how the robot MOVES        arm.py · planner.py · recorder.py
  envs/      the environment a task runs in   base_env.py · scene.py
  solvers/   scripted skills, one module each  pick_place · multi_pick · insert · stack · dual_lift
  tasks/     ONE FILE PER TASK, registered by name
  registry.py · conditions.py

Run one:

export ROBOTWIN_USD=/home/yu/internship_yu/robotwin_usd
python scripts/yam_task.py --list
python scripts/yam_task.py --task grape_box --seed 3
python scripts/yam_task.py --task grape_box --no-randomize      # nominal poses

Writing a task

A task declares what is in the scene and what counts as done. It never touches action vectors, IK or the video writer.

@register_task("grape_box")
class GrapeBoxTask(YamTaskEnv):
    title = "GRAPE -> BOX"
    tags = ["pick-place"]
    grape_spawn = (-0.03, 0.10)              # task constants as class attrs
    grape_spawn_jitter = (0.025, 0.015)      # randomization range
    gripper_effort, gripper_damping = 75, 85

    def _load_scene(self):
        self.scene.build_box({"name": "box", "xy": (-0.02, -0.26), "span": 0.26, "wall_h": 0.05})
        self._placed = [self.scene.place_object(
            {"name": "grape", "xy": self.grape_spawn, "xy_jitter": self.grape_spawn_jitter})]

    def solve(self):
        return pick_place.solve(self, obj="grape", target="box")

    def evaluate(self):
        return self.check(C.labelled("grape in box", C.object_in_region("grape", "box")),
                          C.labelled("grape lifted", C.object_lifted("grape", 0.05)))

Add the module to tasks/__init__.py so the registry sees it. Lifecycle is _load_scene() -> _initialize_episode() -> solve() -> evaluate().

Randomization: any object/container/marker accepts xy_jitter and yaw_jitter; --seed reproduces an episode exactly. Nominal poses with --no-randomize.

Asset gotchas this framework handles for you

Every one of these cost a debugging session; they are now enforced in envs/scene.py rather than repeated per task.

Symptom	Cause	Handled by
cup upside-down, basket on its side	RoboTwin GLBs are authored Y-up	pass `rpy: (90,0,0)`; the container default
container spawns 1.9 m wide	the GLB->USD converter reads `model_data` scale but does not bake it	pass `scale:` at spawn
tall object lands on its side	objects are written in above the table and drop	`reseat_objects()` seats each at its measured height
arm never moves, huge tracking error	a tall prop sits on the arm's home pose	height-aware warning at build time
container asset missing	the `_mesh` variant was never converted	explicit error printing the exact convert command
container swallows objects / they roll off it	convex decomposition fills the cavity	convert containers with `--collision none --suffix _mesh`

Grasping rules the solvers apply

Position from physics, size from the bbox. object_pos() is live; object_size() is the authored bbox and is only valid for extents — it does not follow the object once it moves or is rotated.
Close across the narrow axis. The jaw opens 9.4 cm. A bottle lying on its side is 18 cm long and 5 cm across; jaw="auto" picks the axis that fits.
A stall is not a grasp. ArmController.grasp() rejects a stall at an implausibly wide gap (that is the jaw resting on the body) and then verifies the object actually rises. Without both checks an episode mimes the whole sequence with an empty hand and still reports success.
Clamp force is object-dependent. Thin-walled vessels get crushed through above ~50; heavy solids slip below ~70. Set gripper_effort per task.
Two arms move on one profile. env.move_both() commands both and steps once; driving them in sequence parks the first arm in the second's path.

Verification

The numeric success check only inspects final object poses, and it will happily pass an episode where the cup landed upside-down or one gripper hung empty in the air. Render a contact sheet and have a vision agent judge it:

python scripts/yam_agent_loop.py evaluate            # contact sheets + review prompts
python scripts/yam_agent_loop.py update              # verdicts -> concrete parameter fixes
python scripts/yam_agent_loop.py propose             # uncovered skill axes from the asset library

Only numeric=SUCCESS and visual=CONFIRMED counts as done.