so101-chess-act-pick-wrist-v4

ACT pick policy for an SO-101 robot arm playing chess on a real board.

The current pick policy.

  • Inputs: wrist camera, 6-D state. Output: joint targets, chunk 100. Run it with temporal ensembling (re-predict every frame, coefficient 0.01; --ensemble in so101-chess).
  • Fine-tuned from v3 on all 247 episodes of Dev-devadath/so101-chess-pick-wrist-v4, 30k steps, lr 1e-5.

Results

In a long logged game, 13 picks by the arm: 9 held on the first attempt, 3 on a retry from a hover shifted by a few mm, 1 given up. It re-picks pieces the arm placed itself.

Known issues

  • e2, d2 and e4 are hit or miss. The policy goes to nearly the same pose on a square every time instead of aiming at the piece, so a piece a few mm off decides the outcome. so101-chess retries from a shifted start and avoids opening with the e-pawn.
  • A retrain with joint angles relative to each pick's start, meant to force it to aim by sight, made it worse (1 grasp in 5) and was not published.

Use

Made for the so101-chess setup: the policy starts at an IK hover 8 cm above a piece with the jaws open and only does descend, grasp, lift. It will not do anything useful from another start pose, and it expects the same jaw pads in the wrist view.

hf download Dev-devadath/so101-chess-act-pick-wrist-v4 --local-dir policies/so101-chess-act-pick-wrist-v4
.venv/bin/python chess/play.py --policy policies/so101-chess-act-pick-wrist-v4 --ensemble --skill 3

Built with lerobot at commit 5c43fa1c (0.5.2 dev).

Downloads last month
38
Safetensors
Model size
51.7M params
Tensor type
F32
·
Video Preview
loading

Dataset used to train Dev-devadath/so101-chess-act-pick-wrist-v4