Gripper oscillation issue after full fine-tuning SmolVLA in LeRobot

#13
by captainsama - opened

Hi everyone,

I collected around 200 demonstrations of box grasping using VR teleoperation. All demonstrations start from the same position, and I have already verified that the replayed trajectories look correct.

I am training with SmolVLA in LeRobot using full fine-tuning. However, after training, I found that the robot gripper often behaves abnormally: it switches between opening and closing very frequently.

Does anyone have suggestions on how to debug this issue or improve the training/data setup?

Also, I have a question about the gripper state in the robot status input. For the gripper state, is it generally better to use a continuous value representing the real motor state of the gripper, or a binarized value such as 0/1?

In my current dataset, the gripper part of the action is binary, using 0/1. However, in the robot status, the gripper state is recorded as a continuous value. My thought was that this might help the model determine when the object has been firmly grasped, but I am not sure whether this mismatch could cause problems.

Any advice or experience would be greatly appreciated. Thanks!

Sign up or log in to comment