Upload policy weights, train config and readme

af41a77 verified 5 days ago

17.9 kB

datasets: lerobot/robomme
library_name: lerobot
license: apache-2.0
model_name: drifting
pipeline_tag: robotics
tags:
  - lerobot
  - robotics
  - drifting

Model Card for drifting

This is a drifting policy trained with LeRobot.

This policy has been trained and pushed to the Hub using LeRobot.

See the full LeRobot documentation.

Model Details

License: apache-2.0
Robot type: panda
Cameras: image, wrist_image

Inputs & Outputs

The policy consumes these observation features and produces these action features.

Inputs

Feature	Type	Shape
`observation.images.image`	VISUAL	`(3, 256, 256)`
`observation.images.wrist_image`	VISUAL	`(3, 256, 256)`
`observation.state`	STATE	`(8,)`

Outputs

Feature	Type	Shape
`action`	ACTION	`(8,)`

Training Dataset

Repository: lerobot/robomme
Episodes: 1600
Frames: 768897
Frame rate: 10 FPS
Task(s): "watch the video carefully, then use the stick attached to the robot to retrace the same pattern", "first press both buttons on the table, then pick up the container hiding the blue cube, finally pick up another container hiding the green cube", "first press both buttons on the table, then pick up the container hiding the red cube, finally pick up another container hiding the green cube", "first press both buttons on the table, then pick up the container hiding the green cube", "first press both buttons on the table, then pick up the container hiding the green cube, finally pick up another container hiding the blue cube", "first press both buttons on the table, then pick up the container hiding the red cube", "first press both buttons on the table, then pick up the container hiding the green cube, finally pick up another container hiding the red cube", "first press both buttons on the table, then pick up the container hiding the blue cube", "first press both buttons on the table, then pick up the container hiding the blue cube, finally pick up another container hiding the red cube", "first press both buttons on the table, then pick up the container hiding the red cube, finally pick up another container hiding the blue cube", "first press the button, then pick up the container hiding the red cube", "first press the button, then pick up the container hiding the green cube", "first press the button, then pick up the container hiding the red cube, finally pick up another container hiding the blue cube", "first press the button, then pick up the container hiding the blue cube", "first press the button, then pick up the container hiding the blue cube, finally pick up another container hiding the red cube", "first press the button, then pick up the container hiding the green cube, finally pick up another container hiding the red cube", "first press the button, then pick up the container hiding the green cube, finally pick up another container hiding the blue cube", "first press the button, then pick up the container hiding the blue cube, finally pick up another container hiding the green cube", "watch the video carefully, then place the red cube on the target right after the button was pressed", "watch the video carefully, then place the blue cube on the target right after the button was pressed", "watch the video carefully, then place the green cube on the target right before the button was pressed", "watch the video carefully, then place the green cube on the target right after the button was pressed", "watch the video carefully, then place the red cube on the target right before the button was pressed", "watch the video carefully, then place the blue cube on the target right before the button was pressed", "watch the video carefully, then pick up the container hiding the green cube", "watch the video carefully, then pick up the container hiding the blue cube", "watch the video carefully, then pick up the container hiding the blue cube, finally pick up another container hiding the red cube", "watch the video carefully, then pick up the container hiding the green cube, finally pick up another container hiding the red cube", "watch the video carefully, then pick up the container hiding the blue cube, finally pick up another container hiding the green cube", "watch the video carefully, then pick up the container hiding the green cube, finally pick up another container hiding the blue cube", "watch the video carefully, then pick up the container hiding the red cube, finally pick up another container hiding the green cube", "watch the video carefully, then pick up the container hiding the red cube", "watch the video carefully, then pick up the container hiding the red cube, finally pick up another container hiding the blue cube", "pick up the red cube and place it on the target, repeating this action two times, then press the button to stop", "pick up the blue cube and place it on the target, repeating this action three times, then press the button to stop", "pick up the green cube and place it on the target, repeating this action four times, then press the button to stop", "pick up the green cube and place it on the target, repeating this action two times, then press the button to stop", "pick up the green cube and place it on the target, repeating this action five times, then press the button to stop", "pick up the red cube and place it on the target, then press the button to stop", "pick up the green cube and place it on the target, repeating this action three times, then press the button to stop", "pick up the green cube and place it on the target, then press the button to stop", "pick up the blue cube and place it on the target, repeating this action four times, then press the button to stop", "pick up the red cube and place it on the target, repeating this action five times, then press the button to stop", "pick up the blue cube and place it on the target, repeating this action two times, then press the button to stop", "pick up the blue cube and place it on the target, repeating this action five times, then press the button to stop", "pick up the blue cube and place it on the target, then press the button to stop", "pick up the red cube and place it on the target, repeating this action three times, then press the button to stop", "pick up the red cube and place it on the target, repeating this action four times, then press the button to stop", "press the button to stop the cube just as it reaches the target for the fourth time", "press the button to stop the cube just as it reaches the target for the third time", "press the button to stop the cube just as it reaches the target for the second time", "press the button to stop the cube just as it reaches the target for the fifth time", "pick up the red cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion two times, finally press the button to stop", "pick up the green cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion three times, finally press the button to stop", "pick up the green cube, move it to the top of the right-side target, then put it down on the left-side target, finally press the button to stop", "pick up the red cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion three times, finally press the button to stop", "pick up the blue cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion two times, finally press the button to stop", "pick up the green cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion two times, finally press the button to stop", "pick up the blue cube, move it to the top of the right-side target, then move it to the top of the left-side target, repeating this back and forth motion three times, finally press the button to stop", "pick up the red cube, move it to the top of the right-side target, then put it down on the left-side target, finally press the button to stop", "pick up the blue cube, move it to the top of the right-side target, then put it down on the left-side target, finally press the button to stop", "first press the button, then pick up all cubes that have been highlighteted with white areas on the table", "watch the video carefully, then move the cube to the target in the same manner as before", "watch the video carefully, then grasp the same end of the same peg you've picked before and insert it into the same side of the box", "watch the video carefully, then use the stick attached to the robot to navigate around the sticks on the table, following the same path", "put two red cubes into the bin, then press the button to stop", "put one red cube into the bin, then press the button to stop", "put two red cubes and two green cubes into the bin, then press the button to stop", "put one blue cube and three green cubes into the bin, then press the button to stop", "put two blue cubes into the bin, then press the button to stop", "put three blue cubes into the bin, then press the button to stop", "put two red cubes and two blue cubes into the bin, then press the button to stop", "put three green cubes into the bin, then press the button to stop", "put two red cubes and one green cube into the bin, then press the button to stop", "put three red cubes into the bin, then press the button to stop", "put three red cubes and one blue cube into the bin, then press the button to stop", "put two green cubes into the bin, then press the button to stop", "put one blue cube into the bin, then press the button to stop", "put four green cubes into the bin, then press the button to stop", "put one red cube, two blue cubes and one green cube into the bin, then press the button to stop", "put three blue cubes and one green cube into the bin, then press the button to stop", "put two blue cubes and two green cubes into the bin, then press the button to stop", "put three blue cubes and two green cubes into the bin, then press the button to stop", "put three red cubes and one green cube into the bin, then press the button to stop", "put one red cube and two blue cubes into the bin, then press the button to stop", "put one red cube and one green cube into the bin, then press the button to stop", "put two red cubes and one blue cube into the bin, then press the button to stop", "put two blue cubes and three green cubes into the bin, then press the button to stop", "put four red cubes into the bin, then press the button to stop", "put one green cube into the bin, then press the button to stop", "put four blue cubes into the bin, then press the button to stop", "put one red cube and one blue cube into the bin, then press the button to stop", "put two red cubes and three green cubes into the bin, then press the button to stop", "put two red cubes, two blue cubes and one green cube into the bin, then press the button to stop", "put four blue cubes and one green cube into the bin, then press the button to stop", "put one red cube, three blue cubes and one green cube into the bin, then press the button to stop", "put one red cube and three green cubes into the bin, then press the button to stop", "put one red cube and three blue cubes into the bin, then press the button to stop", "put two red cubes, one blue cube and two green cubes into the bin, then press the button to stop", "put one red cube, one blue cube and one green cube into the bin, then press the button to stop", "put one blue cube and two green cubes into the bin, then press the button to stop", "watch the video carefully, then place the blue cube on the first target it was previously placed on", "watch the video carefully, then place the green cube on the fourth target it was previously placed on", "watch the video carefully, then place the red cube on the second target it was previously placed on", "watch the video carefully, then place the green cube on the third target it was previously placed on", "watch the video carefully, then place the blue cube on the third target it was previously placed on", "watch the video carefully, then place the green cube on the second target it was previously placed on", "watch the video carefully, then place the blue cube on the second target it was previously placed on", "watch the video carefully, then place the green cube on the first target it was previously placed on", "watch the video carefully, then place the red cube on the first target it was previously placed on", "watch the video carefully, then place the red cube on the third target it was previously placed on", "watch the video carefully, then place the blue cube on the fourth target it was previously placed on", "watch the video carefully, then place the red cube on the fourth target it was previously placed on", "watch the video carefully, then repeatedly pick up and put down the same block that was previously picked up for two times, finally put it down and press the button to stop", "watch the video carefully, then repeatedly pick up and put down the same block that was previously picked up for three times, finally put it down and press the button to stop", "watch the video carefully, then pick up the same block that was previously picked up again, finally put it down and press the button to stop"

Training Configuration

Setting	Value
Training steps	30000
Batch size	256
Optimizer	adamw
Learning rate	0.0001
Seed	1000
LeRobot version	0.6.1

How to Get Started with the Model

New to LeRobot? These guides cover the full workflow:

Install LeRobot — set up the lerobot package.
Hardware setup — assemble, wire, and calibrate your robot and cameras.
Record data & train a policy — the end-to-end imitation-learning walkthrough.
CLI cheat-sheet — quick reference for the lerobot-* commands.

The short version to run and train this policy:

Run the policy on your robot

lerobot-rollout \
  --strategy.type=base \
  --robot.type=panda \
  --robot.port=<your_robot_port> \
  --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
  --policy.path=Xihe666/drifting_robomme \
  --task="watch the video carefully, then use the stick attached to the robot to retrace the same pattern" \
  --duration=60

Replace the remaining <...> placeholders with your own values: --robot.port and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.

When --strategy.type=base is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at rollout documentation.

Train your own policy

lerobot-train \
  --dataset.repo_id=${HF_USER}/<dataset> \
  --policy.type=drifting \
  --output_dir=outputs/train/<policy_repo_id> \
  --job_name=lerobot_training \
  --policy.device=cuda \
  --policy.repo_id=${HF_USER}/<policy_repo_id> \
  --wandb.enable=true

Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.

Evaluation

No evaluation results have been provided for this policy yet.

Citation

If you use this policy, please cite the method linked in the description above, along with LeRobot:

@misc{cadene2024lerobot,
    author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
    title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
    howpublished = "\url{https://github.com/huggingface/lerobot}",
    year = {2024}
}